Rodrigo Flores's Corner

What is expected of a Engineering Manager?

2020-10-14T00:00:00-03:00

tl;dr: the three things I expect from an engineering manager are:

Support the members of your team and help them grow.
Follow along the deliveries, setting quality standards, making sure the team has the support they need and upper management the feedback they need.
Keep a constant practice of creating, improving or eliminating team or company processes.

Every organization has its own way of working and your mileage may vary. I’ve collected a few points that I believe that are overall expectations.

Andy Grove on his book High Output Management defined a manager output as the sum of their team output and the teams under their influence output. He also recommends selecting activities producing high leverage: an activity he did as a CEO was an introductory talk to new employees, in which he explained the history, the values and the objectives for Intel. If he does this very well, this can generate great impact on every new employee.

A good example of a high leverage is partnering with other leads: being able to understand where the business is going and helping building a technical roadmap to facilitate getting there. Another high leverage is to guide each member of your team to acquire new skills so they can increase their scope and increase their impact on the organization.

People

Your team’s output is really dependent on your team members, so an expectation for engineering managers is to support the members of your team and help them grow.

A good way to think about hiring and growing your team members is to think that they can replace you. You can’t actually do that unless you know everyone really well. What are their strenghts and weaknesses? What are their goals and what motivates them? Which level they’re operating? What might need them to go to the next one?

1:1s play a big role here: it is your time to learn more about each person. It is a great time to establishes trust and build rapport. Another crucial factor on people is to set expectations: setting a clear expectation on what you expect from each person creates a more concrete notion on what they should and shouldn’t do and what level they should perform: recognizing when someone performed really well and giving constructive feedback when they haven’t reached your expectations. Feedback is a hard subject, but if you can back your feedbacks with examples, it will be much more clear on what you’re recognizing or you’re giving a constructive feedback: a good practice is to give continuous (a practice suggested by Camille Fournier on her book The Manager’s Path: A Guide for Tech Leaders Navigating Growth and Changefeedback: on every 1:1 try to bring an example-backed feedback.

If you find yourself giving constructive feedback but you’re not seeing improvements, make it clear as early as possible. Dealing with underperformance isn’t an easy task, and sometimes we hope that with guidance they will improve, but making it clear that it isn’t working is part of your job and perhaps it is time to set more prescriptive actions and a timeline for improvement: a tool that can help is Personal Improvement Plan in which you set a clear expectations, actions that you think will help them you and an expected timeline.

A hard part of this job is that if someone isn’t performing even after this, you have to let them go. There is also behavior: if someone had an unacceptable behavior, it is part of your job to take action. Dealing with underperformance/letting people go is a tough subject (which I don’t have much experience): I highly recommend asking your HR Business Partner and your manager to help with that.

Your company will also count on you on performance reviews, which happens a few times a year and you can discuss with each person their actual performance along with career progression: if they’re doing really well, you can promote/make a case for their promotion to your peers and senior management. You can also use this time to set goals on what people need to improve and learn for the next cycle. Some companies use this period to set performance improvement plans for people below expectations. Performance reviews usually generate anxiety for everyone: if you’re working on giving continuous feedback every 1:1, it should be an easier conversation without surprises.

As a final recommendation on people expectations, I’d highly recommend learning more about delegation (without micromanaging): allowing people to have autonomy on how they should do a task will allow them to grow and also you can focus on exerting higher leverage. You can’t control every action and if there is a mistake, you have to guide people on learning from it: Ray Dahlio on his book Principles: Life and Work enforces that you should have a culture in which is ok to make mistakes and unacceptable to not learn from them. Google also researched a lot about psychological safety and they found out that in their best teams people feel safe taking risks.

Delivery

Another expectation on an engineering management job is delivery: your company expects that you follow along the deliveries, setting quality standards, making sure the team has the support they need and upper management the feedback they need.

This is where delegation plays its role: to exert leverage, you have to delegate the execution of a project to the team: delegating something doesn’t mean you can forget about that delivery, you need to make sure it is done (or explain why it isn’t) and support that delivery.

Andy Grove tells about 4 types of activities that a manager does: Acquire Information, Communicate Information, Decision Making and Nudging. When support delivering value, we perform these activities: we acquire information about what we’re delivering, we communicate back and forth, we nudge the team on decisions and when the decision has bigger trade-offs or when the team can’t make a decision, we need to make the decision based on the acquired information and communicate our reasoning to everyone else. If a decision is controversial, make it clear and explicit the controversial points and the trade-offs that were considered.

A recurrent is waffling a decision: when the team needs to make a decision and the discussions about it can’t converge: if a decision is a roadblock, it is your duty to understand the tradeoffs, discuss with all relevant parties and make a decision. If it seems complicated, it is because it is.

Communication is probably your most important tool on delivering: acquiring information, partner on generating a purpose along with your peers and communicate the problem being solved to the team and communicate the progress to everyone interested.

When a decision doesn’t seem big and only affects the team, I recommend delegating it to the team and discuss the trade-offs: if prefer one of the options but the other ones doesn’t have significant downsides, nudging is a good idea: telling what path you think it is best and why, but remember: your word carries a lot of weight and people usually expect that a manager makes all decisions.

Another important job is on setting a vision, a mission and objectives: a vision is a long-term objective something that with you expect to reach in 5 year, a mission is a short-term (6 months) and objectives are the guard-rails that will ensure you’re on path to the vision: OKRs is a very famous methodology to help with that.

Process

When teams and companies become bigger it is inevitable that rules, policies and do’s and don’ts start being created. As a leader for your team, it is expected that you keep a constant practice of creating, improving or eliminating team or company processes.

Since the agile manifesto with the tenet People and interactions over processes and tools, processes got a bad reputation: specially the ones more bureaucratic. But the process itself isn’t the problem, the problem is putting it above people and interactions.

I tend to say that the best process is a self regulating one, but I’m aware that it won’t be enough. Self-regulating a team backlog with a 2-3 person team might work, but for a 5-6 person team a regular weekly meeting to discuss and an organized kanban board can be valuable tools.

I tend to start with low or none processes on a new team and when a problem arises, I try to nudge into a process and make a working agreement that we’re going to experiment. With experience, it is likely we will find patterns and solutions we’ve seen elsewhere, but as every team and every context is different, rarely a process took from other team will fit perfectly into a new one.

And if it does not work, you probably need to re-discuss: it is not hard to become the rules cop and start complaining that the people does not follow a process. Instead, do a retrospective and learn what works and what not.

There might be a case in which a process comes from other team and your team is not happy. Your duty is to collect the pain points and bring the concerns to the other team. You actually don’t need to solve everything (and you can’t) but passing along the feedback and listening to the other team concerns as well helps.

Conclusion

I wrote this article as a first reading on someone interested in becoming a manager: I hope that helps you to understand. There are a lot of books and studies about these topics and you will find that improving your skills on them will significantly improve the quality of the work as a manager.

A common path that some companies take is to pick the most senior person on a team and promote them to manager but it takes a lot more than just technical knowledge to be a good manager: being able to communicate well and partner with other functions are other skills that help a lot.

To Learn More

I highly recommend Camille Fournier book The Manager’s Path: A Guide for Tech Leaders Navigating Growth and Change: it gives a comprehensible overview of each step on the management ladder.

A fantastic book about people management and specially feedback is Radical Candor: it tells not only the importance of caring about people but also work on challenge them.

Book Review: Deep Work

2017-05-22T22:12:00-03:00

When you work, work hard. When you’re done, be done.

I’ve read a lot of books about how to increase your productivity and I’ve tried a few techniques, but I think this book goes deeper than any other one I’ve read: it tells about the importance of Deep Work and how to work deeply in an era of instant messaging, open offices and constant push notifications on our phones.

Why is that so important ? If you’re like me and work with technology it is not unusual to discover a new technique/language/stack/buzzword envery now and then, and with every new trend comes learning. It is also expected that you perform on our job at an elite level in terms of both quality and speed. However, our latest trends on our industry are Instant Messaging, Open Offices while we seem to be connected all the time (and proactively receiving push notifications on our phones) to a handful of social networks, which isn’t helping that you thrive. This is making people that learns fast and delivers the best work so well valued on our industry.

The book starts with three chapters making the case for Deep Work, why it matters, how it changed the author’s life, and then it goes to how to work deeper with 4 rules (each one described on its own chapter).

The first rule is Work Deeply this seems the whole goal, but there are various pitfalls: how will your willpower sustain your deep work ? How to deal with shallow commitments such as answering to e-mail or going to meetings ? This chapter explain some strategies and how someone can incorporate deep work in his/her daily work (specially if your occupation requires non-deep work as well).

Embrace Boredom is the second rule: it talks about the internet as an agent of distraction, and if you’re distracted all the time when you’re not working, you will also distract when you’re not bored: when you’re bored, do you usually pick up your phone and access a social network or play a game ? If so, you can’t cultivate focus time. As the book quotes:

Don’t take break from distraction, take break from focus,

so cultivate some time off (specially taking internet-breaks).

Following, the third chapter talks about another agent of distraction: Quit Social Media: time is a limited resource, and when you think about how much time do you spend checking your Facebook or twitter feed ? What I really liked about this is that the author presents the any benefit mindset: you think that because of any benefit you might get, you keep using it, without considering the drawbacks. However, if you coldly analyze, and see if it is helping your not your biggest life goals, you might see that the social networks might be costing you relevant time. Despite the dramatic chapter title, the author asks you to evaluate the social networks you’re using and see if it indeed worth it as mileages may vary and it might be useful for you.

Finally, the last rule is about time management, it is called Drain the shallow: I really liked this chapter: the author makes a point on why you should be more thoughtful with your time: you might be wasting it with something and not be aware of that, so he suggests that you keep a notebook (I’m using a calendar) and put a use to every minute of your day, including scheduling shallow tasks. He also recommends a fixed schedule technique: you should finish your workday by a specific time every day, to make time a more valuable resource (as deep work is limited and if you try to squeeze more you won’t ble) and enforcing you to make decisions on what to commit and to use it wisely and to restore your energy for the following day.

One thing that I’ve liked is that the author studied several knowledge workers like Carl Jung, Ted Roosevelt, Donald Knuth, Neal Stephenson, Bill Gates among others and how deep work made such a big difference on their lives. He also does not present it in a dogmatic way, but he rather gives the whys (and quotes academical works when doing so) such techniques provide great results and some real-life examples of each rule.

I’d recommend this book for everyone that does some kind of knowledge work: it might give you some insights on how to increase your skills. It is a great reading for those interested on how to work better, and how to boost your creative power to achieve your full-potential.

Things I've learned in 2016

2017-01-09T07:40:00-02:00

As we have reached 2017, I think it is time to remember some things I’ve learned on 2016. All links and information on this blog post are my sincere opining here does not generate any form of revenue nor any other advantage to me.

Getting deeper in Clojure

One of the things I noted last year was that I lacked knowledge of some advanced concepts in Clojure. The JVM-powered LISP may be threatening to tackle, but once you get the basics, you find yourself productive. But after getting the basics, how do you organize your code ? How do you deal with mutable parts (like databases) on your code ? How to use Clojure async library ? When to use lists and when to use vectors ? And sometimes, you will need to write or debug macros, how to understand them better ? Even macros being sometimes described as dangerous, you’ll need them at least to understand. So two books I’ve read about Clojure were “Mastering Clojure Macros” and “Clojure Applied” by Alex Miller (and I even wrote a blog post reviewing this last one), so if you think that you’re ok with the basics (I have a blog post about dealing with the basics too) and want to dive deeper on building applications in Clojure, I think these two are great books for this purpose.

Developing some missing technical skills

Even after 6 years working as a Software Engineer/Developer, I think I miss some important skills like performance analysis. I had to do some kind of analysis on this area this area and found out that I’ve been using the Streetlight anti-method too much, and a colleague suggested me reading the book Systems Performance by Brendan Gregg. It teaches you some methodologies and some ideas on how to investigate different resources on a computer: application level, CPU, memory, disk and network. It shows how to not fall into pitfalls like only using top to check on a machine saturation.

It also tells about the USE Method, a technique to check for Utilization, Saturation and Errors for a resource to find performance problems.

Systems Performance

Business-ish Books

I’ve took some time to ~~read~~ listen to some books (I still find weird to say that I read I book that I actually listened).

One I really liked was Delivering Happiness, by Zappos former CEO Tony Hsieh . It shows how he build Zappos, how he got some suppliers to accept being sold online and how he build the company culture. One of the coolest things about Zappos is that they publish every year a culture book with quotes from their employee that aren’t edited (except for grammatical correction). I strongly believe that a company culture isn’t on a wall, an internal website or a leaflet that you deliver to new employees, but on actions and words that employees, managers and directors take everyday. I think this was a very bold move to ensure that you’re keeping the culture.

Another book that I really liked was Creativity Inc. Written by Pixel executive Ed Catmull (which I had the pleasure to “meet” on Steve Jobs Biography book that I ~~read~~ listened in 2016), it tells how the creative process works there, how they discuss ideas and how they take risks. I really liked how they tackle mistakes: they understand that they’re part of doing something new, and they’re also aware that ideas aren’t good at first, but after an iterative process, candid feedback and rework a flawed story (or idea) finds its line.

The Minecraft book and Power of Habit were also two other books that I had the pleasure to read this year. The first one tells the story of Marcus “Notch” Persson, and how he build Minecraft. The second book tells why habits exist and how they can be changed: from Martin Luther King and the Civil Rights Movement, Target stores and how they predict purchasing habits, Alcoa CEO and Safety Concerns and how a NFL head coach made a unpromising franchise one of top teams in the league.

I think this book made me more curious to learn more about Psychology and how the brain works, but this is a subject for the next topic.

Personal Development books

I didn’t write a post like this last year, but one of the books that I’ve ~~read~~ listened last year that I most enjoyed was Soft Skills by John Sonmez and one of the tips he gave is on how to improve yourself not just as a programmer/engineer/developer but as a person. And after reading that I discovered that I had too much to learn indeed.

One of the books recommended was a really classical one: “How to win friends and influence people” by Dale Carnegie. The book starts telling you how to read it. It tells you to read every chapter twice and for you to explain what you have learned to someone else and ask them to monitor how you are practicing what you’ve learned. One of the great teaching I got from the book was “Be hearty in your approbation and be lavish in your praise”, it means that you should use your heart you complement something. But when you have to do some criticism, do it when necessary and be mindful to consider the other side reasons, plus make them see what is wrong instead of just enforcing it.

Another book I’ve read and liked was “Pragmatic Thinking and Learning”: it shows you how your brain works, hence how to learn, how to tackle problems and a different way to study. I think skill acquisition is a important skill: by reading this book you discover that reading books cover-to-cover can be a naive approach on skill acquisition.

Generic Trivia

I also like to read fun books: I really liked Periodic Tales by, a book that tells you the history and how some elements were discovered and used througout the ages. I’m a great fan of science books and I like to learn about science trivia.

Periodic Tales

Learning new things

In 2015 I started learning how to use Emacs and in 2016 I kept learning more about it: I followed Mike Zamansky channel on youtube which tells you some good tips about packages and techniques on Emacs. I also chose org-mode as my main tool for taking notes while reading.

Another library that I’ve learned how to use and its powers was Finagle. Built by Twitter, it intends to be an extensible RPC system for the JVM. It implements several protocols like HTTP, Thrift, MySQL, Redis and has several modules that implement some resilience strategies like circuit breaking, client side load-balancing, budgeting and retrying, that you can use with any of the protocols. One of the great advantages of such approach is that it can give a small rest to a troubled part of your system (with exponential retrials) so it can recover. The retry may also help when only a node is down but others are up (and a load balancers on server side does the round robin) and the budgeting may avoid you retrying too much (and for the server side to recover. It is a must-have piece when you have some relevant load on your applications and you have a multi-service architecture.

Conclusion

2016 was a great year on many aspects: I was able to apply many of things that I’ve learned here on my daily routine, I’ve also deepened some concepts on Clojure and Emacs. Hope in 2017 that I blog more about the things I learned more consistently.

Feel free to recommend any book or article on this subject and a happy 2017!

New Project: A newsletter of cool stuff

2016-06-04T22:19:00-03:00

If you follow me on twitter, you probably notice that I usually post links of things that I find interesting, along with possible rants and cat/dog pictures. However I think twitter is a nice place to post things, but I still think it doesn’t get too much attention: a twitter feed is an stream of interesting stuff/rants/anything else that I usually don’t bother to keep something like an inbox zero approach.

One thing however that I try to read it all are e-mails besides personal e-mail, I try only to subscribe to interesting newsletter. So, I decided to create a newsletter: I’m still not sure to whether I should post or not but most necessarily trivia which is non-coding stuff: as I still have this blog for this purpose.

If you want to subscribe here is the link: https://www.getrevue.co/profile/rlmflores.

Book Review: Clojure Applied

2016-02-25T07:03:00-03:00

The following blog post is about the book Clojure Applied written by Ben Vandgrift and Alex Miller. I have not received a free copy of the book to review it.

As Clojure is becoming more and more popular these days, we have a lot of programmers learning to write code in Clojure, and almost all books tell you how each part works: functions, macros, maps, lists, records, multi-methods, agents, refs, transducers, reducers. But how can you apply all these concepts together ?

I like to think of Clojure like a infinite Lego box: you can build your system using the pieces, but can you make smart choices to select the best tool for each part of your system ? Some parts appear to be similar: lists and vectors, maps and records, multi-methods and protocols, but how do I select one instead of another ?

Clojure Applied is the first Clojure Book I read which wasn’t targeted for people initiating in the Clojure world, so it considers that you have some experience. I like to think of it like a book of answers to questions that if you deal with Clojure frequently you will probably wonder about. I particularly liked the chapter 6 “Creating Components”: on how to structure your application in an organized way.

Another cool thing is that it not only covers features of the language but also some well-known parts of the ecosystem like the core async library, Stuart Sierra’s component library, the schema library and transit and EDN as data serializers. It also covers how to deploy your code to a production server using a platform, a IAAS or using your own servers.

I strongly recommend you to not read this book cover to cover, but to try to write a small application after each chapter applying what you have learned: I did it this way and I really think it was worth it.

A Clojure application generator with Midje

2016-02-01T22:00:00-02:00

If you’re bootstraping a new Clojure application, you would run this command:

lein new app my-awesome-app

And then Leiningen would generate a directory containing the bare minimum to make your application build (with a Hello World example). However, everytime I did that for experimenting purposes, I found that I’ve always added two libraries: Midje and Schema, so to help me stop doing this repetitive work, I created a template (aka generator) for that. Doing this, I can bootstrap an application with both libraries by just running this command:

lein new app-with-midje my-awesome-app

Midje

Midje is a testing framework for Clojure. I like the way tests are written in Midje:

(facts "average of items
  (fact "average of 1 item is itself"
    (average 1) => 1)
  (fact "average of 2 items is the minor item plus half the distance"
    (average 1 5) => 3)
  (tabular
    (fact "calculates average"
      (average ?a ?b ?c) => ?average)
    ?a ?b ?c ?average
    1  9  11 7
    3  12 30 15
    30 60 90 60))

They’re very similar to what you would describe as input and output examples of a function. Also, I think tabular examples shows clearly what you would expect on each case, without having to duplicate code on each fact. It is also possible to mock and redef other functions on a test scope, so you can create isolated unit tests, and it is also possible to build your own checks, so you can create better tests based on your domain. So, in general, I consider Midje to be a great testing library to work with.

Schema

Clojure is not a typed language, so it is fairly common to see maps being used as a kind of typed data. Suppose you have this map describing a person:

(def person {:name "Joe Doe"
             :age  45
             :team "Blackburn Rovers"})

(def another-person {:name "Jane Doe"
                     :age  48})

For some reason, you want to give Joe and Jane a football jersey of their favourite football team. However, you forgot that this field may be not filled, as it is not mandatory for everyone to have a football team, so you wrote your function like this:

(defn ship-football-shirt [customer]
  (ship-shirt customer (buy-shirt (:team customer)))

As you forgot that it is a required, when you call the function with Joe, it works because he has a football team, but when you for Jane, it gives you a NullPointerException (Clojure normally raises a NPE you treat a nil like a map). How to get over this kind of issue ?

To help deal with this kind of problem, there is a library called Schema. After specifying it on your project.clj, you can instantiate on your namespaces and use this way

(require '[schema.core :as s])

(def teams #{"Blackburn Rovers" "Leicester United" ... } )
(def all-teams-ever (s/enum teams)) ; I'm not writing the name of all teams ever


(def Person {(s/required-key :name) s/Str
             (s/required-key :age)  s/Int
             :team                  all-teams-ever})

(s/validate Person person)

To declare a map schema, you should insert the keys and the accepted values for each key. Optionally, you can say that a key is required. If you call the s/validate function with a schema and a map, it will try to validate the type of each value and also the presence of all required keys, throwing an exception in case something doesn’t validate. On our example, it requires a person to have a :name key with any string value, an :age key with any integer value and optionally a team with any value specified in the set teams.

To help fix our problem, we can create a derived schema call FootballFan:

(def FootballFan (assoc (dissoc Person :team) (s/required-key :team) all-teams-ever))

And then you can rewrite ship-football-shirt this way:

(s/defn ship-football-shirt [customer :- FootballFan]
  (ship-shirt customer (buy-shirt (:team customer)))

The :- symbol means that the customer symbol should be validated with that schema. When you call the function above inside the body of the macro (s/with-fn-validation ) it will trigger an exception. Why is this function validation optional ? Performance reasons mostly: makes sense to have this check turned on throughout your code on a testing environment but on production it might spend valuable time checking schemas (but a validate call on strategic places like before inserting something to a database or when you’re receiving data from and HTTP request definitely makes sense).

Will it avoid receiving an exception ? Absolutely not, you will still get an exception, but this time you will receive a specific error telling you that Jane is missing that key/value. This kind of validation avoid not only strange errors, but you also don’t need to program defensively inside ship-football-shirt (as you specified that you only accept maps that validate with that schema).

The App with Midje template

As I use both libraries on almost every application, I created a template for that: a set of files and folders generated given a project name. So, after adding it to my lein profile, I can call lein new app-with-midje awesome-project and it will generate a project.clj with both libraries, an APACHE V2 license,a README, a .gitignore file, a core.clj file to write the application (and a correspondent test file), a repl.clj file that serves as a REPL wrapper to load libraries and functions (like an the autotesting namespace) for general REPL there.

To use the generator, add on your ~/.lein/profiles.clj the plugin and the version (in case you already have the plugin vector, just append it):

{:user {:plugins [[app-with-midje/lein-template "0.1.0"]]}}

And then, to generate the application:

lein new app-with-midje my-awesome-app

You can also check the code for the template on Github.

Clojure Ecosystem

2015-12-31T15:53:00-02:00

One of the things I dislike the most when learning a Programming Language is how frequent you get stuck with something. When dealing with something different than what we’re used to deal with every day we’re pushed out of our confort zone, which is good, but unconfortable. This blog post will give you a few tips on tools that usually help me when I’m writing Clojure code.

REPL

REPL is an acronym for Read Eval Print Loop: which means a prompt where you send expressions and it prints the result of that expression. Rubysts are probably familiar with irb or pry and Pythonists are familiar with Python interactive interpreter that comes bundled with the language (just run python without arguments to see it).

To run it, execute lein repl on a Leiningen project directory and you shall see an output like this one:

nREPL server started on port 57755 on host 127.0.0.1 - nrepl://127.0.0.1:57755
REPL-y 0.3.7, nREPL 0.2.10
Clojure 1.7.0
Java HotSpot(TM) 64-Bit Server VM 1.8.0_65-b17
    Docs: (doc function-name-here)
          (find-doc "part-of-name-here")
  Source: (source function-name-here)
 Javadoc: (javadoc java-object-or-class-here)
    Exit: Control+D or (exit) or (quit)
 Results: Stored in vars *1, *2, *3, an exception in *e

test.core=>

Just like a shell, it is waiting for your command: you can type:

(+ 2 3)

And it will simply output:

And then it will wait for yout next command. So it read the expression (+ 2 3), evaluated the expression to 5 and then output the result: 5.

An useful function is called doc, it will print the docs for a function:

test.core=> (doc +)
-------------------------
clojure.core/+
([] [x] [x y] [x y & more])
  Returns the sum of nums. (+) returns 0. Does not auto-promote
  longs, will throw on overflow. See also: +'
nil

What happened here? doc is a function with a side-effect: it prints the documentation for a given function. When you call it with + it prints the documentation for the + function and outputs the return value for the function which is nil.

Docs

As you have noticed, the first way to find docs about a function is through the doc command. It is a succint description, going straight to the point on what it does along with the different arities (an arity is a possible number of arguments that a function accepts, naturally you can sum 0 up to infinite number of integers, but a function may only allow a limited number of arities, like the doc function that only accepts one arity). However, sometimes you don’t remember the function name, or an usage example could be useful, on these cases you can use other alternatives.

On Clojure.org there is an online version of what the doc function prints. It may be more visually attactive as the arities are on different lines, but it is essentially the same thing. Clojure.org also provides a really useful cheatsheet, that may also be useful as it provides a quick reference for the most common used functions. If you have already played with another language, you will be familiar with the meaning of most of the functions here. Each entry links to the documentation of that function.

Clojure docs is similar to the documentation on the Clojure website, but also contains examples: you can check here how the arities work: you can call + with an arbitrary number of arguments and it will work.

There is also Grimoire, that is similar to clojure docs, but also contains a link to the function source.

Learning

Books

I had my first contact with Clojure through Bruce Tate’s book: Seven Languages in Seven Weeks. It contained a short chapter (to be read and explored within a week) about Clojure. I really liked the reading, but it couldn’t of course go deeply into Clojure. So, my first book to learn Clojure was Stuart Halloway’s and Aaron Bedra Programming Clojure: a really nice book that teaches you not only the Clojure language, but also the Clojure Way. And speaking about Clojure Way, my next reading was Michael Fogus Joy of Clojure a book maybe more targeted to someone with already some knowledge on the language as it goes deeply into concepts and more subtle things.

There is also good options that were recommended by some friends: Clojure for the Brave and the True, Clojure Programming, Clojure Programming and Living Clojure: this last one deserves a special highlight, it contains a training guide to learn Clojure (similar to a Running program like Couch to 5k but for Clojure).

Online

There are a few websites or projects to learn Clojure online. Back on my first days I tried Clojure Koans and I really liked: is some kind of filling the gaps exercise to make some expressions to be true. Now I see that there are other cool online resources: Try Clj: to try Clojure with an interactive tutorial of the language and 4Clojure with a similar experience compared to Clojure Koans but you do both without even installing anything on your computer.

Lastly, there is Wonderland Clojure Katas, a series of proposed puzzles to be solved with Clojure: the exercises come with a failing test suite and you should build code to make it pass.

Testing

After some years of Ruby, I discovered that the best thing you can do with your code is to write tests for it. It works like a safety net: no matter what you do with your code, if the tests are passing you can be sure that it behaves the way it should (or at least the way it was before the changes). And as learning a programming language requires experimentation, testing is a fast way to see if that new thing works the same way than that not-so idiomatic thing you did before.

The Clojure language itself has a test framework called Clojure Test. It comes bundled with the language, so the only thing you need to do is require the namespace and write your tests. To try Clojure Test, you can do this on a REPL:

(require '[clojure.test :refer [deftest is run-all-tests]])

(deftest one-is-one (is (= 1 1)))
(deftest two-is-one (is (= 2 1)))

(run-all-tests)

The first line requires the test namespace and the :refer [deftest ...] part is just a way to avoid having to call the namespace (or an alias) everytime you call something of that namespace. Then we define the tests: the deftest registers your test expressions as tests and run-all-tests looks up for this definitions and calls it. To assert the value of something you can use the is macro (for now, think on a macro as a special type of function that gets evaluated before the code is run: we will get on macros soon): it will show a summary of the test run and also output all failed tests (the falsy expressions).

FAIL in (two-is-one) (form-init7127606434032677106.clj:1)
expected: (= 2 1)
  actual: (not (= 2 1))

Ran 2 tests containing 2 assertions.
1 failures, 0 errors.
{:test 2, :pass 1, :fail 1, :error 0, :type :summary}

Another popular testing framework is Midje. It has a quite simple syntax fn-call => expected result. After setting up midje (as it isn’t bundled in the language you will need to include it on your project), you can write tests this way:

(facts "sum function"
       (fact "one + one is two"
             (+ 1 1) => 2)
       (fact "one + two is four"
             (+ 1 2) => 4))

And the error display is really nice (and colourful):

> (require '[midje.repl :refer :all])
> (autotest)
======================================================================
Loading (test-app-midje.core test-app-midje.core-test)

FAIL "sum function - one + two is four" at (core_test.clj:9)
    Expected: 4
    Actual: 3
FAILURE: 1 check failed.  (But 1 succeeded.)

There is also plenty of other cool features: mocking, loose checkers, exception thrower checker and even the possibility to write custom checking functions. The project wiki on Github also contains some documentation on how to get started with Midje.

Editors & IDEs

As Clojure is becoming more popular each day, I’m pretty sure that almost all popular editors and IDEs may have support for Clojure syntax (or at least a plugin for it). I’ve already written Clojure code in Vim, Lighttable, Emacs, Sublime Text, Atom and IntelliJ with Cursive. I also know that Eclipse also have a plugin for Clojure called Counterclockwise.

But as being a dialect of LISP, you may need more things on an editor/IDE than simply syntax highlighting and REPL integration: you will need a smarter way to deal with open and close parenthesis/brackets/quotes. One of the things that make me more productive in Clojure, is the ability to deal with open and close parenthesis in a more structured way that is called Paredit or Structural Editing. This feature automatically adds a closing parenthesis/bracket/quote when you add a opening parenthesis/bracket/quote. Also, it has a few more commands to manipulate those expresions in order to be able to edit your code more efficiently. You can see a few examples of commands on this blog post.

Of course, this is not mandatory: you can write Clojure without using this techniquebut if you want to be really productive with Clojure, learning this technique is important. But feel free to skip it for now: I would only ask one thing: try to see if it exists on your favourite editor: if it does, that’s fantastic, if it does not, another editor might be a good choice for your Clojure adventure.

Libraries

The best place to find a library is Clojars: it is an equivalente of RubyGems/CPAN for Clojure: there is a search where you can search for the library name or for a term like http, soap or aws, and it will return all the libraries on clojars that match that term. Also, due to Clojure interoperabilty with Java, you can also include and use libraries from Maven.

After finding the library, you can add it to your project through the project.clj dependencies (I explained how to do so on First Steps with Clojure post.

Community

Just like all programming language communities, the Clojure community has some mailing lists, a Planet, an IRC channel and some wikis. You can check a list of these resources on the community page of the Clojure website. Another source for Clojure news and discussions is the Clojure Subreddit where people usually post links, ask questions and discuss. From time to time, there is a thread called “Ask Anything”, where new Clojurists can ask any questions.

Videos

You can also check some Video resources:

Clojure TV: contains plenty of talks of Clojure conferences;
Brian Will’s Series: is a series of videos that teaches you Clojure;
InfoQ: Clojure content on InfoQ;

Summary

The goal of this blog post is to talk about some useful resources when learning Clojure. So, I hope you found your way through the REPL, was able to write tests, found a good editor, learned where to search for libraries and where to ask questions and discuss about the language.

Next blog post I will go deeper into Leiningen: the idea is to create a boilerplate for Clojure apps.

First steps in Clojure

2015-11-16T21:36:00-02:00

I remember the first time I had to deal with Clojure. I just came back after a job interview and I was asked to deliver a solution on a functional language to an exercise. Naturally, I opened a terminal and typed:

$ brew install clojure

brew is the package manager that install open source stuff on OS X. But instead of installing Clojure, homebrew returned this error:

Error: No available formula with the name "clojure"
Clojure isn't really a program but a library managed as part of a
project and Leiningen is the user interface to that library.

To install Clojure you should install Leiningen:
  brew install leiningen
and then follow the tutorial:
  https://github.com/technomancy/leiningen/blob/stable/doc/TUTORIAL.md

Unlike Ruby or Python, Clojure does not have a native compiler for OSX or Linux: it runs on the Java Virtual Machine (also called JVM). So, you don’t need to install a compiler, you just need to install Java, download a JAR and you should be ready to run Clojure.

Running Clojure from the JAR

But wait, the error said that I need to install Leiningen? Yes, we will get on Leiningen on a moment. But, for now let’s run Clojure directly from a JAR. You can download the most recent version of the Clojure JAR from this website: http://clojure.org/downloads and unzip it. Then, you can run on a terminal:

java -cp clojure-1.7.0.jar clojure.main

This -cp option allows you to specify a list of directories, JAR archives and ZIP archives to the Java application launcher, the other argument clojure.main will be the namespace whose function -main will be called. You can also run a Clojure file like a script by passing the Clojure source file as another argument. To make it happen, I’m adding this content to a file called hello_world.clj:

(println "Hello world")

And I will run:

java -cp clojure-1.7.0.jar clojure.main hello.clj

And now you have run your first Clojure program. It is a simple program, the only dependency is on Clojure language. What if I want to write a program that needs to do an HTTP request? HTTP Kit is probably the most popular HTTP client in Clojure (and also can be used as a Server). Let’s write a code to do a HTTP request:

(require '[org.httpkit.client :as http])
(println @(http/get "https://api.github.com/users/octocat/orgs"))

For now, no need to focus on the code, just trust me that it will work: I will explain what is this @ later. To run it, you will need something more: you will need to specify the http-kit JAR.

java -cp clojure-1.7.0.jar:http-kit-2.1.19.jar clojure.main hello.clj

And voilá, it should work.

Leiningen

Definitely this is not a straightforward way to run code. You will need to manually download JARs, and remember to insert all of them on your command to run your program. Happily, Leiningen can help us to make it really easy to bootstrap a Clojure application, to manage the dependency and to create different profiles (think of a profile as a “different” way to run your application). If you’re familiar with Ruby on Rails, it does basically what Bundler (dependency management), Rake (make-like tool to run different profiles of your application) and Thor (code bootstrap generator). It also compiles your Clojure code into JVM bytecode and also can create a JAR file for you to export it. If you wabt to a comprehensive explanation on everything that Leiningen does, you can check this excellent article about it.

So, to bootstrap a Clojure application with Leiningen you first need to install Leinigen: you can do so by downloading a script, through a package manager or if you’re a M$ Windows user, you can use a installer.

After installing it, you should be able to run

lein new app blog-example

lein is the command to invoke Leinigen, new tells Leinigen to bootstrap a new project, app is the template (this argument is optional and it defaults to a library project) and finally blog-example is the name of the application we’re bootstrapping.

It should have generated something similar to this:

CHANGELOG.md	README.md	project.clj	src
LICENSE		doc		resources	test

./doc:
intro.md

./resources:

./src:
blog_example

./src/blog_example:
core.clj

./test:
blog_example

./test/blog_example:
core_test.clj

To build our hello world application, let’s for now focus on two files project.clj and src/blog_example/core.clj. project.clj is the project declaration: it contains the configuration that will be used to run your project, the name, description, the license information, etc.

(defproject blog-example "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.7.0"]]
  :main ^:skip-aot blog-example.core
  :target-path "target/%s"
  :profiles {:uberjar {:aot :all}})

Let’s focus on two things on this file: :dependencies and :main: the first one is a vector (for now, think of vectors as the same as what is called as an array: I will explain more about sequences later) of all the dependencies along with the versions, and :main contains the namespace that will be run when you run lein run: the function that will actually run is the one called -main. Let’s take a look at the file core.clj in the directory src/blog_example:

(ns blog-example.core
  (:gen-class))

(defn -main
  "I don't do a whole lot ... yet."
  [& args]
  (println "Hello, World!"))

The -main only contains a println call that outputs "Hello World", and this function will be called when you call lein run.

The coolest thing about Leiningen is that you don’t even need to install Clojure to run the project: just install the java, Leiningen and the first time you run the program, it will install all the dependencies (including Clojure). Also, you don’t even need Leiningen to run your program: you can easily pack it with lein uberjar and run with java -jar jarfile.jar and it will run your Clojure program. So, to summarize Leiningen helps a lot.

If you want to add the http-kit you can add it to the dependencies vector:

lein deps only install the dependencies: but if you run lein run and there is a missing dependency it will install it anyway.

Last, you can add the same code inside the -main function:

(ns blog-example.core
  (:require [org.httpkit.client :as http])
  (:gen-class))

(defn -main
  "I don't do a whole lot ... yet."
  [& args]
  (println "Hello, World!")
  (println @(http/get "https://api.github.com/users/octocat/orgs")))

And then running lein run will actually do the request.

Missing Parts

I know: I’ve skipped some parts to make your understanding of this basic tutorial easier. I will make a quickly explanation here of those parts:

@ signal on http call: this is related to the asynchronous nature of HTTP Kit: a call without to the http/get function will be asynchonous and a callback can be passed as an argument and the code inside the callback will be run once the request returns something. Using @ actually turns it synchronously: it will do the request and wait the result. This construction is called promise and I will cover it later.
project.clj: besides your app metadata like description and license, it also includes the dependencies, where your compiled code will be generated target-path, the main function (:main) that will be called on lein run and the profiles: that are different possible switches that you can enable or disable: like testing libraries, debug libraries and so on: by default the only profile tells that when generating an jar through the uberjar command it should do the ahead of time compilation. Remember this is just a subset of all possible options for a a Project.clj: it has lot of possible options and configurations and all of them are explained here.
core.clj: ns is the namespace declaration: it contains what namespaces should also be required and may also contain a gen-class that will generated a named Java class: this is useful here because the JVM requires a public main method and this gen-class directive will generate this method. Last, the -main function contain an & args as argument: it will contain the arguments given on the command line this & is actually a way to tell a clojure function that the function may receive an arbitrary number of arguments (besides the ones declared before the &) and will be stored on a vector.

Summary

So I hope you learned a few things here:

How to “install” Clojure
How to run Clojure code only with Java
How to install Leiningen
How to run code inside Leiningen
How to add a new dependency
How to require it on your library

Next time I will speak more about the Clojure ecosystem: what they are and how they work:

Where to look for Libraries;
Editor recommendations;
Project.clj overview;
Core.clj overview;

8 Small git tips

2015-03-31T21:03:00-03:00

As git is one of my daily tools, I’ve compiled 8 useful (and short) tips that I use almost everyday.

Selectively add with -p

When you want to commit something, you can either: select all the files through git commit -am or add a specific file through git add file. However, you may want to only add only a part of a file to the commit. You can use git add -p and select interactively what parts you want to add to your commit.

After selecting interactively all the chunks you want to commit, just do a git commit (without -a) and you will only commit the selected parts. There is also the git checkout -p to select changes to be reverted. After adding, you can see what you have selected using git diff --cached.

Interactive Rebase

If you’re working on a branch and you made some WIP commits that you want to squash or you want to remove a commit plus the reversion of this commit, you can do a interactive rebase to reorganize your commits.

To do it, you just need to run git rebase -i <commit> where the <commit> is the sha1 of a commit before the one you want begin rewriting. Then it will open on your editor (the one specified on $EDITOR env var or the one specified on your git config) some instructions to change your commits history where you can pick, squash (merge two commits on a single one), reword (change the commit message), edit or even remove a commit.

Be aware that this change the history, so if you have pushed this commits, you will have to force to push it again, so never do this on master branches or branches that you’re not the only one using.

Stashing

If you in the middle of something and you have to change a context to fix something else, you can git stash the current changes. However, you may end up forgetting about this stashed changes some time later. So I try to keep a stash zero (just like inbox zero but for stashes) policy. Every time I have a stash a dollar sign appears on my prompt and then I check it through git stash show -p and then I can pop it through git stash pop or discard it through git stash clear.

Global gitignore

You can specify what files git should ignore through a .gitignore file on the root of a project. But if you have files that git should ignore but you’re the only one that generates that files (like vim’s bkp files or something that your editor or OS generates like .DS_Store files for OSX), you can specify on your configure a global gitignore file that uses the same syntax of a project .gitignore that will .

git config --global core.excludesfile=/Users/flores/.gitignore

Warn whitespace

I have to admit: sometimes I forget some trailing whitespaces on my code. But I normally don’t commit them because I use this option: apply.whitespace=warn. Every time I’m adding a chunk of a file through git add -p, and this chunk contains trailing whitespaces, git warns me of it so I can fix it before committing.

Auto setup rebase

Another cool tip is to auto setup rebase: if you have a branch with some commits that aren’t pushed and someone else also commits and pushes on that branch, when you pull, git will create a commit merging your commits to the commits upstream. As this commit is meaningless, I prefer to setup auto rebase on pull through the configuration: branch.autosetuprebase=always. Doing so, on every pull, git will try to reapply your commits with the current version of the upstream branches.

Better logging

Have you tried to find an specific commit merged from a branch ? git log provides some basic information, but you can use git log with a more useful message:

git log --graph --decorate --pretty=oneline --abbrev-commit

--graph will generate lines between commits and will expose branches, --decorate shows where branches are located in this commits, --pretty=oneline will show only the sha1 and the title line and --abbrev-commit will reduce the sha1 to the first 7 chars (which is normally unique in your repository). You can check a more detailed explanation of these options (and a whole lot more) on explain shell.

Rewrite a commit message

So you’ve commited something but did a poor job describing it. Or else, you just made a typo. You can rewrite the message with git commit --amend. You can use the -m to set a message through the command line or it will open the default editor with the commit message so you can change. You can also include new things with git add and add it to the previous commit. Remember that just like Interactive Rebase, it changes the history, so if you have pushed that commit, you will need to force push this changes.

If you use git on your job, you probably know a git trick that increases your productivity. Can you share it on the comments ?

Book Review: Javascript: The good parts

2015-02-27T21:18:00-03:00

After reading Douglas Crockford chapter on Coders at Work (and his criticism on Javascript) and being recommended to read it by several friends, I got eager to read his book about Javascript: Javascript: The good parts. So, I spent last carnival holiday to read it.

I think the main proposal of this book is to not get really deep on all Javascript features, but to overview good features of the language (and also the most common pitfalls). It could also serve as a introduction to Javascript to someone that knows his way around programming (i.e. knows what a for, a while and a switch means), as it briefly covers the Javascript syntax (but does not goes deep on it). It also covers arrays, objects, regular expressions and how to deal with exceptions.

Object oriented Javascript

As Javascript is normally used for small manipulation of pages (like filling a cities select input through AJAX after selecting a state), it can be written as a procedural script. But as things evolve, more and more code like this end up being added, and like a snowball you Javascript code end up becaming a mess and every time you have to touch it, you start considering rewriting it.

On being a Object Oriented language, it is possible to split responsabilities in objects and use important concepts of the Object Oriented world like encapsulation and inheritance. The book also gives a good overview on how prototypes work and how to use them to create objects based on other objects.

Functions as objects

One of the greatest features of Javascript is that functions are objects (just like a number, a string or an array), so it is possible to store functions on a variable, pass them as an argument or make functions return functions. If you know how to listen to events, you probably did this by adding a function to be executed when an event is triggered.

The book explains the benefits of this property: cascading, currying, how to create a memoization mechanism, and callbacks. It also explains important concepts on functions: dealing with an arbitrary number of arguments, closures and how scopes in JS work.

Language downsides and JSLint

This chapters are on the appendix, but I think they deserve a mention as they explain why caring for good code in JavaScript is important (and how JSLint you can help you with this).

JavaScript has a lot of pitfalls: a naive a = 1 expression, when a was not declared turns a it into a global variable, the typeof behaviour for NaN is strange (who would think that typeof NaN is Number), the weak-typing misleadings ('1' + 1 being '11') and the evil == that does type coercion to both sides of the expression and end up doing nasty things like evaluation to true '0' == false and 0 == false. The book goes through all these things and explains the problems with them.

Fortunately, there is a tool (created by the author) to catch this problems before your code goes to production: JSLint . You can use this tool to check for some traps in your Javascript code (and you can also specify when you want to do some things in purpose, so JSLint will not complain about them). The book has an appendix that explains the options on JSLint.

API Documentation

One part that I think could be improved on this book is the API documentation or a big method overview . I think book pages are not the right place for an API documentation, as it is really boring to read and although you may learn a thing or another, the biggest part of it you end up reading and forgetting a few days later.

There is a chapter that goes through small set of methods of all the standard types: Strings, Arrays, Numbers, Regexps etc. It is really well written and contains good examples but I don’t think a book telling the language good parts should do a method overview.

Conclusion

This book is a must-read if you touch any JavaScript code, as it is normally easy to write JS code, but difficult to write good quality Javascript code due to its inumerous traps and OO and function features of the that are not known to a major extent of the developer community. Even if you consider yourself fluent in JavaScript and write it for a living, you will for sure learn a new trick or something new on the language by reading this book.

The author wrote an article about how JavaScript is misunderstood, that is also worth reading if you are not a big JS fan.

Letting Git Bisect Help You

2014-11-30T22:42:00-02:00

Git has a wealth of tools that can improve our workflow. One of them is bisect, which is not as well known since it doesn’t get used too much, but can still prove itself very useful when you need to discover that exact point where a good branch became a bad branch. Which commit broke something? That’s where bisect comes into play.

Binary search

Bisect is based on the binary search algorithm. Given a sorted array of elements, it returns the index where an element can be found (or whether it doesn’t exist in the collection), based on the following invariant: once you have compared an element, you can discard all elements that come before or after based on the outcome of the comparison.

If you’ve ever been to a library, you may have noticed that books are sorted by author within a particular section. Say you’re looking for a book whose author’s last name is Martin, and you have just found out the author of the last book on a given shelf starts with an F, you can then be sure the right shelf is somewhere after this one. By following this method, you can narrow your search until you find the book you’re looking for.

Binary search works the same way. If you’re searching for an x element in an array with n elements, you pick the n/2 element and compare it with x. If it is greater, you do the same with the sub-array starting at (n/2)+1 and ending at n. If it is less, you do the same with the sub-array that starts at 1 and ends at n/2, and then proceed recursively. Here’s an interesting demonstration of this algorithm.

As for the number of comparisons, it is on average a lot smaller than you would get from the naive search algorithm (where you compare each element in the array). It is also faster on average and very useful to find elements on a sorted collection.

Bisect

Bisect uses binary search to find the commit (i.e. the diff) that introduced a particular change to your branch.

To be able to know where a bug appeared, first you need a way to detect it. You can write a test for it – or, if that’s not possible, try to have at least a list of steps that reproduce it.

With this test in hand, you can start bisecting your code. Remember you must have two commits: one where you’re sure the bug exists and one where you’re sure that the bug does not exist. You can use hashes and tags to point to these commits. When you bisect you mark a commit as good (the test passes) or bad (the test fails).

In the diagram above, you can see the steps that the bisect performs (green points are good commits, and red ones are bad commits): it moves to the middle of the commits after this (in case of a good commit) or to the middle of the commits before this (in case of a bad commit), ending when it finds the first bad commit (in our case the commit number 6) after 3 steps.

I have generated a git repository with 1024 commits appending integers (from 1 to 1024) on a text file, one integer per line. Our mission is to find which commit introduced the integer 1013 (suppose having this number in our file is a bug).

First, we find a way to quickly determine whether this number is present. Let’s call this the test. Doing this is very simple:

$ grep 1013 file.txt

With this command we can start doing our bisect:

$ git bisect start

Running this command on master returns the not-desired result, so we will mark it as bad.

$ git bisect bad

Now, let’s point out a good commit: suppose the first commit (7c0dcfa) is free of this bug. So we can either checkout to this commit and mark it as good or run git bisect bad with the hash commit.

$ git bisect good 7c0dcfa
Bisecting: 511 revisions left to test after this (roughly 9 steps)
[8950f7db7e7cad0b2dc394ff9b75fc3d38c9d72a] added 512

A short way to roll those three commands into one is this:

$ git bisect start master 7c0dcfa

The start command accepts the bad and the good (in this order) as arguments.

Nice, git bisect has put us in the middle of the collection. Running the test here doesn’t reveal anything, so we mark it as good.

$ git bisect good
Bisecting: 255 revisions left to test after this (roughly 8 steps)
[a01ba83f3500b48da97c5f5c33052623aaa4161a] added 768

Again, the test returns the desired state. Good it is.

$ git bisect good
Bisecting: 127 revisions left to test after this (roughly 7 steps)
[4a4a668bf3363d09af5fd1906bc4272aacdb4495] added 896

Good again.

$ git bisect good
Bisecting: 63 revisions left to test after this (roughly 6 steps)
[9059c5b8b898159e8d1d797bff3b1febd1fd6a1c] added 960

Take a look at the message: apart from the current commit and how many revisions are between this one and the first bad one, it tells you how many times you are expected to make this comparison: 6 times. As it is good again, we may mark it as good.

$ git bisect good
Bisecting: 31 revisions left to test after this (roughly 5 steps)
[0c844d0b33ef297b742206ebc293f4925705b083] added 992

Again, we’re in a good state.

$ git bisect good
Bisecting: 15 revisions left to test after this (roughly 4 steps)
[0ee17eb17bd96b321a01c73eb13a8929a68b1239] added 1008

4 steps remaining, and we’re in a good state.

$ git bisect good
Bisecting: 7 revisions left to test after this (roughly 3 steps)
[dfb1e71736dcfffa2a30aecd7299f45f757c057e] added 1016

But now, running the test shows us the non-desired result. So, let’s mark it as bad.

$ git bisect bad
Bisecting: 3 revisions left to test after this (roughly 2 steps)
[6e6d08c374df5162fed65fed82859b69f86b936e] added 1012

We’re near the end and we have a desired result, so let’s mark it as good.

$ git bisect good
Bisecting: 1 revision left to test after this (roughly 1 step)
[1d23b7045a8accd254efa859d7fc66f1f58a59f0] added 1014

Almost at the end, we have a bad commit.

$ git bisect bad
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[458eab0eb8d808e16d98ec7039a7c53855dd9ed6] added 1013

Last step and bad it is.

$ git bisect bad
458eab0eb8d808e16d98ec7039a7c53855dd9ed6 is the first bad commit
commit 458eab0eb8d808e16d98ec7039a7c53855dd9ed6
Author: Rodrigo Flores <mail@rodrigoflores.org>
Date:   Tue Oct 21 22:31:05 2014 -0200

    added 1013

:100644 100644 7bc3db7f48a43ccf1a8cc7c26146912cc88c1009 b393a2138a96c1530f41f701ab43cca893226976 M  file.txt

So, now we finally have the commit that introduced the number 1013, as we wanted. You can review the steps taken by using git bisect log.

git bisect start
# bad: [740cdf012013dc41a39b41d4b09b57a970bfe38f] added 1024
git bisect bad 740cdf012013dc41a39b41d4b09b57a970bfe38f
# good: [7c0dcfa7514379151e0d83ffbf805850d2093538] added 1
git bisect good 7c0dcfa7514379151e0d83ffbf805850d2093538
# good: [8950f7db7e7cad0b2dc394ff9b75fc3d38c9d72a] added 512
git bisect good 8950f7db7e7cad0b2dc394ff9b75fc3d38c9d72a
# good: [a01ba83f3500b48da97c5f5c33052623aaa4161a] added 768
git bisect good a01ba83f3500b48da97c5f5c33052623aaa4161a
# good: [4a4a668bf3363d09af5fd1906bc4272aacdb4495] added 896
git bisect good 4a4a668bf3363d09af5fd1906bc4272aacdb4495
# good: [9059c5b8b898159e8d1d797bff3b1febd1fd6a1c] added 960
git bisect good 9059c5b8b898159e8d1d797bff3b1febd1fd6a1c
# good: [0c844d0b33ef297b742206ebc293f4925705b083] added 992
git bisect good 0c844d0b33ef297b742206ebc293f4925705b083
# good: [0ee17eb17bd96b321a01c73eb13a8929a68b1239] added 1008
git bisect good 0ee17eb17bd96b321a01c73eb13a8929a68b1239
# bad: [dfb1e71736dcfffa2a30aecd7299f45f757c057e] added 1016
git bisect bad dfb1e71736dcfffa2a30aecd7299f45f757c057e
# good: [6e6d08c374df5162fed65fed82859b69f86b936e] added 1012
git bisect good 6e6d08c374df5162fed65fed82859b69f86b936e
# bad: [1d23b7045a8accd254efa859d7fc66f1f58a59f0] added 1014
git bisect bad 1d23b7045a8accd254efa859d7fc66f1f58a59f0
# bad: [458eab0eb8d808e16d98ec7039a7c53855dd9ed6] added 1013
git bisect bad 458eab0eb8d808e16d98ec7039a7c53855dd9ed6
# first bad commit: [458eab0eb8d808e16d98ec7039a7c53855dd9ed6] added 1013

We have on this example 1024 commits, and traversing through all of them took 10 steps. If we had twice this number of commits, it would take us only one more step to find the desired commit, as the efficiency in the worst case scenario is O(log n).

Even so, it is a tedious task to run the same command 10 times. So, let’s think about a way to automate it.

Automating

Git bisect can use a script return code to determine whether a revision is good or bad. If it is 0, it is good; if it is not 0, it is considered bad. Luckily, grep has a return code of 0 if it finds a line with the desired pattern and 1 if it doesn’t find a line. You can check this by running echo $? after running the test.

This is the opposite of good and bad, so we need to write a shell script that negates the result (I’m not used to writing shell script, so please let me know if there’s a cleaner way to do so).

#!/bin/sh

if [[ `grep 1013 file.txt` ]]; then
  exit 1
else
  exit 0
fi

Let’s save it to a file called test.sh and give it permission to execute.

Then, after marking the good and the bad commits, you just need to run:

git bisect run ./test.sh
running ./ola.sh
Bisecting: 255 revisions left to test after this (roughly 8 steps)
[a01ba83f3500b48da97c5f5c33052623aaa4161a] added 768
running ./ola.sh
Bisecting: 127 revisions left to test after this (roughly 7 steps)
[4a4a668bf3363d09af5fd1906bc4272aacdb4495] added 896
running ./ola.sh
Bisecting: 63 revisions left to test after this (roughly 6 steps)
[9059c5b8b898159e8d1d797bff3b1febd1fd6a1c] added 960
running ./ola.sh
Bisecting: 31 revisions left to test after this (roughly 5 steps)
[0c844d0b33ef297b742206ebc293f4925705b083] added 992
running ./ola.sh
Bisecting: 15 revisions left to test after this (roughly 4 steps)
[0ee17eb17bd96b321a01c73eb13a8929a68b1239] added 1008
running ./ola.sh
Bisecting: 7 revisions left to test after this (roughly 3 steps)
[dfb1e71736dcfffa2a30aecd7299f45f757c057e] added 1016
running ./ola.sh
Bisecting: 3 revisions left to test after this (roughly 2 steps)
[6e6d08c374df5162fed65fed82859b69f86b936e] added 1012
running ./ola.sh
Bisecting: 1 revision left to test after this (roughly 1 step)
[1d23b7045a8accd254efa859d7fc66f1f58a59f0] added 1014
running ./ola.sh
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[458eab0eb8d808e16d98ec7039a7c53855dd9ed6] added 1013
running ./ola.sh
458eab0eb8d808e16d98ec7039a7c53855dd9ed6 is the first bad commit
commit 458eab0eb8d808e16d98ec7039a7c53855dd9ed6
Author: Rodrigo Flores <mail@rodrigoflores.org>
Date:   Tue Oct 21 22:31:05 2014 -0200

    added 1013

:100644 100644 7bc3db7f48a43ccf1a8cc7c26146912cc88c1009 b393a2138a96c1530f41f701ab43cca893226976 M  file.txt
bisect run success

And then, you’ve found your broken commit without hassle.

Conclusion

You probably you won’t use it everyday. Maybe once a week or even once a month. But git bisect can give you a hand to find a broken commit. Keeping the practice of doing small commits, while not required, can help this process a lot. Finding out the culprit commit won’t be of much help if you’re forced to look through a gigantic diff. So, having small commits may help you in the long run, even if that means having too many commits.

What about you? Do you use bisect frequently?

Kudos for the friends @mdepolli and @p_balduino for the kind reviews!

Ruby Patterns: Webservice object

2013-07-16T08:28:00-03:00

This is a series about Ruby Patterns, which will explain some common uses of Ruby syntax. The second post is about a webservice based. I like to call it a pattern because it is very common and tends to repeat (on a not-duplicated way) on Service Oriented Architecture based applications. Of course, this code may be too sophisticated for such a small script like this, but it may be a good way to handle things on a more complex application.

So, you’re given the task to write a class that access a webservice and returns the info on it (e.g. the github repos for a given organization). A simplistic implemetation can be like this:

require 'faraday'
require 'json'

def retrieve_repos_for(org)
  connection = Faraday.new(:url => 'https://api.github.com') do |faraday|
    faraday.adapter  Faraday.default_adapter
  end

  response = JSON.parse(connection.get("/orgs/#{org}/repos").body)
  response
end

retrieve_repos_for('github').each do |repo|
  puts repo['clone_url']
end

Obviously, this is an example of a procedural implementation, so, let’s make it more object oriented.

require 'faraday'
require 'json'

module Github
  class Organization
    def initialize(organization)
      @organization = organization
    end

    def repos
      connection = Faraday.new(:url => 'https://api.github.com') do |faraday|
        faraday.adapter  Faraday.default_adapter
      end

      response = JSON.parse(connection.get("/orgs/#{@organization}/repos").body)
      response
    end
  end
end

Github::Organization.new('github').repos.each do |repo|
  puts repo['clone_url']
end

Nice, we now have it inside a class. But we can extract some private methods here.

require 'faraday'
require 'json'

module Github
  class Organization
    def initialize(organization)
      @organization = organization
    end

    def repos
      response = JSON.parse(connection.get(repos_url).body)
      response
    end

    private

    def connection
      Faraday.new(:url => 'https://api.github.com') do |faraday|
        faraday.adapter  Faraday.default_adapter
      end
    end

    def repos_url
      "/orgs/#{@organization}/repos"
    end
  end
end

Github::Organization.new('github').repos.each do |repo|
  puts repo['clone_url']
end

Well, the public methods seems to be more concise now and we have extracted some methods that can be more easily reused. But there is a few flaws: if we call the repos method twice, it will make two requests, but this is easy to solve: just add some memoization.

require 'faraday'
require 'json'

module Github
  class Organization
    def initialize(organization)
      @organization = organization
    end

    def repos
      @repos ||= JSON.parse(connection.get(repos_url).body)
    end

    private

    def connection
      Faraday.new(:url => 'https://api.github.com') do |faraday|
        faraday.adapter  Faraday.default_adapter
      end
    end

    def repos_url
      "/orgs/#{@organization}/repos"
    end
  end
end

Github::Organization.new('github').repos.each do |repo|
  puts repo['clone_url']
end

We’re almost done here. I’m not satisfied with the JSON.parse(connection.get(repos_url).body), it seems such a complex line. Let’s extract some methods here.

require 'faraday'
require 'json'

module Github
  class Organization
    def initialize(organization)
      @organization = organization
    end

    def repos
      @repos ||= get(repos_url)
    end

    private

    def connection
      Faraday.new(:url => 'https://api.github.com') do |faraday|
        faraday.adapter  Faraday.default_adapter
      end
    end

    def get(url)
      JSON.parse(connection.get(url).body)
    end

    def repos_url
      "/orgs/#{@organization}/repos"
    end
  end
end

Github::Organization.new('github').repos.each do |repo|
  puts repo['clone_url']
end

The repos method seems simple enough now, and we have moved the parsing responsability to the get method. But we can get rid of it delegating to someone else to do that. There is a great gem called faraday-middleware that parses it for me, based on the content type header and returns a hash, so, let’s use it.

require 'faraday'
require 'faraday_middleware'

module Github
  class Organization
    def initialize(organization)
      @organization = organization
    end

    def repos
      @repos ||= get(repos_url)
    end

    private

    def connection
      @connection ||= Faraday.new(:url => 'https://api.github.com') do |faraday|
        faraday.adapter  Faraday.default_adapter
        faraday.response :json, :content_type => /\bjson$/
      end
    end

    def get(url)
      connection.get(url).body
    end

    def repos_url
      "/orgs/#{@organization}/repos"
    end
  end
end

I’ve also added a memoization on the connection (we don’t need to instantiate a new one every time).

Two days later, a new requirement: get the organization info and add it on the api. This implementation makes it really easy:

require 'faraday'
require 'faraday_middleware'

module Github
  class Organization
    def initialize(organization)
      @organization = organization
    end

    def repos
      @repos ||= get(repos_url)
    end

    def info
      @info ||= get(info_url)
    end

    private

    def connection
      @connection ||= Faraday.new(:url => 'https://api.github.com') do |faraday|
        faraday.adapter  Faraday.default_adapter
        faraday.response :json, :content_type => /\bjson$/
      end
    end

    def get(url)
      connection.get(url).body
    end

    def repos_url
      "/orgs/#{@organization}/repos"
    end

    def info_url
      "/orgs/#{@organization}"
    end
  end
end

org = Github::Organization.new('github')

puts org.info['name']
org.repos.each do |repo|
  puts repo['clone_url']
end

Neat! It is indeed really easy to add new endpoints support to our class. But I think it has a lot of responsability: it is dealing with the connection to the API. Let’s extract a new class that does that and refer to it on the client method.

require 'faraday'
require 'faraday_middleware'

module Github
  class Client
    def initialize
      @connection = Faraday.new(:url => 'https://api.github.com') do |faraday|
        faraday.adapter  Faraday.default_adapter
        faraday.response :json, :content_type => /\bjson$/
      end
    end

    def get(url)
      @connection.get(url).body
    end
  end

  class Organization
    def initialize(organization)
      @organization = organization
    end

    def repos
      @repos ||= client.get(repos_url)
    end

    def info
      @info ||= client.get(info_url)
    end

    private

    def client
      @client ||= Github::Client.new
    end

    def repos_url
      "/orgs/#{@organization}/repos"
    end

    def info_url
      "/orgs/#{@organization}"
    end
  end
end

org = Github::Organization.new('github')

puts org.info['name']
org.repos.each do |repo|
  puts repo['clone_url']
end

Now we have a pretty simple class, which I finally consider a final implementation, it splits the responsability to parse to another place and now I only have to specify the endpoints and get (or post/put/patch/delete) it. Another improvements may be to add a condition to do something when we have a 404 on an endpoint.

What about you ? Would you recommend another improvement ? Do you use something similar ?

octopress and capistrano

2013-03-30T12:05:00-03:00

To deploy your octopress blog with Capistrano, you should do these steps:

First, add capistrano gem to your Gemfile. Then, after running a bundle install, run bundle exec capify . (or bin/capify . if you use binstubs) to generate the Capistrano files.

After that, you should add a content like this to your config/deploy.rb file.

# Set this forward agent option to not have to add your server's ssh public key to your repository's host authorized keys
ssh_options[:forward_agent] = true
require "bundler/capistrano"

set :keep_releases, 5
set :scm, :git
set :scm_verbose, false

# Set your repository URL
set :repository, 'YOUR REPO HERE'

# Set your application name
set :application, "YOUR APPLICATION NAME"
set :deploy_via, :remote_cache

# Set your machine user
set :user, 'YOUR SSH USER'

set :deploy_to, "/home/#{user}/apps/#{application}"
set :use_sudo, false

# Set your host, you can use the server IPs here if you don't have one yet
role :app, 'YOUR HOSTNAME', :primary => true

default_run_options[:pty] = true

namespace :octopress do
  task :generate, :roles => :app do
    run "cd #{release_path} && bundle exec rake generate"
  end
end

after 'deploy:update_code', 'deploy:cleanup'
after 'bundle:install', 'octopress:generate'

Now, you should add the group production to the development group on your Gemfile. Doing this, capistrano will be able to run octopress:generate on your server.

source "http://rubygems.org"

group :development, :production do
  gem 'rake', '~> 0.9'
  gem 'rack', '~> 1.4.1'
  gem 'jekyll', '~> 0.12'
  gem 'rdiscount', '~> 1.6.8'
  gem 'pygments.rb', '~> 0.3.4'
  gem 'RedCloth', '~> 4.2.9'
  gem 'haml', '~> 3.1.7'
  gem 'compass', '~> 0.12.2'
  gem 'sass-globbing', '~> 1.0.0'
  gem 'rubypants', '~> 0.2.0'
  gem 'rb-fsevent', '~> 0.9'
  gem 'stringex', '~> 1.4.0'
  gem 'liquid', '~> 2.3.0'
end

gem 'sinatra', '~> 1.3.5'
gem 'capistrano'

Finally, before doing the first cap deploy, do a git clone of your blog’s repository on the server (or try to connect through ssh to the repository server on your blog server), you will need to use the -A option on ssh command to forward your keys. This is needed because ssh asks for the fingerprint confirmation on the first ssh connection. As capistrano won’t do the ‘yes’ on the confirmation you should do it manually.

Doing this, you will be able to deploy your blog through capistrano. Do you have any tips on how to improve this capistrano recipe ? Please say them on the comments :).

Ruby Patterns: Method with options

2012-07-16T14:00:00-03:00

This is a series about Ruby Patterns, which will explain some common uses of Ruby syntax. The first post is about methods with options.

Let’s start with an example where it can be useful. Suppose, you’re given an object and you should write a method that prints the object’s attributes in HTML. Also, you’ll also have to follow this other requirements:

You may specify the attributes that you want to show (by default you should show all of them);
You may specify the attributes that you don’t want to show (by default, you should show all of them, i.e. this list is empty);
You may specify the format: possible formats are :ordered_list, :unordered_list, :table (by default, present it as a :table);
You may specify the strftime format of the objects of Time type (default is :month/:day/:year :hour::minute :timezone in strftime format);
You may specify the strftime format of the objects of Date type (default is :month/:day/:year in strftime format);

So, given this requirements. Let’s write a function:

def object_to_html(object, only=nil, except=[], format=:table, time_format="%m/%d/%Y %H:%M %t", date_format="%m/%d/%Y")
  only ||= object.attributes - except

  if only - except != only
    fail "You can't put in except an attribute that you've put on only"
  end

  # Code that does what we proposed
end

Some examples of the usage of this function (and what someone will say after reading your code):

object_to_html(user) # Seems OK
object_to_html(user, nil, :encrypted_password) # What is this nil ?
object_to_html(user, [:email, :phone, :created_at], [], :ordered_list, "%d/%m/%Y %H:%M", "%d/%m/%Y") # WTF is this empty array ?
object_to_html(product, nil, [], :table, "%d/%m/%Y %H:%M", "%d/%m/%Y") # Gosh. My eyes are bleeding. Call an ambulance.

What a creepy code! If you want to change the last attribute, you will have to know all the default values of the other ones to be able to change it. You will also have to know (or constantly look at its implementation or documentation) the exactly order of the attributes. Also, if you decide to remove one attribute, you will have to find all the calls to this method and change it on all of them. Definitely, this is what I call ugly code: difficult to use and hard to maintain.

So, this pattern comes in handy: method with options!

def object_to_html(object, options={})
  default_options = {
    :format => :table,
    :only => nil,
    :except => [],
    :time_format => "%m/%d/%Y %H:%M %t",
    :date_format => "%m/%d/%Y"
  }

  options = options.reverse_merge(default_options)

  options[:only] ||= object.attributes - options[except]

  if options[:only] - options[:except] != options[:only]
    fail "You can't put in except an attribute that you've put on only"
  end

  # Code that does what we proposed (now it uses the options hash instead of local variables)
end

The same usage examples come along:

 object_to_html(user)
object_to_html(user, :except => [:encrypted_password])
object_to_html(user, :only => [:email, :phone, :created_at], :format => :ordered_list, :time_format => "%d/%m/%Y %H:%M")
object_to_html(product, :time_format => "%d/%m/%Y %H:%M", :date_format => "%d/%m/%Y")

Now to the explanation of how this works: the Hash#reverse_merge! method does the key role here. It merges the two hashs (the one containing the default values with the one that was given as a parameter) and in case of the same key in both, it keeps the one that belongs to the Hash that received the method call (in our case, the one given as a parameter to the method). The method is called reverse_merge because the Hash#merge method solves the conflict in the opposite way: in case of the same key in both, it keeps the one that belongs to the hash given as argument on merge. We can easily use merge instead of reverse_merge if we write the line 10 this way:

options = default_options.merge(options)

And then the code would work the almost the same way: the only difference is that instead of accessing local variables we may access the arguments in the options hash, but this can be easily converted. Also, note that we’re adding a bang (!) so, it will modify the hash and do the merge on itself instead of not modifying and returning other instance of hash.

Also, if you want to enforce that one option is always given, you can do this:

begin
  options.fetch(:format)
rescue KeyError # Hash#fetch raises KeyError when a key is not found
  raise ArgumentError, "The :format option should be passed"
end

Doing this, you will treat a missing argument the same way Ruby does with the function calling. However, I’m not a big fan of this approach: I think that if an argument is mandatory, it should be passed as a regular argument and not on as a function.

This pattern is heavily used on famous Ruby libraries like Rails, Bundler or Devise and plays a big part on pretty DSLs like Rails routes or Gemfiles. So, IMO, you should use it wherever you find a need to include more than 2 or 3 arguments on a function. Also, it is possible to use this on Javascript: if you take a look at jQuery’s ajax function it works the same way: you can pass the url as the first argument and the other parameters (like callbacks, fallbacks, urls, etc) as an object.

A big disadvantage in doing this is that it is very difficult to know what is the default options, and even looking at your code, you may only understand what is the default behaviour passing through all the code (which is far from good). So, I suggest that you document the default parameters and what does they mean like Rails do in this example with the has_many method.

UPDATE: As Rafael França suggested, I’m not overriding the argument anymore. As hashes are passed as a reference, when you execute merge! you actually change the parameter outside the function as an undesirable side effect.

UPDATE II: Adam Meehan reminded me on the comments that Hash#reverse_merge is part of ActiveSupport and not part of Ruby standard library. To be able to use it, you will have to require activesupport/core_ext/hash.

conference speaker 101

2012-05-31T11:43:00-03:00

TLDR: Here I talk some of the things that helped me to present my first talk at a conference

Last year, I presented my first talk at RS on Rails, which is a Rails (and Ruby) conference which happens in Rio Grande do Sul at the south region of Brazil.

Before that, I never had given a 40 minute talk at a conference (I’ve only talked about 20 minutes at FISL 2008 on a short talk about Archlinux) and as a regular geek I’m shy and I don’t think I’m someone which masters the skills to delivery great talks like Steve Jobs.

This posts intends to tell you some tips that helped me to prepare my talk, how I rehearsed it and what I did on stage.

Content is king

If you’re not a well known speaker, people will not watch you unless you have something great to tell them, so spend some time thinking about nice things to talk and if a regular attendant will be able to watch it. I remember about going to an event where the speaker talked about Ruby’s meta programming and 90% of the attendants were doing their first steps on Ruby and Rails. The speaker was really a good speaker and taught really nice things, but clearly he missed the point talking about this on an event with that amount of newcomers. You should measure if the subject that you will talk is too basic or too advanced for the event.

Before writing any slides or sending any proposal, create a list of topics that you will talk about. Validate this with a colleague and see if he thinks that you had chosen well the topics. If you’re not secure about the topics and your colleague didn’t get the point of your talk, try to write a draft of the slides and present it to your colleagues in order to get some feedback. If you have friends that talked on events, you can get valuable feedback and discover what is worth talking and what isn’t.

Study about creating great slides

Every time I think about boring things, I remember a professor in college that used to give his lectures using power point slides. I remember that most of the students slept or played games on their mobile phones. You may say that students should pay attention on what professors say, but in our defense, I say that his lectures were really really boring. Obviously, even if he is a great researcher and knew a lot about the subject he was teaching, he lacks knowledge on presentation preparation to keep the audience focused.

Of course, lectures and talks are more than great slides. But in order to deliver a great talk, you should prepare beatiful slides.

But how do we prepare a good talk ? There is some resources that teaches you how to prepare great talks. Due to a friend recommendation, I’ve read Presentation Zen: Simple Ideas on Presentation Design and Delivery by Garr Reynolds before preparing the slides. This book tells you how to prepare and present a great talk and design great slides. I’ve also watched the talk Your Slides Suck by Shane Becker, that tells you how not write slides. Other resource that was published after my presentation is Zach Holman’s blog post about slide design which tells developers (that generally lack design skills) how to create beautiful and great slides.

Rehearse, rehearse, rehearse

When you’re writing slides, you probably know what to say on each slide. But, do you really know how to connect it to the previous and the next slide? Also, you know how to explain it clearly? As expected your answer is probably “no”. So, you should rehearse. Check if you’re saying what you need to say in your presentation time frame without having to hurry or skip any subject. Also, check if you can explain clearly every topic (if you’re afraid to forget something and your presentation software have this feature, write a note visible only to the speaker about what you’re supposed to talk). If you’re not secure, present it to your colleagues and get some feedback about your explanations or examples. When I wrote my talk, I present it at least 4 times to my colleagues at Plataformatec and every rehearsal I got valuable feedback about it. I also presented it to myself several times, and the last one was on the night before the event.

Also, ask a friend to play the devil’s advocate and find flaws on your talk.

Rehearsing is really important, but keep in mind that you’re rehearsing to practice what you will say, not exactly what you will say. Don’t try to memorize everything, because listening to someone reading (or saying something that has been memorized) is way boring of listening to someone telling a story (unless you’re an actor, which is probably not the case).

On the stage

When you’re on stage, you will probably get nervous with all that people watching you. I remember, while presenting a lightning talk on Ruby Conf Brazil, a moment when I forgot what to talk and, on stage, I saw the crowd looking at me, and I thought I would not be able to continue presenting. Fortunately, I quickly remembered what to say and continued my talk. Asking my friends later, they said that this moment took around a few seconds, but appeared to me that it took at least fifteen seconds.

You will probably get nervous, but if you have rehearsed it enough you will calm down and keep it going.

Adding UTF-8 support to rubies compiled through ruby-build

2012-04-25T07:58:00-03:00

If I had to point one thing that gives me true headaches, I would certainly choose encodings. I have already spent a lot of time dealing with encoding problems and I usually get stressful when a have to deal with a new one (I think it is common sense to suppose that everyone would prefer spending time dealing with other things instead of encodings).

So, last week I was using irb and pry to prepare a small course about Ruby and I found out that I couldn’t input UTF-8 characters. When trying to insert a á I was getting this unicode character code:

After a quick glance in some search results for irb utf-8 problem, I discovered that I was having this problem due to a Ruby compilation without Readline (which is a library with some useful functions for command lines).

Fixing this was really simple due to the possibility of passing options to include on Ruby compilation on ruby-build. So it was only a matter of executing this line to recompile and reinstalling Ruby with Readline support.

CONFIGURE_OPTS="--with-readline-dir=/usr/local/Cellar/readline/6.2.2" rbenv install 1.9.3-p194

Of course, if your readline dir is somewhere else, please put the correct directory instead of the one I put there.

After that, irb and pry started to accept UTF-8 characters.

UPDATE: Fixed some grammar issues.

UPDATE II: I’ve put a better picture for the solution

Why we should choose Stack Overflow instead of Mailing lists for Open Source support

2012-01-18T23:36:00-02:00

This is a very opinionated blog post about why we should use Stack Overflow instead of mailing lists for open source support. This post doesn’t reflects the opinion of the company that I work for and I also didn’t receive any kind of benefit from Stack Exchange or its partners. As you may notice, I’m not an english native speaker, so if you find an gramatical or any error that may be related with a bad explanation, please provide some feedback in the commentaries. I will correct/explain better it as soon as I can.

In the past few months, I spent some time answering some questions about Devise on Stack Overflow and I found out that it is an awesome web application to ask and answer questions regarding open source projects (and everything else). I also try to follow some mailing lists of my favoured projects, but I realized how open source support works better on Stack Overflow than on Mailing Lists.

But firstly, what is Stack Overflow?

Stack Overflow is a free (as in free beer) Q & A website about programming, so if you have a doubt, uncertainty, problem or question about a library, framework or programming language, you can ask it there and there is a good chance for you to get an answer about it. It was the first site of Stack Exchange, a collection of 76 sites about a lot of subjects including System Administration, Cooking, Startups, Gaming, Sci Fi, Firearms and a lot of other websites about a lot of stuff.

But why is it so famous? Well, It is not a simple Q & A website. Given a question and a answer, you can give a +1 to the answer if it is useful or a -1 otherwise. You can also accept an answer for a question you asked, that is, you believes that a given answer was correct or solved the problem that you had. You can also assign a tag to your question, and some people who follow that tag may be able to answer it. You can also attribute a bounty for questions users find relevant but hasn’t received any good answers (and this bounties are basically points, which I will explain later on this text). Besides this, you can comment on questions and answers to ask more information about it, without having to pollute the answers with this. Needless to say that if your question doesn’t fit with the question policy it will be closed without mercy.

It also doesn’t have a “Big Brother” team that moderates questions: all questions are moderated by the community. But not all members can moderate questions: when contributing with questions or answers you may receive points, and depending of the number of points that you have, you gain some trust and some “power” become available for you. In the beginning you can only ask and answer questions but after some points you can edit questions and answers, assign bounties, comment on questions or answer and even moderate questions and answers. You can also earn badges, based on goals. In summary, you earn trust and power through desert. Ask wise questions and provide great answers and you gain power. Promote or post spam, off-topic or flamewars and you will be banned.

Well, IMHO, I find it awesome. A lot of questions get answered in reward of points and badges that can make a lot of people (including recruiters) hear your name. This number of points, also appear besides your name on every question that you answer, which provides you more credibility. It is nothing more than a “gamificated” and improved internet forum that works.

But what about Mailing Lists? Mailing lists exists in a long time and became very popular with Yahoo groups and Google groups among other famous options. They’re very easy to use (just a matter of sending and replying to e-mails) but they do lack of a good search tool (it is not easy to find a question about something with only a naïve text field search) and usually have problems with off-topic discussions and , spam and flamewars. Ok, this is a human problem, but well moderated communities tend to have less problems with that matter. Also, e-mails aren’t a good place to discuss about code: the lack of a source code formatting turns very difficult to read it. The moderations is also difficult: you can’t remove an e-mail after it was received by the list, so irrelevant or off-topic questions may get more repudiating replies and attention than well founded questions about topics that really matter. Although they’re not great for support, they’re great to ask opinions and provide discussions and even announce new versions.

So, I’m not suggestions that we should stop using discussion lists. I’m only suggesting that we may reduce its scope to not accept (or accept and recommend posting it on Stack Overflow next time) support questions (some open source software already have two lists to avoid questions being asked on the development mailing list, which may be used for discussions about the development of the software, e.g. MongoDB-user and MongoDB development for support question), so it is only a matter of encourage users to send questions to Stack Overflow instead of posting on the support list.

To summarize, we have two options: one that seems to be outdated and its search only based on full-text search, without giving relevance to the content (as tags do), and not easily moderated. And another that provides support to source code formatting, tag assignment (and tags may have more relevance than words on a text) and that can be easily moderated. Other great advantage: we may sort the answers by the sum of all scores, so the answer that gets the biggest number of +1 (and is probably more useful than the others), will appear first. In contrast, there isn’t anyway to say that an answer solves your problem on a mailing list without having to send a message and the mail thread starts to get difficult to read.

So. here is an idea: why don’t we stop using it for support and only use it for discussion or announcements? OK, users may not like this at first and we may think about a smoothly transition, but in long term, we may have better resources to find questions and to follow a tag and answer questions about it.

What do you think about this? Would you use it for your open source project support?

EDIT: Fixed some broken links

Rails documentation

2012-01-11T22:17:00-02:00

As a Rails developer, I frequently have to check some definitions or get some help about a method or class on the api documentation. In my first months as a Rails developer, I used the official documention, but after some months I discovered other alternatives, which I preferred over the official documentation.

But firstly, what’s wrong with the official documentation ?

Some days before Rails 3.1 release, the official documentation was redesigned and it became so much better than it was, I liked the navigation sidebar, where you can find a method or a class easily. But (and this is IMHO) I still find it very unpleasant to read. However, as an official documentation, it is updated by the same time that the code is released and I expect it to have documentation for “edge-ish” Rails features (there is also a version which is synchronized with rails master branch).

But what are the options ? There is two options that I found out to be great. The first one is Rails Searchable API Doc and the second one is Apidock. There is also another options like RailsBrain which seems to be abandoned (the latest Rails version there is 2.3.2) or to import the documentation into the OSX dictionary which I have not tried it yet.

Rails Searchable API Doc

Rails Searchable API Doc is the documentation I currently use when I develop Ruby on Rails applications. It has a nice look and also have the cool sidebar (by the way, the new design of Rails official documentation uses this engine, but with different colors). It also allows adding the Ruby language documentation and another popular libraries documentations as well. So, as a developer that most of the time uses Ruby, Rails, RSpec and Nokogiri, the possibility of adding this other documentations on the same window is great. Other cool option is the possibility of selecting the documentation for a minor version, so you can select the documentation for Rails 2.3.x, Ruby 1.8.x and Nokogiri 1.3.x. Besides this, we can download the full documentation (including generating a custom package of documentation) to access it locally (if you have a mac, you can use Fluid to setup an App with the documentation. Ok, you can do the same with the Rails documentation, but it is not easy as this. If you like this alternative and you also uses jQuery you can use a similar version to check the library’s documentation.

However, there is some points you should consider when using railsapi documentation: it is not updated as the official one (the current version is 3.1.3 but Rails api is still on 3.0.8rc1, one minor version behind). Other problem resulting of the possibility of adding many libraries, is that the results of a search can show you some irrelevant results, as you can see in the image above.

If you want to access enumerable#merge (which is probably the most popular between these results) you will have to look for results that aren’t what you want.

Api Dock

I used Api Dock a lot, but as I prefer an offline based version of a documentation, I choose Rails Searchable API Doc. But, there is some really nice reasons try Api Dock out.

First of all, it allows people to make comments to make the documentation more comprehensible (there is a nice example on has and belongs to many documentation) and it also allows other people to thank who wrote the commentary. Besides this, it gives you a nice look about a class/method among Rails versions, so you can see the point where something has been added or removed. It also lists some related methods whose documentation can be useful. It is also more updated than Rails Searchable API Doc, as of the time of writing this post, it is on version v3.1.0. It also shows the current status of the documentation, if it is extensive or there is only a few lines of explanation.

One shortcoming of Api Dock is that it is not open source, so you can’t generate it on your computer and access it locally, although it is free (as in free beer) to use. It is also only available for three projects: Ruby, Rails and RSpec and they can’t be accessed simultaneously (which can be bad if you like to see several documentations together but the search results doesn’t shows a lot of irrelevant results).

What about you? What documentation do you use? Have you tried these three I mentioned? Do you want to recommend me another documentation ?