Balint Erdi

Lessons learned from solving 4Clojure problems

2012-12-29T14:55:00+01:00

A few days ago I completed the last problem on the 4Clojure site. If you want to learn Clojure, solving these problems is a great way to do it. Several of its features -that I’ll highlight below- make it a great learning tool. Other features probably arise from Clojure being a (pragmatic) functional programming language. Coming from mostly an OO background these were also new to me and thus deserve their own paragraph.

I hope that after reading through the list you’ll end up being persuaded of the merits and want to solve (some of) the problems yourself. If you do, please let me know how it went and what you learned from it.

If you really get stuck, there is a Google group dedicated to the 4Clojure problems. You can also leave a comment here so I can help or go directly to check my solutions. Let’s jump in.

Why is 4Clojure a great learning tool?

Looking at others’ solutions

After solving a problem, you can check how the users you follow solved it. That’s arguably the most important feature when it comes to learning since it is essentially a code reading exercise when the functionality of the code is well-known (since it solves the same problem you’ve solved) and the authors are probably more proficient.

On several occasions I saw solutions that were both more concise and clearer than mine (especially when tackling a hard problem). Dealing with the inferiority complex on the very short term is dwarved by how much wisdom you gain from these. For what it’s worth, the users I learned most from are hypirion, jafingerhut and chouser.

(If you’d like to follow me, I’m balint there.)

Executable, well thought-out test cases

To submit your solution, you paste your code into a textbox and click a button. The test cases, which are visible, are then checked one by one. If all the lamps become green, your solution is accepted. If not, you get an error message and have to try again.

This method has several advantages. First of all, it eliminates the imprecisions you might have had after reading the description of the problem. Second, it gives you a set of examples to work against. Third, they force you to think deeper about the problem since they are constructed to reject a partial solution.

Timeouts

Your solution can be functional and pass all the test cases but if it does not finish in a certain time, it will get rejected. I bumped into this on several occasions. Most of the time it was because I came up with the brute force solution to a hard problem and hoped I’d get away with it. In other cases it was because of a technical issue, like allocating too much memory.

In the first case, it made me think again about the problem (see Hammock-Driven Development) and come up with a more ingenious solution. In the second case, I learned something about a technical aspect of the language. In both cases, I grew a bit wiser about optimization -which is a “real-world” coding skill- so I’m happy the authors of 4clojure chose to implement this constraint.

What does one learn about (functional) programming?

Hammock-Driven Development

Also known as “step-away from the computer to solve hard problems”, Hammock-Driven Development is a term coined by Rich Hickey, the creator of Clojure, in a keynote speech.

Apparently ridiculing Test Driven Development (TDD), HDD holds that the most important activity to solve a hard problem is to think deeply about it without any distractions. Most of the time sitting in front of the computer is a distraction in itself since it begs to be typed on and prevents actual deep thinking to happen.

This one is really hard to get used to because whenever we write code we feel like we’re getting closer to a solution. Thinking, on the other hand, does not provide any tangible output.

However, HDD has rung ever more true with me as I progressed. When tackling hard problems, I tended to think about them for some time and then started to type in actual code. The problem was, when I felt that the solution became convoluted I did not go back to the proverbial hammock but carried on with the implementation. Most of the time it either turned out to be a dead-end or a solution I’d much better hide.

Even more importantly, a cleaner solution is one that is easier for others to understand. Since code is mainly for others to read and occasionally for computers to execute, more thinking up-front results in less time spent developing and maintaining the code down the line.

The REPL is a powerful developer tool

The power of the REPL is one those realizations most of us coming from OO will come to. Since FP languages have very little state and side-effects and thus a lot of idempotency, trying things at the REPL is taken to the next level. You launch a REPL once and then copy-paste the building blocks of your solution between your editor and the REPL (and there are better solutions then copy-pasting).

Use higher level functions

FP languages strive to have a small set of data structures and a high number of functions that operate on them. Clojure is no exception. Though it’s not hard to assemble the higher-level function you need yourself, in the majority of cases, it’s just extra work: it’s probably already defined in the core.

This is something that I learned by reading others’ solutions and learning about awesome functions (frequencies, merge-with, condp come to mind). After a while I looked up the high-level function myself from the cheatsheet.

Switch to Command Line Vim on iTerm

2012-03-30T12:04:00+02:00

Switching between applications is more of a mental context switch than just switching between the panes in the same window. Or at least that’s what I told myself when I decided I switch from using macvim to the simple command line version. It also brings me pretty close to using tmux which I would primarily use to send commands between panes. Most of the posts about using tmux only briefly touch on switching to command line vim so I thought I’d fill in the gap somewhat.

I use iTerm2 on a MacBook Pro and run 10.6.8 (Snow Leopard), although I think it should run fine under Lion, too.

First, you’ll have to install a fairly recent vim. Homebrew has a policy of not having packages which the operating system already provides, but fortunately there is a homebrew-dupes repository that has vim:

$ brew install --HEAD https://raw.github.com/Homebrew/homebrew-dupes/master/vim.rb

This will configure, make and install vim.

Next, I looked for a theme that looks good on both a light and a dark background and has iTerm support (a colorscheme for the terminal emulator). I’ve found Ethan Schoonover’s solarized theme really cool. Just follow the steps in the README of the repository to download both the vim and the iTerm colorscheme.

To install the iTerm colorscheme, go to Preferences -> Profiles to select your profile and then click the Colors tab. There is a “Load Presets” dropdown from which you have to choose Import and find the solarized itermcolors file.

To use the vim colorscheme you have to add the colors file to somewhere where vim finds it. There are many ways to do this, so check out the README to find the method that suits you.

Once, done you have to set the colorscheme in your vimrc:

colorscheme solarized

(vim finds out the appropriate background automatically, so you don’t have to explicitly set the background)

I was all set up and most things worked just like in the GUI version. One thing that bugged me was that my cursor keys stopped working. I don’t use them for moving and text input but I do for cycling in the command history or flipping between file matches for Command-T. It took me a while to find out that iTerm sends a wrong escape sequence for the arrow keys. I could fix that by selecting another set of key presets for the terminal’s profile. In iTerm, go to Preferences -> Profiles and select your profile. Select Keys, then Load Preset and select xTerm with Numeric Keypad.

There is one more trick I find handy. If you set

set clipboard=unnamed

in your config, then anything you copy from vim by the usual vim commands (y, d, x, etc.) will be available on your system clipboard and thus pastable.

An Annotated Assortment on Mockist Testing

2011-11-30T21:30:00+01:00

Most of us read blog posts every day. We read them, take an idea out of them and then, most of the time, forget about them. Some of them are stashed away in the back of our minds, ready to jump out if we face a related problem.

A precious few of them, however, we keep thinking back to without a specific reason.

I’ve bitten by the “mockist” testing bug when I read this one a while ago. It expresses a contrarian opinion about how to test Rails applications which struck me as odd at the time. That’s probably the reason I read it several times.

A few weeks ago I watched Gregory Moeck’s Why You Don’t Get Mock Objects and I was stung by the same bug only more deeply, this time.

Using that video as a starting point I then roamed Gregory’s blog for more and felt like I was beginning to grasp it. Now, obviously, I’m at the beginning of this journey and still have a lot of teeth-cutting to do. Nevertheless, I want to share with you the gems I’ve found so far.

Gregory’s blog has a very good primer on the difference between stubs and mocks: “Stubbing is Not Enough”. I’d even go as far as to claim that it explains its subject better than Martin Fowler’s classic “Stubs Are Not Mocks”, although that latter goes into more detail and is a definite must-read, too.

James Golick has another great piece that drives home the point better than his first post I mentioned: “On Mocks and Mockist Testing”.

Along comes Avdi Grimm with his strict sounding “Demeter: It’s not just a good idea. It’s the law.” , with a very interesting discussion in the comments. The same gentleman wrote “Making a Mockery of TDD” in which he touches on the concept of using mocks as a design tool.

Nick Kallen’s “Why I love everything you hate about Java” is clearly provocative and definitely worth to contemplate on. It is also the only one of the bunch that does not use Ruby (but Scala) for the code examples.

Finally it seems like the fountainhead in the matter is the “Growing Object-Oriented Software, Guided by Tests” book by Steve Freeman and Nat Pryce. I’ve only gotten until putting it on my reading list so please chime in if you did read it.

Please, also pipe in if there are any materials in the subject you’d recommend. It would also be cool to see open source projects that extensively use mocks for testing, the only one I found so far is friendly.

Event loop primer

2011-10-19T00:06:00+02:00

I recently got into developing a web application with node.js (aka cancer). Coming from a synchronous world, it took (and to be honest, still takes) quite a while to grok how writing asynchronous code differs from my previous experiences (AJAX calls with jQuery only go that far).

As with many fine technologies or methods (TDD, NoSQL, functional programming come to mind) it’s your whole thinking that has to change. In this post I want to share an example from Trevor Burnham’s excellent Coffeescript book that gave me one of those aha moments.

(If you don’t read Coffeescript code, you can go to the Coffeescript web site and paste the above example in to get compiled Javascript)

The example above is broken. It gets stuck at the until countdown is 0 row. In an event loop system events (callbacks) only get run after the “main line” of execution (or, in other words, all other code) has completed. So the until loop blocks out the callback of the setInterval, countdown never gets decremented and thus an endless loop ensues.

I’m sure that there are many ways to fix this, I came up with the following (and wonder if there is one closer to the original):

And that’s it. I hope this simple example pushes you up on that pesky learning curve.

(The snippet was published by the kind permission of the author.)

Five wasted years - on the futility of university education

2011-09-28T00:00:00+02:00

I graduated from the Technical University of Budapest to earn a M.Sc. in Software Engineering. Albeit it is supposed to be an asset on my CV I’ll argue below that the long years of university education was just not worth it.

When trying to summarize what advantage university education brought me there is precious little I can think of. Five years is a lot of time to spend without actually getting something out of it for one’s professional career.

Obviously I’ve formed my opinion based on my university experience which might not (and probably is not) be applicable to all higher education. Philosophy, economics and law all require different formation and practice might be harder (or outright dangerous) to attain in some areas (think medicine).

Also, there is a great variance between countries although even the famously high-standard US education system seems to yield not actual but rather looks-good-on-my-CV benefits, say some smart guys.

Learn to learn?

Tech is changing extremely rapidly. Today’s hotness could be a thing of the past in a few years. Consequently universities should not try to keep up with the pace and teach students state-of-the-art stuff. Higher education needs to transcend short-term utility and provide a base one can build on for the rest of his career. So we were told or made to believe.

What is probably considered the essence of higher education is “learn to learn”. It is the idea that universities need to teach future-engineers how to quickly adapt to new fields and techniques (programming languages, databases, architectures, etc.).

This is appealing but universities don’t do that. I had to sit through long hours of material not even vaguely related to software engineering. The practical stuff (e.g programming languages) was taught with stone age style methods (programming on paper). The other subjects, those that were supposed to provide us the broad vision, I suppose, were way too much in volume and failed to achieve that goal.

Make to learn

In my current job I’m lucky to work with some guys who dropped out of college and started to work. They might not know about Prim’s algorithm or the intermediate value theorem, but I think the result of the time they spent making things greatly surpass the time I spent learning the above.

Programming (or, in its more CV-friendly, hire-me name: software engineering) is a task that could be the modern equivalent of wood carving. You can learn all that you want about the craft, the only thing that really matters is doing it, a lot. Only, programming is way better. There is virtually no waste (unless you publish what you make :) ) and the tools are more accessible and cheaper.

Take it from someone who, unlike most kiddos at the university, started programming late: it feels awkward and weird at first. And then the second and third time, too. Slowly, though, you start to feel like it’s actually fun and sometimes more than that. You’re building something which you’ll look at ashamedly a few months later but at least your program does something extraordinary, like sorting a list of numbers. That’s science!

Math data constructs, probability theory and cryptography notwithstanding, have you got this feeling of coolness (dare I say, awe) out of university lectures?

Total waste, really?

In fairness, and as a measure to counter my arrogance, I had to consider arguments on the pro-education side, too. Here is what I came up with:

Math

When faced with a programming challenge I can recall on some occasions an algorithm used to resolve a similar problem. That’s not to say you can’t google up a solution if you have a vague idea what to look for. Nevertheless if you can mentally page through the solutions for a given problem from your university classes, their time and space needs and their constraints then you surely save a lot of time. I forgot all the relevant facts about any algorithm and have to look it up every single time, but it could be me.

Deciding on one’s vocation

Most kids don’t know what to do with their life when they are 18. Real life still seems distant and most want a few more years of canteen, beers and idling. Whether it is beneficial for them or society (taxpayers) as a whole to be allowed to do that is another matter. Nevertheless, I’m convinced that a high number of students can have a clearer picture about whether they want to do “computer science” for the rest of their lives after a couple of years.

Outstanding teachers

Even though I reckon you can learn everything you need to know on your own (from the Internet), having a good teacher can squeeze your learning curve. Although I believe this is more attainable with a small group (and even more in a one-to-one, mentoring relationship) it’s definitely possible for an outstanding teacher to speed up absorbing knowledge with a bigger group, too. Unfortunately, with one notable exception, my teachers were not of this type, but that’s a weak argument against college education in general.

Even if all these pro-education arguments are valid, however, a couple of years is surely enough to derive all the advantages they bring. Then, you still have 3 years to go and carve wood.

On navigation

Let me finally share a quote I’ve just found in Walden by H.D. Thoreau and that summarizes my intent with this post in one swell sentence:

To my astonishment I was informed on leaving college that I had studied
navigation! – why, if I had taken one turn down the harbor I should
have known more about it.
Henry David Thoreau Walden

Please defend the status quo (or indulge in bashing it)

As I stated in the introduction, my opinion is just a drop in the ocean, a tiny slice in a big cake, a lone voice in the NY Stock Exchange (you get my point).

If you share your opinion, there will be two voices already and we’ll have more information to decide about whether we should advise our children to go to college, for example. Then three. We may even reach four voices.

Joking aside, if you went to college to learn computer science in Hungary or in another country, I’m interested to hear your opinion. If you studied something else, don’t be discouraged, please share also. My points are mostly valid for computer science but I’m curious to hear which other fields they hold true in (or in which fields they don’t).

Powered by Octopress

2011-09-05T21:22:00+02:00

Octopress is a blogging engine on top of Jekyll. It got my attention since it provides themes and a layout that looks great on mobile devices, too.

I had been playing with the idea to do my own simplistic theming long enough to realize that I would never do it. The other feature I like is its plugin system: some I’ll use right away (e.g Github style codeblock) but I also like the idea that I can write a plugin for any specific need that might arise later.

I had had my blog on Jekyll so migration was not really hard. I hit a few minor roadblocks on the way but documentation is great and the author, Brandon Mathis was really helpful on the support forum so I could eventually sort them out.

If you like to blog like a hacker, want to own your content and don’t want to be bothered with styling, I encourage you to join the squid team.

Git rebase to fix your local commits

2011-07-19T00:00:00+02:00

Let’s say you have the following three commits in your local repository:

% git log -3 --oneline
b648f1a Fix propagating errors in findOrInitialize. (28 seconds ago)
8789cd1 Non-destructive filtering for bumblebees. (12 hours ago)
1a35285 Propagate errors the node.js way. (14 hours ago)

The last commit, b648f1a conceptually belongs to the first one, 1a35285. It only came later because say you haven’t run the tests before committing it and only realized later you introduced a bug. Or some other misdemeanor. Whatever the background is, it would be great if there was a way to squash the two related commits together. Turns out there is: interactive rebase.

The syntax of the git-rebase is the following:

git rebase [-i | --interactive] [options] [--onto <newbase>]
             <upstream> [<branch>]

What happens when you do git rebase is that the commits that are on the current branch but are not in upstream are saved. The current branch is reset to upstream and then the saved commits are replayed on top of this.

It’s worth to mention that you should only do this if you have not pushed out these changesets to a remote where others might have pulled from it. Rebase creates new commits and if your collaborators pull the new commits, chaos can ensue. (See “Perils of Rebase” in the ProGit book)

This can be used to achieve what we want:

% git rebase -i HEAD~3

Since the commits that are on the current branch but not on the commit three commits from here are the last three commits, here is what we get:

pick 1a35285 Propagate errors the node.js way.
pick 8789cd1 Non-destructive filtering for bumblebees.
pick b648f1a Fix propagating errors in findOrInitialize.

We want to meld the “fix” commit into the “propagate” commit since that’s how it should have been in the first place. So we move b648f1a up and squash it into the previous commit:

pick 1a35285 Propagate errors the node.js way.
squash b648f1a Fix propagating errors in findOrInitialize.
pick 8789cd1 Non-destructive filtering for scales.

After a successful rebase this is how the new log looks like:

% git log -3 --oneline
73eed18 Non-destructive filtering for bumblebees. (9 seconds ago)
1e63d17 Propagate errors the node.js way. (39 seconds ago)
1b24891 Minor fixes in Bumblebee buzzing. (16 hours ago)

Note that the three commits we had before have now been nicely compacted into two, and the propagation commit is now consistent and fixed. It can now be pushed.

ps. You might wonder what we use bumblebees for in our project. Actually they are faux. They serve to obfuscate real names in propietary code. I hope I can one day write code where bumblebees will be first-class citizens, though.

Don't Tell Michelle, Facebook privacy as it should be

2010-10-10T00:00:00+02:00

It’s been a quiet four months over here. My excuse is that I’ve been working. I joined a startup, Secret Sauce Partners in June and we have started to build our first product a couple of weeks later. Now I am proud (actually, quite proud) to announce that we have released a first version last week.

The problem we are solving

I’m sure you have a Facebook account. You probably have around 130 “friends”. Chances are you want to share lots of things but on several occasions you don’t want to tell everybody. Sure, you can create friend lists and post to them based on what you want to say. Or, you can pick individual friends for your message (good luck with that if you would like to speak to more than 3 friends). It would definitely make sense (and would hugely improve the state of the world!) if you only showed your Farmville achievements to your friends who play Farmville and nobody else. Especially not your colleagues during work hours.

Wouldn’t it be cool if you did not have to fiddle with setting the proper audience of your posts every single time you share something? If the people whom you post to would be determined from the content and the context of your message? Better still, if friends that join later could not see your posts prior to that?

The solution

Let’s go back to Farmville-land (I know, I know, but bear with me for a few more minutes). If you only want to show your Farmville posts to selected people the only way to achieve it is to set your default privacy to that list of selected friends prior to diving in to FarmVille. Then you set it back to your default posting setting. Not quite comfortable.

That’s where our application comes into the picture. To solve the above problem, you set up a rule that says: “Hide posts from Farmville posted during Work Hours from my Co-Workers”. You lay back, go feed your piggies, and water your sunflowers safe in the knowledge that your colleagues will not know about it. Suppose you don’t want your mom to know where you spend your weekends. Here is your rule: “Hide All posts from Foursquare from Family”. Feeling the urge to swear like a sailor some times and don’t want your little cousins or nieces to know about it? “Always hide posts with Swear Words from Kids”. Don’t want to spam your Twitter followers on Facebook? “Always hide posts from Twitter from Twitter followers”. Then, there are the “Show” rules*. For example, an even more sensible rule for Farmville posts could be: “Always show posts from Farmville to Players”. (* “Show rules” are the next feature we are going to work on.)

You see, the possibilities are endless, so why don’t you set up your own rules and give it a try? I bet you will never want to go back to (broken) standard Facebook privacy setting.

Ah, and if you don’t want your wife to know how much high-cholesterol food you eat, then “Don’t Tell Michelle”

It's a spec, not a test

2010-06-03T00:00:00+02:00

You must have heard the question several times on the Rails mailing list and different IRC channels: “Should I test validates_uniqueness_of”? The standard answer to that one is “No, you definitely should not. It’s Rails framework code, and it’s already thoroughly tested. If you followed this path, you should also test whether objects are properly persisted in the database.”

I think, however, that the question is wrong and thus you can not give a correct answer. It is wrong because validates_uniqueness_of is the implementation, not the requirement. If you approach it from this angle, the question turns into whether you should test the specific implementation or whether you should verify that (business) requirements are met.

That, in turn, comes down to tests vs. specs (short for specifications) and this is again an opportunity for specs to shine. If you write specs instead of tests (or, to put it in a more mind-warping way: if your tests are actually specs), then the above question is a no-brainer: it’s part of the specification that no two users can have the same email address, so you must have a spec for it:

user_spec.rb

describe User do
  it "has a unique email address" do
    Factory(:user, :email => "jeff@topnotch.com")
    lambda { Factory(:user, :email => "jeff@topnotch.com") }.should
     raise_error(ActiveRecord::RecordInvalid)
  end
end

On the other hand, if you stick with calling your tests tests (how orthodox! ;) ) then not only you have to think (which consumes a lot of resources), but you can also come to the wrong conclusion and emit a strong business requirement from your test suite. And then you might not remember to have the implementation for it after modifying the code for whatever reason. And then bad things might happen.

(This thought came to me when coming to work in the subway this morning. I was never quite comfortable with the name “specs” but now it’s starting to make a lot of sense to me. You are encouraged to disagree. Dissent is what makes the world progress.)

Remove'em trailing whitespaces!

2010-02-16T00:00:00+01:00

Some of you reading this probably use TextMate. It is an excellent editor with two caveats. The first is that you can only see one file in the editing window (no screen split), the other is that there is no save hook. This latter gave me headaches since I can’t stand any trailing whitespace in source code and the easiest solution would have been to run a script to remove those when the file is saved.

Without further ado I’ll paste my solution below. Obviously this is not a difficult task to accomplish so the goal is to share not to show off. I use Git for SCM and the following solution parses out the files that have been modified and runs the whitespace eraser script for those. If you use something else (why do you?) you should obviously change the first building block:

parse_modified_files_from_git_status.rb

#!/usr/bin/env ruby -wn
modified_file_pattern = /^#\s+(?:modified|new file):\s+(.*)$/
puts $1  if modified_file_pattern =~ $_

rm_trailing_whitespace.rb

#!/usr/bin/env ruby -wn
$:.unshift(File.dirname(__FILE__))

require "trailing_whitespace_eraser"
TrailingWhiteSpaceEraser.rm_trailing_whitespace!($_)

trailing_whitespace_eraser.rb

class TrailingWhiteSpaceEraser
  FILE_TYPES = ["rb", "feature", "yml", "erb", "haml"]

  def self.rm_trailing_whitespace_from_file!(file)
    trimmed = File.readlines(file).map do |line|
      line.gsub(/[\t ]+$/, "")
    end
    open(file, "w") { |f| f.write(trimmed) }
  end

  def self.rm_trailing_whitespace!(root)
    root = File.expand_path(root)
    files = File.directory?(root) ? Dir.glob("#{root}/**/*.{#{FILE_TYPES.join(',')}}") : [root]
    files.each { |file| rm_trailing_whitespace_from_file!(file.chomp) }
  end
end

And then you run it by typing:

rtwsp.sh

git status | parse_modified_files_from_git_status.rb | rm_trailing_whitespace.rb

If you decide to use this, it is more convenient to download the raw source

Hopefully I did my tiny bit to have less trailing whitespace in OS code.

2010 is the year of the Javascript

2010-01-28T00:00:00+01:00

If you’ve read the The Pragmatic Progammer - From Journeyman to Master (and if you have not, believe me, you should) you might remember one precious advice: learn a new programming language every year. So when the year draws close to its end (around April, that is) I really start thinking about which language should be the one for me next year.

This reflection is very pleasant; it is such a joy to immerse oneself into a new language, to slowly discover its structure, its subtleties and learn new ways of thinking, of approaching a problem (because that’s probably the biggest gain of it, even surpassing adding another language to your arsenal) that its prospect is already rewarding in itself.

My choice in 2009 fell on Clojure. I love functional programming, its elegance, its cleanness, its statelessness and after getting my feet wet with ANSI Common LISP a while ago I felt like Clojure would propel me on my way of becoming an FP guru. Guided by a great book I wrote a couple of, admittedly not too difficult exercises (hey, a sudoku solver is already something, isn’t it?) and I would certainly like to continue to do more with it in 2010.

Now, on to 2010.

Thomas Fuchs is certainly right when he predicts that Javascript will play an increasingly significant role in web development. I was definitely wrong when I had thought of JS as an undebuggable, structure- and featureless, incoherent language. Of course, you can write Javascript code that is like this but I now think that -at least in the case of Javascript- this tells more about the craftsman than about the language.

Make no mistakes, I can already do Javascript! I can miraculously make a check box appear if you click on a link, or update a database record without reloading the page when you press a button and a ton more such wizardry, really. However, I always felt like there was a whole lot more to Javascript than putting some dumb-looking functions in a file and then include it from my html (no, I don’t think inlining it makes a difference) and I felt uneasy about my javascript functions just standing there on their own without any apparent belonging to a group or some kind of cohesive force or idea that holds them together.

My recent enthusiasm comes from the fact that I had to browse through some of the source code of scriptaculous and I really liked what I saw. The enthusiasm is not totally new but looking at that source code demolished the last shreds of my Javascript skepticism, so to speak. So a year later than Peter Szinek I am hopping on the same train: Javascript, here I come! If you have read a good book that enhanced your Javascript foo, please tell me about it in the comments.

And you? Have you already chosen your language for 2010?

Twuckoo 0.3.5 - With email notification

2010-01-26T00:00:00+01:00

As some of you might know, I am running a few twitter channels that use my brainchild, twuckoo, to periodically send a tweet.

In two of these cases the program takes its next message from a file and takes care not to tweet the same thing twice by keeping track of the messages already sent. This means, however, that the pool of sendable messages are depleted after a while and the script simply does not send any more tweets, the channel becomes dead.

This has happened a couple of times in the past and I only realized it with a few days of delay which is kind of awkward (if you were subscribed to one of these accounts, I apologize). So I decided to have twuckoo send a notification in this case so I can go in and refill its queue. Hence twuckoo version 0.3.5 was born which sends an email to the address set in the configuration file when it failed to send a message.

To install it, add gemcutter to the list of gem sources if you haven’t already:

install.sh

gem source --add http://gemcutter.org

And install twuckoo:

install.sh

gem install twuckoo

You might be thinking that it would be more straightforward to have the program replenish itself so that no intervention from the part of the developer is needed. And you would be right, this could be a task for the next version. I still think it is a good idea to send a notification so the (wo)man in charge knows it happened.

So go ahead, have a look at the README, set up your own twitter channel and don’t fret about when the queue would run dry. You’ll be notified.

My tech book list for 2009

2010-01-11T00:00:00+01:00

The title probably tells it all, below is the list of technical books I read in 2009. I am -unfortunately :)- not getting paid for any recommendation that lies herein.

My Job Went to India by Chad Fowler

This one is a real motivation booster. Or it might function like love in the case of separated lovers: if you were motivated in the first place, you’ll get a kick out of this book and want to do everything it recommends right from the next day. If you were not it might depress you to have to do so many things to stay afloat. Full of useful advices mostly drawn from the author’s personal experience in India. 5/5!

(There is a second, reworked version under the name: The Passionate Programmer, you should probably buy that one.)

User Interface Design for Programmers by Joel Spolsky

I real like Joel Spolsky’s essays both for the content and for his witty style. I laughed out loud on several remarks in this book, too. The book reaffirms how important good UI is, makes it accessible for programmers by giving guidelines and provides some examples of good (and especially bad) examples. Its style makes the “dry” technical stuff fun to read.

The Pomodoro Technique by Francesco Cirillo

A time-management and productivity tool, the Pomodoro Technique is the latest craze. The technique can be summarized in a few paragraphs, the details fill about 40 pages which comprise this book. The e-book can be downloaded for free. Although I do not follow all the advices described in the book fully (like administering the tasks you have done at the end of each day) I think the technique is a great way to achieve focus especially if you have problems with it. (The PragProgs have also recently published a book about it)

Programming Clojure by Stuart Halloway

Clojure is a functional programming language that runs in the JVM (hence the “j”, I guess). This excellent book contains everything you need to know to start (and continue) programming in Clojure from multimethods to infinite sequences, from ways to deal with concurrency to understanding and writing macros. Clojure rocks and so does this book!

The Principles of Beautiful Web Design by Jason Beaird

I bought this book to be able to do a fancier design than putting black text on white background and using ugly textboxes with Times Roman fonts. It has certainly lived up to this goal and it has some very practical advices and links to resources (I particularly liked the chapter about color). Just as its title says, it explains principles rather than implementation details, which is what I was looking for. That said, I still feel I need another book (and a decent amount of practice, of course) to beam me to the level I had planned to reach before buying it. Any suggestions?

Test-Driven Development By Example by Kent Beck

A true classic, I reckon this book is a must-read for anyone aspiring to adhere to TDD or in fact for anyone already doing it at any level of mastery. In the first two chapters two distinct functionalities are developed (the first one, the Money example, in Java, the second, the xUnit example, in Python) by tiny steps using TDD, at each step pondering on the problem at hand and giving an explanation of the chosen solution. The third and final chapter deals with TDD patterns and is an invaluable addition to the first two chapters. Now that I am writing about it I feel like I should reread this book in 2010 (and then probably once every year). Red, green, refactor!

Pragmatic Thinking and Learning by Andy Hunt

The fact that I have not finished only goes to show its splendidness, I simply don’t want it to end yet. (I read this in a Paul Graham essay but I can’t find it right now). Its subtitle, “Refactor your wetware”, is telling: it shows you how to make yourself more productive by harnessing the innate capabilities of your brain and debugging its intrinsic “bugs”. I guess it is also a very refreshing read because it is about our mind and its workings and not some dry technical stuff. The stories and anecdotes linked to some of the sections makes it easy to recall the message. In fact, I liked this book so much I created a “twitter mini-framework” and a twitter channel for the concise version of the advices.

Life is not just about stubbing, view helpers and achromatic color schemes

On a Christmas evening, I sat down on a comfy sofa with Révolutions by J.M.G. Le Clézio in my hand and was totally immersed in it right from page one. I realized I need occasions like this to repeatedly remind myself how much I miss of life if I only ever read technical books. (Sadly, my list of novels I read last year is a lot shorter than the above one but I heartily recommend you A Thousand Splendid Suns by Khaled Hosseini and Poisson d’or by J.M.G. Le Clézio)

Any good books you have read?

I am also making a (so far, only mental) list of books for this year and would like to hear your opinion, too. So have you read any books lately that you would passionately recommend for others?

i15r handles non-english source

2010-01-08T00:00:00+01:00

The other day thinking about how i15r could be improved I realized that there are times when the language of the non-internationalized site is not English (surprise, surprise). Add to this that lots of languages have “special” characters, characters that may not be properly found and replaced by the matchers. So I first learned the basics of how Ruby handles encodings (from James Edward Gray’s excellent series) and then adjusted a few matchers to catch those special characters, too. (Note: one can’t underestimate the utility of a spec suite that covers most of the code)

The specs pass but I am pretty certain there are some cases where the matchers might not be up to the task. If you come through such a case, please submit a bug report. Since there has been a huge improvement in string encoding in Ruby 1.9 (basically, strings now have an encoding as opposed to in Ruby 1.8), there might be some cases where strings are properly internationalized using Ruby 1.9 but not with 1.8.

So if you think this is a valuable addition or just have not tried i15r yet, go get it:

install.sh

gem install i15r --source http://gemcutter.org

New year, new blog

2010-01-04T00:00:00+01:00

I guess it all began with this blog post. I saw that there is another way for composing a blog post than typing it into a textarea with a WYSIWYG interface (which is, by the way, never really WYSIWYG). It also seemed a lot cooler, something a “real hacker” would prefer hands down. The decision to switch my blog over to a textarea-free motor has been subconsciously made the time I read that post but the implementation needed a year to be realized. I reckon the final push was to see how easy the migration of the blog’s content is.

The tools

I decided to use Jekyll, a static-site generator since it integrates nicely with Github and has all the features I need. Being a static-site generator I needed a client-side solution to handle comments. I chose Disqus because that’s the one I knew and liked but there are alternatives.

The process

Content migration

I needed continuity and thus migrating the old posts and comments. Fortunately Jekyll includes migration scripts for several blog motors, among them Wordpress, so that was a piece of cake, everything worked out of the box.

For the comments, I used a modified version of Lackac’s script, that he so kindly directed my attention to (thank you, Lackac). Clearly it was more work than migrating the posts but nothing to shy away from. (If you plan to switch from Wordpress to Disqus I believe you’ll be able to use the script with a few minor modifications.)

New design

For my Wordpress blog, I used a design template that looked ok but was in PHP and was a PITA to modify or to add some styling to. Since I am looking to improve my capabilities as a designer I thought designing my own blog would be a suitable task to get my hands dirty with. I am not concealing the fact that I borrowed ideas from several blogs I liked but hey, they say it’s not a shame to copy good ideas so what’s wrong with that?

Layout, site feed, etc. with Jekyll

Have I said I love open source? There is a Jekyll wiki that lists Jekyll sites along with a link to their sources so seeing how to set up my blog with Jekyll was quite straightforward. I guess this was still the most time-consuming part of all but I did not really have to think of anything, the steps are well-defined. (Here is the source of my blog if you’d like to take a glimpse.)

Buying a Github plan to be able to use a non-github domain

If you’d like to host your blog under a custom domain (http://codigoergosum.com instead of http://balinterdi.github.com) you’ll need to buy a Github plan. Starting at 7$/month for the Micro plan it’s not an insurmountable obstacle, in fact it costs less than the Wordpress hosting I am moving away from.

Changing the name

Back in the spring of 2008, returning from the European Ruby conference (and having finished The Pragmatic Programmer on the train) my mind was buzzing with new ideas and I could not wait to set myself on a craftsman’s journey.

I felt I needed a blog immediately. The hard part of course was coming up with a name. Since I felt it was more important to get something going than to idle on finding the perfect one (I guess we can call this agile, can’t we? :)), I launched http://bucionrails.com. During the almost two years that have passed since that day I have grown somewhat uncomfortable with that name so I decided to change it. I hope you like the new one :). (I set up a feed redirection with Feedburner to notify you of the feed URL change after 15 days but you could directly subscribe to the new feed now)

The end result

It felt appropriate to start the new year with a new blog and -to my surprise- the switch was a lot easier than I had thought. I hope you, the faithful readers of my blog, welcome the change and ameliorate it with your precious comments.

Now I am blogging like a hacker, too!

i15r 0.3.1 is out - now with haml matchers!

2009-12-14T00:00:00+01:00

I’ve just released a new version of the i15r gem which alleviates the tedious task of internationalizing a Rails site by finding the “static” texts and replacing them with I18n message strings.

The good news -and probably the reason I am writing a blog post about it- is that it can now process haml files, too. This must cheer up those who -like me- have come to like haml so much they now use it for their projects. So go get it and let me know how it works for you.

Rubygems under the hood: hooking into require

2009-10-28T00:00:00+01:00

(See the introduction to the series.)

When you want to pull in a Ruby library, you require it. The library you require has to be on Ruby’s load path ($LOAD_PATH) in order to succeed. So let’s see how Rubygems hooks into this process.

The secret lies in custom_require.rb. You’ll see a nice comment near the top.

When you call “require ‘x’”, this is what happens: * If the file can be loaded from the existing Ruby loadpath, it is. * Otherwise, installed gems are searched for a file that matches. If it’s found in gem ‘y’, that gem is activated (added to the loadpath).

In fact, that explanation looks so straightforward to me that I doubt if I can add more words to precise it, so let’s look at the code together:

module Kernel
  alias gem_original_require require
  (...)
  def require(path) # :doc:
    gem_original_require path
  rescue LoadError => load_error
    if load_error.message =~ /#{Regexp.escape path}\z/ and
       spec = Gem.searcher.find(path) then
      Gem.activate(spec.name, "= #{spec.version}")
      gem_original_require path
    else
      raise load_error
    end
  end
(...)
end

Aliasing a method, redefining it and calling the original version from the redefined method is a very familiar pattern if you have read Rails source code (called method chaining).

So, just as the above description says, if the file can be loaded “without Rubygems” it is and Rubygems never comes into the picture. If not, we make sure that the load error comes from the file not found on the load path and then find the gemspec for that file and activate it.

Activating it adds it to the load path, so we can call the original require safe in the knowledge that it will now succeed and we won’t get another load error this time around (and go into an infinite loop). Clever, heh?

This is one of the cases when the code is so simple for a moment I think I could have written it myself. With its <10 lines of code Rubygems is empowered to load a myriad of useful Ruby libraries. Isn’t this wonderful?

RubyGems under the hood: Introduction

2009-10-20T00:00:00+02:00

I am not sure about you, but one thing I keep bumping into while developing ruby applications is rubygem version conflicts. If you have ever come across “can’t activate activesupport (= 2.3.4, runtime) for [], already activated activesupport-2.3.3 for []” type of errors, you might be sympathetic to the cause.

Since rubygems was and is being written by some excellent Ruby minds I knew the fault probably lies with me and not with the tool. Being a motivated and proud craftsman I felt like I needed to know my tools and rubygems is probably the number one tool most of us Ruby developers use on a daily basis.

I intend to write a mini-series of Rubygems posts. A post will be added to the series when I explore some murky corner of rubygems inner mechanics. Some of the time this will come from a particular problem I come across while coding while on other occasions I might just look into the source code and try to understand what a particular piece of code does.

Now, I realized that when I read on a blog: “I will go into more detail about X in a forthcoming post”, it is the surest sign that you will never read about X again. At least not on that very blog. And I am not saying this in a mean way, I think it is very natural for our plans to change (“evolve” may sound better) even on the short term. So I am aiming low and hoping to give you more than you expected. (Notice I wrote “mini-series” which may just mean a couple of posts.)

Do not set your expectations high

I am absolutely not a Rubygems expert so please do not be fooled by the “under the hood” title. This will not be commensurate to Jamis Buck’s series on Rails routing. I do not -yet?- have a deep insight and I am not even a contributor to Rubygems.

The forthcoming :) series is just a humble attempt to gain an understanding of a very important part of the Ruby arsenal. I hope to understand it better by explaining to you how it works and I hope you will understand it better by reading my explanation. And last, but not least, I hope we’ll both spend less time debugging those tricky issues.

Having said that, I encourage you to participate in the series. Tell me about a particular problem you had with Rubygems. Correct me if I am wrong. Point me to an article where all this is already explained, and much better. Give feedback so this might evolve to be a discussion rather than an academic lecture. Thank you.

Cucumber meets Selenium meets Integrity

2009-09-25T00:00:00+02:00

Continuous Integration (CI) is a basic building block of any project done in TDD style. In brief, having a CI server properly set up guarantees that developers can rest assured that the application passes all its tests and can thus be deployed at any moment. It does this by sending some kind of a warning if something is broken so it can be fixed immediately.

Now I noticed that if a CI server is not in the mix right from the beginning of project, chances are it will never be, that was one of the first things I installed on a new Rails project. My server of choice is Integrity mostly because it is so easy to set up and quite straightforward to use.

Ok, so next we need some tests that Integrity will run at each commit and make sure the app can still be built. As a high-level acceptance test framework, I use Cucumber, which plays very nicely with Selenium for automated in-browser testing. Since nowadays even the most basic web application will have some amount of client-side scripting code (that is, javascript) if you really want to test the features of your application you’ll need Selenium tests.

That’s when matters get a bit more complicated when it comes to integrating these with the Integrity server. Why? Because on the server you usually don’t have the “desktop environment” which is available on the machine you do the development on. By default, you don’t run a desktop manager and have a graphic display on a server. A tool called xvfb comes into a picture that emulate a dumb framebuffer so you can still programs that need a graphic display.

To have some practical guidance, here’s what I did on an Ubuntu server to enable all of the above:

Install java since Selenium runs in the JVM:

apt-get install java-common sun-java6-jre

Install firefox since that’s what Selenium runs by default and that’s a very good choice.

apt-get install apt-get install firefox

Install X since that’s what the xvfb launches and Xvfb itself.

apt-get install xorg xserver-xorg
apt-get install xvfb

Next I found a wiki that describes how to launch the Xvfb correctly. Log into the server and do:

startx -- `which Xvfb` :1 -screen 0 1024x768x24 2>&1 >/dev/null &

So Xvfb will run on the DISPLAY :1. So far so good. But something was still not quite right. When integrity launched the test suite that included some Cucumber-Selenium tests I received an error message basically saying that no browser sessions could be started. And the solution to that, in fact, is where this post wants to get at.

After a decent amount of head-scratching and code mining I realized that the Selenium server starts the browser on the same display where the server itself (the jar file) runs. I have found the relevant code that assembles the command that starts the Selenium server in the selenium-client gem and figured it was not meant to be run in graphic hardware-less environment since I saw no options to define which display it should run on. So as an easy hack I added the hardcoded “DISPLAY=:1” before it and crossed my fingers.

Bingo, it worked and I had a green build again! What’s more surprising is that it still runs perfectly on my Macbook so it seems to run in (some) graphical environments, too. It seems a bit strange to me to have found so little information on this subject since Rails, Cucumber, Selenium and CI are all en vogue so it is possible that I missed something obvious and there is an easier way to do all this. I am very eager to hear how others set up their CI to run automated-browser features. Do you use another tool, not Selenium? Do you use a CI server with a monitor?

Anyway, I certainly hope my hackish solution will prove to be useful for some of you TDD-minded developers out there who run their CI on a simple <name of your favorite provider here> slice.

UPDATE: As a fellow developer and the author of the selenium-client gem himself pointed out library code is not the place for enviroment specific settings. Rather, it should go into the application’s code. I guess that leaves the choice of putting it into the code of the application you are building or the configuration of the CI server. This latter seemed more clean to me so I put the following line into config.ru in Integrity’s directory:

env["DISPLAY"] = ":1"

There is still something left, though. When integrity -or, to be precise, the selenium process that was launched from integrity- wants to access display :1, it will be denied. You need to explicitly enable local connections by putting “localhost” in a file:

echo 'localhost' >/etc/X99.cfg

, and then using that file as the access records list when you launch the server:

xinit -- `which Xvfb` :1 -screen 0 1024x768x24 -auth /etc/X99.cfg 2>&1 >/dev/null &

(Note that I launch xinit and not startx as before. startx somehow adds another -auth option which messes things up)

There, that should do it. It works and there is no code where it does not belong.

A prime example of not eating your own dog food

2009-07-14T00:00:00+02:00

I realized how 20th century (plus boring and error-prone) it is to write one’s invoices by hand. So I went looking for an invoicing application. My expectations were quite low: I wanted one which can be used from my MacBook, has some basic functions that cut down on typing (e.g a client “database”), has a decent design and UI, can make out invoices in different currencies and did not cost more than ~$100.

It seemed like I have found one, which, though it had several minor bugs, seemed acceptable, so I purchased the Basic version. Everything went reasonably well until I had to make out my first invoice in euros. It turned out that the Basic version can only make invoices in Hungarian forints in spite of having a couple of other functionalities (e.g set the VAT rate to 0% automatically on invoices made out to EU companies) which made me think it could make it in euros, too.

Ok, so accounting hell got me again. I sighed and asked the company how much it would cost to have this funcionality. I was informed I had to buy the upgrade to the Business version which costs ~$120 more, roughly the double of the original price I paid! That was too much to take, so I went looking for another invoicing application. After all, it should not be an insurmountable task to write such an app, I murmured to myself.

I did find a couple of very good applications which even overdo what I want them to accomplish. However, the Hungarian homologue of the IRS basically shuts out any foreign competitors by regulating very heavily which conditions must be fulfilled by an invoicing application for a Hungarian company to be able to use it. Bummer.

“There is still a glimmer of hope.” - I thought. I checked the Hungarian scene still thinking it could not be that difficult to meet my expectations. Well, apparently I was wrong. The ones I found are either overcomplicated to use, look like home pages in the 80s or are way too expensive. Sometimes all of these. So I put off the challenge of finding an acceptable piece of software and got back of writing invoices manually.

Now you might wonder where the dog food is in the story. Well, today I received the invoice from the company of my purchase of the Basic version of the application. It was written manually on an invoice sheet :) That’s tantamount of Steve Jobs using a PC with Windows Vista.