First QCon San Francisco. Except for JAOO, QCon is the best general developer conference I’ve ever been to. Go check out the schedule at http://qconsf.com. This year I’m very excited about doing a full day tutorial about domain specific languages together with Martin Fowler.

I’m also in charge of the languages track, where I have five people who will talk about their experiences with different languages. This time there will not be much introduction to the languages, but instead experience reports, objective descriptions of what worked, what didn’t work and how you can improve your chances of success. The languages covered are Scala, Clojure, Ruby, Groovy and F#. Should be great fun.

Hopefully I will have lots of time to see other presentations too. There are many I would love to see. ThoughtWorks also happens to be a sponsor of QCon, so there will be a booth where it’s a big possibility you can find me or my colleagues.

I will do one day of RubyConf - the Saturday. Funnily enough I haven’t ever been to RubyConf, so I’m looking forward to this too.

Finally, the first ever JRubyConf will happen next Sunday. The program looks really interesting. I’m going to be talking about testing, and also be part of the ending JRuby Core Team panel.

I’m very excited about these conferences. Hope to see you there!

But a big part of it is basically just the fact that Øredev is an outstanding conference.

Some of my impressions, things I learned and did in no specific order:

• Hadoop is really cool and I wish I had time to learn more about it. Alex Loddengaard from Cloudera did a very good job introducing this technology in his tutorial. We got to do way fun stuff!
• People liked my talk about Ioke - and I was very happy with how it went too.
• Stuart Halloway is really good at introducing Clojure - I’m looking forward to his talk at QCon SF even more now.
• Me, Tyler Jennings, Neal Ford, Dan North and Stuart Halloway spent several hours of BoF time to create a new BDD framework for Clojure - this was way fun hacking, interesting from a group management and design perspective and just plain fun. There is a distinct possibility that me and Neal will give a talk at the TW US Away Day about this, if anyone is interested.
• Ze Frank is amazing. Really great evening keynote/entertainment.
• Niclas Nilsson and Hans Brattberg did a very accurate depiction of common problems and failure modes of pair programming. Good stuff.
• Tyler Jennings gave an introduction to Software Craftsmanship. Glad I didn’t miss this presentation. Very nicely done.
• Kevlin Henney did a great presentation about agile modeling. I enjoyed it a lot.
• We did a very fun closing panel that was basically just six geeks disagreeing about lots of stuff. I hope everyone else enjoyed it as much as the panel members.

Conclusion: Øredev was a great conference, I was honored to get the chance to speak there and I’ll definitely try to go back next year.

One of the discussions related to DDD that have turned up several times the last few months at conferences
is how you handle ubiquitous language when your domain is not in English. Since most programming languages are based on English, you end up mixing English and Swedish for example, if you are working with a Swedish domain. Of course, the benefits of working with these concepts in Swedish are very hard to argue against. But the dichotomy between the programming language and the domain language is definitely something that hurts my eyes, so I’m generally not very fond of that approach.

In fact, I haven’t heard anyone come up with a good solution to this problem, and this post is not really a solution either.

One of the things I’ve proposed to make this situation better is to create an external DSL that is fully in the domain language. The implementation of that DSL can then be implemented in English. The main benefit is that there is a clear separation.between the domain language and the programming language. On the other hand, the overhead of creating the DSL and also the complexities involved in translating the domain concepts into programming language concepts can become problematic too.

One interesting idea in Cucumber is the idea that you can easily add new natural languages to write the features in. When it comes to user stories at the level of testing that Cucumber provides, it’s really important to use the right language. So it got me thinking, could you use the same kind of approach in a general programming language too?

As an experiment I took a small example program for Ioke, and translated it into Mandarin, with simplified Chinese characters. Of course I used Google Translate for this, so the translation is probably not very good, but the end result is still interesting. I’m not going to try to get this into my blog, so take a look at the file at github instead: http://github.com/olabini/ioke/blob/master/examples/chinese/account.ik. As you can see there is nothing in there that even reeks of English. If you don’t understand Chinese characters it is probably hard to see what’s happening here. Basically an Account object is created, with a “transfer” method and a “print” method. Further down, two instances of this Account object is created, some transfers are made, and then the objects are printed. But provided my translation is not too crappy, this code should make sense to someone reading Chinese.

Now, this is actually extremely simple to implement in Ioke, since it relies on several of the features Ioke handles very easily. That everything is a message really helps, and having everything be first class means I can alias methods and things like that without any worry. Obviously your language also need to handle non-ascii identifiers correctly, but that should be standard in this day and age.

When thinking about it, something similar to do this can be created in languages like Lisp, Smalltalk, Factor, Io and Haskell - but most other languages would struggle. If you have keywords in your language, it’s really a killer - you would need to branch your parser to make it happen.

Of course, this approach only works when you can simply translate from one word to another. If the writing system is right to left, or top to bottom, it’s much more tricky to create a good translation.

I’m also not sure if this is actually a really good idea or not. It might be. The other thing I’ve been thinking about is how to handle multilingual editing. What if you want to be able to switch back and forth between languages? How can you handle identifiers with more than one name. Would you want to?

Lots of unanswered questions here. But it’s still funny to think about. Communication is the main goal, as usual.

The argument usually goes like this: “Dynamically typed languages are fine for smaller programs and simple web applications, but if you’re building something big, something that will be several millions of lines of code, you really need all the tools you can only get from a statically typed language”. Where the statically typed language mentioned is typically Java. The argument is still common enough without any specific language name mentioned though.

Interestingly, there are several problems and fallacies in this seemingly simple argument. I’m not going to tackle all of them here, but just focus on the beginning. Namely the idea that you are building something big. First of all, how do you know that? Have you done anything like it before? And how do you know it would be big in a different language? How do you know you can’t decouple the application in such a way you won’t have to build anything big? There are so many assumptions that can be questioned here. But at the end of the day, my glib answer that summarizes this usually goes something like this:

“Optimizing for your software project becoming big is the same as optimizing a car to hit a rock wall - you are optimizing for failure”

I firmly believe that becoming big is really failure. Once you have a big enough project you have lost. It might still work, but the cost will be extreme, and maintaining it will be a large burden too.

This is the reason I like agile. It emphasizes small, working pieces all the time. If you work with code this way, you can’t really become big. Instead, your project will be forced to be modularized and divided into smaller, more logical components that are highly cohesive and decoupled from each other.

But first, why do I need explicit iterators in Ioke? Ruby has gotten by without them for a long time, only implementing a Generator library using continuations, in the standard library. It’s pretty nice, since you don’t really need to do anything explicit to get external iterators from internal ones. Of course, the problem is that it’s very inefficient to implement them like this. So I decided that Ioke should have an explicit protocol for external iterators. You can implement internal iterators using external ones efficiently, but not the other way around.

The two major objects for this in Ioke is called Sequence and Mixins Sequenced. Sequenced is the mixin that gives you access to several helper methods if you implement the “seq” method. If you implement “seq” and mixin Sequenced you will also get an “each” method and Enumerable. The “seq” method is expected to return something that mimics Sequence and has one “next” method, and one “next?” method. That’s all. The “next?” method returns true if there is another element in the sequence, and “next” returns the next one. The protocol is undefined if you call “next” when “next?” would have returned false.

Sequenced give you an “each” method that in addition to the regular each-protocol will also return the result of calling “seq” if you don’t give any arguments to “each”.

Except for that, you will get several methods that just call “seq” and calls the same method on the result of that. These methods are: “mapped”, “collected”, “filtered”, “selected”, “grepped”, “zipped”, “dropped”, “droppedWhile” and “rejected”. These methods are also the same as exist on Sequence. These methods return new sequences that implement the same behavior as the methods with similar names on Enumerable.

Finally, Sequence also mimics Mixin Enumerable. Once you call one of the Enumerable-methods, the whole sequence will be realized, or as much as is necessary to give an answer. A small example of how you could use it:

(1..100000000) mapped(x, x*x) filtered(x, x % 3 == 0) takeWhile( < 10000 )

This example creates a range from 1 to 100,000,000 and finds all the squares that are less than 10,000 an d that is evenly dividable by 3.

JAOO this year was great, just as you can always expect. What makes JAOO so fantastic is the combination of extraordinary presentations of all kinds, together with the socializing with all the fantastic speakers, and hanging out with the JAOO crew. All in all it’s a lovely time, and I never get enough sleep for some reason.

This year ThoughtWorks was there in force - we had about 12-14 people there, and 8 of us presenting. It’s always fun to be surrounded with TW people.

I’ll not go through the whole schedule, but I do want to share some of my favorites.

Rich Hickey was there, presenting about different aspects relating to Clojure and concurrency. As usual he was excellent, and I heard many good comments about both his presentations.

Intentional Software presented their Language Workbench, which I’ve been playing around with for some time. The presentation generated substantial shock-and-awe from the audience, which was fun to see.

The Tuesday featured the concurrency track, where I spent most my time. The whole track was very good, but it was capped off by Simon Peyton-Jones excellent talk about Nested Data Parallelism in Haskell, a very good presentation that meshed well with my interests. Simon is also a highly entertaining presenter. All in all, that presentation was definitely my favorite one this year.

On Wednesday the two presentations that stands out in my mind was Aino’s about design patterns - interspersed with dating design patterns. Very funny. And the other was by my colleague Richard Durnall, talking about lean.

Very nice stuff, all in all. Martin Fowler, Neal Ford and Rebecca Parsons gave another version of their DSL tutorial the following day. It’s amazing how much this tutorial have evolved since I first saw it.

On the Friday I saw parts of Sam Aaron’s Advanced Ruby tutorial; it’s good. I also gave my tutorial, which went fairly well too.

And that’s JAOO in a nutshell. A great week. It’s weird to come back after such an intense time.

On the Friday there were five presentations, and on the Saturday it was an open spaces. The five presentations were all focused around the area of communicative programming. I talked about JRuby and did several demonstrations of how JRuby can be used to call out to different languages. My examples included talking to Clojure, Erlang and Haskell.

After me, Aslak Hellesøy talked about Cucumber and how Cucumber supports lots of different programming languages. Very cool. Aslak always give good presentations.

We then had lunch, and then Sam Aaron gave an interesting talk about communicative programming, and the essence of what we are doing. Very cerebral, definitely something that sparked lots of thoughts in peoples minds.

Adam Wiggins gave a talk about Heruko. I haven’t actually tried Heruko yet, but it looks very cool.

Finally, Matz gave a talk about the different styles of programming in Ruby, tied in with his history of creating Ruby and what the inspirations were. Very nice.

On the Saturday my colleague Dan North facilitated the open spaces discussions. I gave a 30 minute talk about Ioke - people seemed to enjoy it. After that Dan North, me, Aslak and a few others had a discussion about static versus dynamic typing.

After lunch I held a discussion about Ruby 1.9, getting some ideas why people weren’t using it, and what problems the people using it had encountered.

Finally, me, Aslak and Sam sat down to add Ioke support to Cucumber. This went really well - and I liked pairing with Aslak. Sadly I couldn’t stay until we were done, but Aslak and the others continued while I was heading out to the airport.

All in all, RubyFoo was a great conference, and I hope they can keep the same size the next time. 50 people were really a great size, and I liked the discussions we had.

For reference, the way the parser used to work was that the Antlr generated lexer and parser gave the Ioke runtime an Antlr Tree structure. This tree structure was then walked and transformed into chained Message’s, which is the AST that Ioke uses internally. Several other things were also done at this stage, including separating message chains on comma-borders. Most significantly the processing to put together interpolated strings and regular expressions happened at this stage. Sadly, the code to handle all that was complex, ugly, slow and frail. After this stage, operator shuffling happened. That part is still the same.

There were several problems I wanted to solve, but the main one was the ugliness of the algorithm. It wasn’t clear from the parser how an interpolated expression mapped into the AST, and the generated code added several complications that frankly weren’t necessary.

Ioke is a language with an extremely simple base syntax. It is only slightly more complicated than the typical Lisp parser, and there is almost no parser-level productions needed. So the new parser does away with the lexer/parser distinction and does everything in one pass. There is no need for lookahead at the token level, so this turns out to be a clear win. The code is actually much simpler now, and the Message AST is created inline in the new parser. When it comes to interpolation, instead of the semantic predicates and global stacks I had to use in the Antlr parser, I just do the obvious recursive interpolation. The code is simple to understand and quite efficient too.

At the end of the day, I did expect to see some performance improvements too. They turned out to be substantial. Parsing is about 2.5 times faster, and startup speed has improved by about 30%. The distribution size will be substantially smaller since I don’t need to ship the Antlr runtime libraries. And building the project is also much faster.

But the real gain is actually in maintainability of the code. It will be much easier for me to extend the parser now. I can do nice things to make the syntax more open ended and more powerful in ways that would be very inconvenient in Antlr. The error messages are much better since I have control over all the error states. In fact, there are only 13 distinct error messages in the new parser, and they are all very clear on what has gone wrong - I never did the work in the old parser to support that, but I get that almost for free in the new one.

Another thing I’ve been considering is to add reader macros to Ioke - and that would also have been quite painful with the Antlr parser generator. So all in all I’m very happy about the new parser, and I think it will definitely make it easier for the project going forward.

This blog post is in no way saying that Antlr is bad in any way. I like Antlr a lot - it’s a great tool. But it just wasn’t the right tool for Ioke’s syntax.

The tutorial is a half day tutorial given by Martin Fowler and me. We will talk about domain specific languages.

If this sounds interesting, go in and find more information and register here. Hurry, though - places are limited!