Cre8ive Thought

Availability

2014-06-18

I am looking to for some interesting opportunities starting from about September or October (right after summer). Which I know is still about a million years away from now. By then I am sure we will have at least 10 more javascript MVC frameworks and Erlang will still rock.

But if you have some interesting work to be done that would fit my profile then please contact me either at: email: mark.nijhof@cre8ivethought.com, twitter: @MarkNijhof or on skype: marknijhof.

Last year

I spend the last 14 months writing exciting code in Erlang and Javascript (React.js), building media encoding/decoding servers, an RTSP server and an online video editing application. All to help our customers provide real-time streaming and VOD services to their customers.

cv: link

What

I love working with Erlang, so my preference would be to continue doing that. But I realise that there are still many cool projects done in other languages as well, so this is completely open for debate. I am also quite happy to work with javascript (client/node), ruby, c# or something completely different.

Where

I have not been known for being stuck in one place. Having said that; now after working more then a year remotely from home, it is preferred over any other arrangement.

So remote when possible, for shorter engagements we can make different arrangements of course. And if the job is awesome and the location as well, well then we could definitely move. Nice weather, good food and good schools is almost all I am asking for, oh and we also love good snow ;)

Form

Initially I am thinking about doing some short term freelance work, but when the right opportunity comes along then I am very happy to commit for a long term period as well (employed or freelance).

Contact me

email: mark.nijhof@cre8ivethought.com
twitter: @MarkNijhof
skype: marknijhof

About Programming

2013-08-15

A few weeks ago I started an other Twitter account About Programming and the accompanying Facebook Page to bring you programmer insights from influential people (mostly programmers themselves).

Short 140 character sound bites, a source of inspiration and joy.

“If debugging is the process of removing bugs, then programming must be the process of putting them in”
– Edsger Dijkstra source

“The best way to get a project done faster is to start sooner”
– Jim Highsmith source

The goal

I want to inspire you to be better! After reading these types of sound bites I have often found myself thinking about how they apply to my current situation, often I think they are funny.

It is funny because it is true!

I am Dutch, so this account will have its share of sarcasm.

“Real programmers don’t document. If it was hard to write, it should be hard to understand” source

The About Programming mailing list

This is where the nicest nuggets will be shared with you, the most popular posts. But also any special offers that I can tear from the hands of programmers that have writen books, created screen casts or have anything else of value for you.

I believe that sometimes all you need is a small push into the right direction, a special deal to pickup that special book that teaches you something completely new. An amazing screen cast that shows you you should change the (way|place) you work!

These special offers will only be given in the mailing list as I will need to provide some predictability around the numbers. A tweet can suddenly be seen by millions, with a personal e-mail this is much less likely.

Signup now!

Signup for the mailing list via [this signup form](http://eepurl.com/DrdOT)

You can leave anytime you want! I am not selling your data! I am the only one who will send you stuff!

Action points:

Signup for the mailing list above or via this signup form
Follow About Programming on Twitter @abt_programming
Like the Facebook Page if that is more your thing
Don’t forget to share the joy with your friends!

Language Specification page count
C++ 865
Java 644
C 540
C# 511
Ruby 311
Smalltalk 303
Erlang 31

Well played Erlang. Well played
– @jonelf source

“Beware of bugs in the above code; I have only proved it correct, not tried it”
– Donald E. Knuth source

CQRS the book

2013-04-08

In 2009 I have had the pleasure of spending a 2 day course and many geek beers with Greg Young talking about Domain-Driven Design specifically focussed on Command Query Responsibility Segregation (CQRS).

The example project I created based on these discussions was very well received by the community and regarded a good reference project to explain and learn the patterns that make up CQRS. I decided to add the different blog posts I wrote about the example into a single book so it is easy to find and read.

You can find the book here: https://leanpub.com/cqrs

Follow me on Twitter Google Reader is dead

2013-03-14

So they terminated Google Reader; a huge pain because now I have to keep track of my comics in an other way. But I noticed that many subscribers actually use Google Reader heavily. So to them I would like to suggest to follow me on Twitter instead, to receive updates: @MarkNijhof

In the meantime have you seen my latest project: http://ls4d.com

The Amazon Experiment

2013-02-12

This is a post about an Amazon Experiment I executed on the 24th of January. It was mostly for fun, and to see how long it would take Amazon to stop it. My hypothesis was that it would at least take a few days (~3 days) for Amazon to take action.

I was only partly wrong :–)

How it all started

Opening an Amazon seller account

This was easy, actually the whole process was really easy.

Creating the “Remote Freelance Developer” product

Here is a screenshot after I had done some promotion, and received already 2 positive reviews.

Promotion via Twitter

The power of social media is amazing!

Detected!

As you can see it took Amazon only 5 hours and 19 minutes to figure out that I was doing something they didn’t like.

Actually I looked at their guidelines and couldn’t actually find which rule I broke. So, I asked, their response:

“Thank you for writing. We took this action because we restrict the listing or sale of these products to pre-approved sellers. According to our records, you have not been authorized to offer these products on Amazon.com.”

Further asking about how to become such a pre-approved seller didn’t get me anywhere.

Google search

Funnily enough if you would now search on Google for “Remote Freelance Developer” you would still find my Amazon product (in my case last result on the first page).

Unfortunately when you click on the link:

The reviews

An other curious thing is that the reviews given to the “Remote Freelance Developer” product are still accessible.

The outcome

So I was absolutely right in the assumption that this would be fun to do. I was however wrong in my assumption that it would take Amazon about 3 days to detect my experiment, they did it in only 5 hours and 19 minutes. And that is impressive actually. As a side effect my Amazon seller account it locked for 90 days, but that is ok as I don’t have much else to sell anyway.

Btw going with the saying “a picture tells a thousand words” then this is one of my bigger posts :–)

Updated RSS Feed

2013-01-05

Just a very short note: I updated the URL for my RSS feed, please subscribe to the following URL to keep getting my updates: http://feeds.feedburner.com/Cre8iveThought

The reason for this is that there is no way anymore that I can login to my previous dashboard :( Google seems to have forgotten about me.

Thanks and a Happy New Year!

Actually, People Love Predictable Change

2012-11-08

I just read an interesting post by Tim Kastelle claiming that Actually, People Love Change. In his post he has some interesting points but I am thinking that he may be oversimplifying the experiment a little bit.

I think that predictable change is something that people love, f.ex. the questions Tim asked were not really radical in itself.

Getting married is a very predictable event and change -at least that is what we expect- many people around us do it and we can see the results of it. The same goes for getting children. You know what you are getting into -who are we kidding here, really- but again it is not something radically different. Actually these things are very accepted and even expected -right mom- changes. So because we see it all around us it becomes a much more predictable change, even though they are huge changes by themselves.

Now about changing jobs; I think this also has a lot to do with the people Tim asked. In our industry it is expected that you switch between jobs, it increases your knowledge and experience. I.e. it makes you more valuable. It also is very easy to change jobs, actually changing a job is not that much of a change. Most often you would be doing very similar work, yes there would be different people and a different location, but nothing radically changed.

Moving to a different city is also not that hard for people in our industry, we can pretty much find the same type of work where ever we live. Being able to work and get an income is a very reassuring thing to have, making the change of location much less scary. So changing cities becomes much more predictable if you already know of have a job that you feel comfortable doing.

Now moving between countries is a much bigger change and I was surprised to read that half of his audience had indeed done that. So I was curious were Tim lived or held the speech. Things became a lot clearer when I saw most of his work is done in Australia. I love that country, many other people love that country, so no surprise that many people have moved there. Especially from our industry because we actually can get a visa.

Now only expecting your life to improve by moving to a different country is in my encounter with many people not enough for them to take the step. I have met many people that told me they would have loved to live in country X, but never took the plunge.

Now I believe better questions to ask would have been in the form of:

How many people have moved to a different country where they didn’t speak the language?
How many people have actually changed the type of work they did while changing jobs?
How many people tried food from a very different culture?

Tweak the System

2012-08-13

“Tweaking refers to fine-tuning or adjusting a complex system, usually an electronic device. Tweaks are any small modifications intended to improve a system.”
-Wikipedia

I work as an Agile Coach currently at Avira, our goal is to faster deliver valuable software to our customers. So as an Agile Coach I try to tweak the system in which we work.

In the past I have been more focussed on optimizing the parts that I saw and could optimize; but I have never really tried to understand what the effect of these optimizations would have on the whole system. Thankfully I am also being coached here by Vasco Duarte who has already enlightened me (a bit) with his Meta Coaching exercises (more on that in a next post).

So now instead of optimizing parts of the system I now try to see what effect such optimizations would have on the whole system, not just to the part being optimized.

Personally I am a big fan of visualizing things using graphics, so together with my college Luis Gonsalves I thought of a small example of how I would like to visualize the throughput (velocity if you will) of a development team.

The team

The composition of the team is rather complex, we came from an environment where different expertise’s (like development and testing) where setup in different departments and they formed the well known silo’s. In an effort to minimize communication overhead and many other problems we now have feature teams which are composed out of people from the different departments. A team as such is cross-functional but the team members are not.

So one team is composed out of:
– 1 Project Owner
– 1 Project Manager (comparable with a Scrum Master role)
– 5 Developers
– 0.5 Localization (shared between teams)
– 0.5 Documentation (shared between teams)
– 2 Manual testers
– 0.5 Test Automation (shared between teams)

Then when we look at what each mini silo can do within a 2 week iteration. I am not tracking the PO here as there should be a significant backlog and the PM is a facilitator and as such he/she does not produce output. The developers can finish about 8 stories during an iteration and the same applies for the documentation people. However the manual testers and he localization people can finish about 6 stories per iteration. And finally the Test Automation person can automate the manual tests for about 3 stories per iterations.

The number of stories per iteration:
– 8 for Developers
– 6 for Localization (shared between teams)
– 8 for Documentation (shared between teams)
– 6 for Manual testers
– 3 for Test Automation

When this team selects their stories they mostly use the 6 story baseline ignoring that Test Automation cannot finish all the work.

Visualization

Of-course when looking at it like this it is quit easy to pinpoint a bottle neck, but what I am looking at achieving is a way to make this process even quicker.

So I want to draw a picture of the system as we see it (of-course this is still only a small part of the whole system in which we work).

What you see here is a quick draft of the throughput of this team, now you can see in one quick look that the biggest bottleneck lies with Test Automation. They are only able to do 3 stories per iteration.

Now depending on how many stories get selected we can easily calculate and show where people are waiting and where a queue with unfinished work is formed.

In the case where the team takes 6 stories we can see that the Developers are waiting 25 percent of the time and that there will be a queue of 3 stories that Test Automation was unable to finish.

Variables

What if we could change some variables from the above setup?

Having the Developers wait for 25 percent of the iteration is of-course not good. We could have them do 8 stories instead, but that would only increase the queues for the others.

What if instead of waiting we ask the Developers to spend the 25 percent on writing Automated tests that normally would be defined by the Manual Testers and then automated by the Test Automation person?

Let’s say that the Developers are able to create about half of the tests the Manual Testers would define. That would mean that potentially the velocity of the Manual Testers gets increased by 50 percent i.e. they would be able to do 9 stories per iteration.

But since the Developers have automated these tests also the Test Automation person has less tests to automate, in this case he or she only has to automate the remaining 50 percent of the tests (the manual tests) for each story which would increase the number of stories he or she can finish to 6 per iteration.

When we now take a look at the second picture we can clearly see a big difference in through-put. Suddenly the whole team would be able to finish 6 stories per iteration instead of 3. It may be true that the developers are not producing the optimum output they could achieve which would be 8 stories per iteration, but the system as a whole benefits greatly from this by having the output doubled.

The question

My thought is; being able to show with visualizations what the impact of certain changes would be, like having the developers write automated tests, will make it easier to understand for most people.

You may have noticed that I ignored the order in which the work happens, but this could also greatly impact how a team works.

What other visualizations would you use to tell the story?

Just explain it to me

2012-08-03

Today I was sitting in the office enjoying the view from the window; there were some ducks playing in the pond outside when suddenly the door opened and a guy came walking in.

He started talking -his voice sounded slightly annoyed- about some problem he had. So he started explaining it to me. Honestly at that time I had no clue what he was talking about, but I just let him continue as it seemed to matter very much to him. Perhaps after hearing more about it the problem will get clearer to me. So the guy continues to talk to me for another 5 minutes.

I got distracted as there was some sort of commotion at the pond, I think it was a cat or something, it went too fast for me to clearly see it.

Then suddenly I was brought back to the here and now, the guy had abruptly risen from his chair. I thought he was upset about the lack of response I had given him to his problem. But when I looked him in the eyes I saw he was happy, actually he was thanking me for helping him.

Then he just walked out of the room and left me wondering what this was all about. Strangely this sort of thing happens to me all the time.

I looked out the window again, the commotion at the pond had settled down, I wish I could be there as well.

-Rubber Ducky

Sometimes all what is needed to solve a problem is to explain it to someone else

The art of communication

2012-07-05

Communication is vital for anyone to feel included, functional and from my perspective alive. It is one of the very first skills you learn as a human being. As a little child you quickly learn how to communicate certain needs or wants. I am still amazed how our youngest boy was able to let us know exactly what he wanted without having to say a single word. It also depends on the child, he was certainly better at it then his brother and sister, or perhaps we became better at it as parents. Anyway after the basic instinct of crying/screaming children learn to communicate before much else.

And for a skill learned so early in life it stays one of the hardest things to master, perhaps it is something we forget as we grow up when it seems to become less important.

Personally I have found a new interest in the art of communication. A few year back while working for Bekk we received a course about communication, public speaking and body language. This was extremely interesting, we practiced conversations in the form of small presentations, after which we received feedback from the other course attendees. It should be something that you know and preferable are passioned about. We then look at the body posture/language and how you come across. Are you making eye contact with the people in the room or only focussed on one person. How does that reflect on the others. Do you actually see and respond to signals from the audience. We also learned about “reading” people, what is their body language telling us about their engagement in the conversation. I think it is admirable for a consulting company to provide such courses, it clearly indicates they know most problems are due to misunderstandings.

Misunderstandings

For example when during a code review I say: “this code could have been extracted into different functions”. What does that mean? Depending on who the other person is there are different interpretations of what I just said. I.e.; I think you are an idiot because only idiots write code this way. Or am I trying to be helpful by suggesting improvements to the code.

Now how am I going to know how the other person in the conversation understood my intend with the comment? It can be very hard to extract this. In my experience the more negative the message was received the harder it becomes to understand that it was received in that manner. People, at least in most cultures that I have worked in, don’t like to confront other people. So the other person is probably not going to say; “So you think I am an idiot for writing the code this way?” or “I don’t appreciate you calling me an idiot”. Or for the lesser extreme; feeling dumb (but very common); “I am sorry for being so silly by not thinking about that”.

But everybody is telling you how he/she took the comment by their body language. This language has different nuances depending on the person, learn to pay attention to these things. For example; someone hanging back in their chair could be less interested in the topic then someone sitting up straight or even leaning towards you. Is the body in a defensive position or vulnerable position. Once you learn and start to pick-up these signals also try to think about your own body language, it does make a difference.

It is important how your message is received because if it was received wrongly then you probably get the wrong results. i.e. if my intend was to help the person improve the code, but he/she thinks I just called him/her an idiot, how big is the chance he/she will be open for suggestions from me. Then and later.

So it is important that you can deduce how the message was received, but it is even more important that you think about how the message could be received. For example instead of saying: “this code could have been extracted into different functions” I could say something like: “I think we can improve the readability of the code by extracting it into different functions, what do you think?” or “I find it difficult to understand the code, could we extract some functionality into different functions to make it show more intent?”. We arrive at the same destination but the way towards it was completely different.

Interpretation

Think about the person you are talking to and try to understand where they come from, what their values are, what makes them tick. But also think about how they might see you, are you a threat to them, just annoying or interesting, pleasant to be around with. This all influences how your message is received.

Someone who is naturally defensive will need an other approach then someone who is open to being wrong, but it also depends on how they think you think about things. What if I am talking to a person in favor of a technology and they think I am not in favor of it while we are discussing that technology. It then is nearly irrelevant of how I actually feel or how I express my ideas, they will most likely be seen negatively. I don’t know how to solve this, but it is good to at-least understand this.

The next steps

I now got an other boost in this topic, last week at Avira we received a course from Ben Fuchs about Non Violent Communication and it was awesome. Ben has a degree in Psychology and has even done couples counseling, but now he focusses on Management Consulting. During the course we learned about the different ways we can influence people and divert conflicts.

Influencing

Many discussions have the purpose of influencing the other party in the discussion, and this usually goes both ways. Think about it, have you ever been in a situation that goes like this: “You should use tool XYZ”, “Yeah it is awesome”. Once you agree the discussion is pretty much over.

You might as well have talked about the weather; “It is nice and sunny today”, “Yeah it is great”. The discussion quickly dies and then what. Now on the other hand we could have; “It is nice and sunny today”, “Nah I like rain”, “You do what? Don’t you like the sun?”, “No I prefer the rain it is much nicer”. What happened here is two people not agreeing with each other and trying to convince the other party of their point of view. A discussion like this can take forever.

So if it is about influencing other people then there are two main differences of how you may achieve this; you have Pull and you have Push.

Pull is the least confronting approach, it is about listening and asking questions. Trying to understand the other persons position and empathizing with it.

“Why do you like the rain?” “I like the sound it makes” “You like to sound of falling water drops?”
“Yeah”
“Do you like walking in the rain as well, getting all wet?”
“No that not so much”
“What do you feel about the temperature drop whenever it rains?”
“I’ll take it for granted, it is part of the rain”
“So you like the sound rain makes, not so much the cold feeling? Why do you feel that way?”
“Yeah, It always relaxes me when it rains”
“Could it be that you feel more relaxed by the sound of rain because it takes your mind of the daily stress?”
“Maybe”
“Do you like listening to ambient music as well?”
“Yeah I like that as well”

Maybe here I have not yet persuaded the other person that is it not so much the rain he likes as it is the calming sounds of falling water drops. I ran a bit into a dead end with this conversation as I find it hard to imagine why someone might like the rain. But at least now I understand the other person much better. Maybe if I wanted to I could use a little bit of push to convince him about my stance.

Push is about actively advocating and speaking about your point of view, trying to persuade the other person en engaging him in the conversation. I could continue the conversation in the following way:

“Have you tried sitting in the sun with some relaxing ambient music playing in the background? This could give you the same relaxed feeling without the temperature drop that rain brings”
“Hmm that might work”
“An other benefit is that if that works then you can get the relaxed feeling whether it rains or shines”
“I guess you are right about that”
“I like that it is nice and sunny today!”
“Yeah”

This was of-course an extremely silly example, but I wanted to highlight the difference between Pull and Push. Pull will increase my understanding of the problem much better, instead of thinking that this person is a bit crazy for liking rain over sun I now understand the reasoning behind it. And that is valuable whether or not I managed to convince him/her about my point of view. Because now I can use this information in following conversations. I now may know to not bother him/her when it rains as this is a moment of relaxation/reflection for him/her.

Asking about the other persons feelings, their needs and values is a good way to understand the real reasoning behind an opinion. Next time you are in a discussion with someone, just try to understand where the other person is coming from, this understanding may be all that is needed to come to a neutral understanding and perhaps even agreement.

Now I am only a beginner at this, so next time you see or hear me discussing something without understanding the other side then feel free to point this out to me.

New Job, Agile Coach

2012-05-16

On the 1st of June I will start with a new job in the south of Germany to help transform the company into a more Agile way of working, the title I got there is Agile Coach and I am the second one they have employed. The company is Avira the free anti-virus company also one of the top web-companies in the world with 100+ Million customers.

This new role is a huge change from what I have been doing so far; instead of writing software I will get a facilitating role helping others write (better) software. Initially I will have a strong technical focus as that is where most of my knowledge lies; this is also what is really needed right now. But I will gradually expand my knowledge to be able to help out on management levels as well.

I have long been dreaming for such a job where I can really focus on helping others and now it has come, thanks to J.B. Rainsberger for the introduction.

Some of the tasks I expect are:

Help developers with the transition to Agile
– by pair programming
– by implementing different processes like proper CI
– by helping write good tests
– by doing (nearly) anything that helps them improve

Help with creating an internal community
– to make people enthusiastic
– to make daily work fun
– to facilitate knowledge sharing
– to enable outside of the box thinking

Help non-developers with the transition to Agile
– the whole organization needs to change, this will be strange for all of them, this is also a new area for me

I also expect that I’ll be blogging more again.

I will be working with my colleague Vasco Duarte who has been working in this role at Avira for a while now. I expect to learn a lot from him while working there.

I will bring my family as well (as of 22nd Cattleya has joined the ranks as #4) as we will move to Friedrichshafen right at the Bodensee. And this also completes one other goal I had in life; move to a better climate. All though this is not Spain or Australia; it is nicer then at home and still close enough for frequent visits.

Re-design

2011-09-15

I re-designed my blog again; the last time I wanted better readability and more focus on content. And this time it is no different, again I wanted the experience on my blog to be about the content. It should be easy to read and no distractions.

I got inspired by IA Writer which is a really good editor (on the Mac) to write text. Its whole philosophy is about focus on the task of writing text and no other distractions. What I did is port that feeling to my blog, let me know what you think.

Software Development is not a Jenga game

2011-08-23

This is an extract from the talk “Software Development is not a Jenga game” Republished from 16 May 2009 because I had some requests for it.

Does developing software feel like you are playing a game of Jenga? Is making a change to your code very complicated? Are you afraid that making this change in your code will break something unexpected elsewhere in the code? If so then that is a clear sign of bad code, bad code will slow you down, not only months from now, but even days from now, have you ever written code that you didn’t understand the next day? I have been watching many presentations by Uncle Bob and I really like the part where he asks his audience if they have ever been significantly slowdown by bad code, and after everybody sticks up their hand he asks “So why did you write it?”.

To address these issues we have to change the way we write code, we have to think ahead and write code that not only allows change, but we have to write code that expects change. Write it like you will change it again tomorrow. And to help the developer write such code there are several different patterns and principles available; SOLID is an acronym for 5 different principles which are very important to design code that greatly improves the ability of your software to cope with change.

Apparently Uncle Bob felt to strong about the order of the principles that he didn’t see the SOLID acronym himself, Michael Feathers was the one actually proposing it.

Quote from Tim Barcz

I feel that this is the best order in which to explain the different SOLID principles, mostly because the principles are so integrated with each other. I have been doing the presentation a few times internally and each time I struggled with the SOLID order. So here we go S DOLI

Look at the beautiful knife that can do it all, until one item breaks and you will need to replace the whole knife, one broken part affects the whole module.

Single Responsibility Principle is the first principle in SOLID and states that “There should never be more than one reason for a class to change”. Something with one purpose, one responsibility is always going to be easier to change, purely because you can much clearer see what it does and what it does not. Also because there is only one responsibility there is no chance that the change in the code will affect other code, behavior perhaps, code no. And it greatly improves reuse of code. Finally it makes you code so much easier to test, since you can test one responsibility in isolation of the rest.

Look at the different responsibilities in the OrderProcessor class, there is: Composition of the order process, saving the order to the database and sending a confirmation message. So quickly these are three different responsibilities. So there are three different reasons why this class could change.

As you can see we have been able to separate the different responsibilities into three different classes. One look upon each class clearly identifies what the class should be doing; it also enables reuse of the different responsibilities.

Not as painful as it looks, not even close

Dependency Inversion Principle is the fifth and last SOLID principle and states that “High level modules should not depend upon low level modules, both should depend upon abstractions” and “Abstractions should not depend upon details, details should depend upon abstractions”. Wow, well basically this means is that nowhere in you code you should depend upon an actual implementation; instead you should only depend upon interfaces or abstract classes. And why you should do this is because this enables you to just change any actual implementation with another implementation and none of the other code knows or cares about it. Yes sir, your other code does not care.

In this example you can see that my OrderProcessor is depending upon actual implementations (details) of the Repository and MailSender. So if I would like to change the way I send notifications to the client I would create another class SmsSender and I would have to change the OrderProcessor as well to be able to use it, which is bad, because I suddenly made the OrderProcessor aware of how it should send messages.

Now the OrderProcessor has no dependencies upon actual implementations anymore, it only relies on interfaces; IRepository and IMailSender. So if I now want to change the way notifications are send I only have to create a new implementation of IMailSender and provide that implementation to the OrderProcessor, OrderProcessor would simply continue to work as it is, its behavior has change, but it doesn’t care about that, it only cares about its own specific logic.

For a hairdresser she is pretty much closed for modification but you can give her a wick which makes her open for extension.

The second SOLID principle is the Open Closed Principle which states that “Software entities should be open for extensions but closed for modifications”. This basically means that you should be able to change the external behavior or external dependencies of a class without having to physically change the class itself. You would want this kind of behavior because this enables you to make a change in one part of the code without you having to change other parts of the code as well as long as you can work within the boundaries of the existing contract.

You would recognize this class, it is the result of the Dependency Inversion Principle, and this class is already adhering to the Open Closed Principle, if you want to see a class that doesn’t go back to the start of the Dependency Inversion Principle. So does that mean that the Open Closed Principle can be achieved with Dependency Inversion? Yes that is correct, but that is not the only way you can make a class extendable.

As you may have seen in the previous slide I added an interface on top of the OrderProcessor, the reason for this is that I can now easily create another implementation with the same contract. Then I created a decorator class DecoratedOrderProcessor which adds behavior to my original class without having to modify the code of OrderProcessor. I can even add this behavior during runtime to my class. Another great example of a similar construct is how behaviors work in FubuMVC.

The garbage truck doesn't care about the color of the containers, as long as they all are opening from the top and not from the bottom.

The third SOLID principle is the Liskov Substitution Principle which is described as “Functions that use pointers or references to base classes must be able to use objects or derived classes without knowing it”. That means that when our code uses a specific class or interface it should be able to use a derived class or different implementation of the interface without having to change its internal behavior. This again is to minimize the impact change will have on your code, it feels like DÈj‡ Vu.

Look at how PriorityOrder is violating the Liskov Substitution Principle because when a client of Order is checking IsValid it expects a Boolean, but when a PriorityOrder is passed to the client suddenly the client needs to try and catch the potential exception that is thrown when the order is not valid. So suddenly the client needs to know with which specific type it is dealing, i.e. Order or PriorityOrder, which is violating the Liskov Substitution Principle.

This is an example that Uncle Bob uses himself, and as you can see that the Square class is providing a completely different implementation to setting the Height and thus all clients of Rectangle that assume the surface is calculated by multiplying the Width times the Height are in for a nasty surprise.

Don't tell the doctor everything only that what he needs to help you feel better, hmm depending on the doctor that could include telling him your live story

Interface Segregation Principle is the fourth SOLID principle and states that “Clients should not be forced to depend upon interfaces that they do not use”. Which means that when an object has different usages, not to confuse with different responsibilities, then there should be a specific interface for each of these usages? And I am not going to repeat myself by saying that this is to minimize the impact change will have on your code.

Consider a simple web shop where customers can add and remove items out of their shopping cart and of course inspect the item currently in there. After that they go to check-out and are asked to fill out their personal details, and the final step is a review of the complete order and personal details on which they can submit the order. Now here are three different usages of the order, not every step needs to have or want to have all the information the Order can provide.

Now we have created specific interfaces for the specific steps in the order process. The Order object still looks exactly the same as it did in code, but the different clients have a completely different view on it. This enables you to also create different objects one for each step implementing only the needed interface.

The programmer's magic hat

Doing Dependency Inversion without an Inversion of Control container is going to be a pain. You will end up with actual dependencies in your code be it not in the injected classes but then at a higher level in the code. So you are still in a bad place, this is where Inversion of Control containers come into place.

Just take a look at this code, as you can see the different new statements are still needed at this level. BrrrrÖ

Hey that looks much cleaner, and I mean much cleaner. What I show you here is the usage of the Common Service Locator which is a common interface that all the different Inversion of Control containers support. So here is also no dependency upon a specific Inversion of Control container. A proper Inversion of Control container can automatically resolve any dependencies a requested class has and inject them in the newly created class. In other words it automatically does what I just did manually.

It would not be fair to leave you hanging there; you will have to actually configure your Inversion of Control container. And this is specific for the Inversion of Control container that you use. In this example I am using StructureMap. The beauty is that this configuration is the only place in your code where you define your actual dependencies; this is the only place in your code where you have to know about the different implementations. Most Inversion of Control containers also support configuration via Xml but I would advise against that as you would lose your type safety and refactoring support.

Test Driven Development or Test Driven Design is a technique that really enforces you to do proper design. Writing your tests up front is really helping you think about your design and about what you want to accomplish, so any “lost” time in creating these tests are surely won back by the better design that you get from it. Oh yeah besides the fact that your code is being tested of course which in case will help you deal with change more easily in the future. There really is no good excuse for not doing proper testing. Testing is done in any other industry, so why not in Software Development?

Don’t Repeat Yourself, I have said it many times before but when you have the same functionality multiple times in your code, then you are risking that when change comes you forget something. And it is not just that code that you have duplicated, you also have duplicate tests; it makes your code fragile. Don’t Repeat Yourself, damn just did it again!

You Ain’t Gonne Need It, it happens so often, you are working on some code and think, hey this is something they want need later on; I’ll just add it now since I am working on it anyway. Then when you get to the point that you have a better understanding of what they actually need you’ll realize that what you had created is not it. So instead of saving the time by doing it when you were working in the same context, you lost the time because you have to do it over again. When you have designed your code properly, adding the functionality is not going to be a problem anyway. Expect Change right!

This is so true, and even truer is the fact that you will probably be your own Psychopath. So be kind to yourself and start producing quality code.

Keystoning Features

2011-07-11

Today I wanted to explain someone what I learned a while back on a course “Software G Forces” with Kent Beck in Bergen Norway about deploying software. Going from yearly to monthly to weekly to daily and finally to hourly and perhaps even continuous deployment of software takes a lot of discipline and good techniques to get right. One of the difficulties is that when you deploy your software very often you will possibly have features that are not yet completed in each deployment.

^{I wanted to send a link, but no such link could be found, so here we go.}

Incomplete Features

We can all agree that we do not want to confront our users with incomplete features, this will only confuse and annoy them. So how do we build our new features without our users becoming annoyed, because annoyed your users is a bad thing (just so we are clear).

Feature Branches

One way to accomplish this is to use feature branches, this means that each new feature is build and completed in a separate branch. Once it is completed the feature branch will be merged with the mainline/trunk/master and the whole things is deployed to production. Of-course after passing all the tests.

Now it turns out that this is a very difficult process to get right; Sarah Taraporewalla coincidentally wrote a really good post about that yesterday (also read the comments).

What it basically comes down to is that because the longer the code is away from the source of truth (mainline/trunk/master) the more risk you carrie of getting merge conflicts and thus the loss of time.

Sarah then also mentions the term feature toggles which is a way to “hide” functionality from your users by a toggle, so when it is disabled the user just doesn’t know it is there. But feature toggling does actually not say anything about where the code lives while it is being build.

Keystoning

An other way of building new features in your software is to build them right into the mainline/trunk/master, but use the principle of keystoning to only expose this new feature at the last moment when it is done.

A keystone is the final top stone in an ark keeping both sides from collapsing. Once an ark is created it should be self supporting i.e. done.

So lets project this idea to our world of software development. Imagine that you are building this new feature, stone after stone you are building this ark. Each stone added is in the code and will potentially be deployed, so you are making sure it does not break any existing functionality. When you are done implementing your two sides of the ark then you place the final stone, the keystone, completing the ark and exposing it to the user.

So until you place the keystone all functionality you have created is invisible to the user, as if it is not even there, but it is and that is the secrete, no more merge conflicts. All code ships.

So keystoning is basically the same as feature toggling? Well I would say yes, except that feature toggles can be used for different purposes. You may use it to do A/B testing or to temporarily disable some functionality.

Keystoning is all about how you add new features to your code, once a keystone is set you won’t remove it to disable the functionality temporarily because this would collapse your ark. So if you want that then you will need a feature toggle as well.

Client Express - Require and Exports

2011-07-11

The RSS reader you are using doesn't support JavaScript; because of this you will not see the embedded code gists. Consider opening the post in your browser instead.

I recently introduced you to Client Express where I explained that the goal is to copy the API of Express (Node.js) but for use on the client. This will enable you to share code between the server and the client, specifically the routing.

And I got a pretty decent part done when I blogged about it, but that was one major difference and that was that on the server you use require to include different JavaScript libraries and within these libraries you would use exports to export your functionality to the application.

require(‘…’)

Below here you see two common uses of require, the first one require(‘express’) is to add the Express module and the second one require(__dirname + ‘/examples/server_routing’ ).example_routing() is to load a local JavaScript file and execute the function example_routing() which is defined inside this JavaScript file.

exports….

Here we see the local JavaScript file that was loaded using require in the above example. Important to note here is that we see how exports is used to expose the function example_routing() to the user of the library.

And now in the browser

Well both require and exports do not exist in the browser, so we need to add them. There are already different approaches that enable you to use require I believe all of them are an implementation of CommonJS so I added my own implementation for that to Client Express and after some tuning and a pull request I got that to work nicely.

The thing I didn’t see was how people solved the exports problem. And because I didn’t want to stop halfway I started thinking and came up with the following solution. My require will try to download the requested JavaScript file if it cannot find the module in the registered modules. When it is downloaded it will then do something EVIL it will do the following:

That’s right I am using eval to evaluate the JavaScript text found in the downloaded JavaScript file and attach the exported functions to my exports object. After the eval my exports object is then placed in the registered modules so this process only happens once. After that the exports object is returned from the require function exposing the functions defined in the just downloaded JavaScript file.

This enables me to use the exact same syntax on the server as on the client for both require and exports. See below here the full implementation:

Now as you can see I am always defining these functions even if an other library has defined them already, but I do copy the registered modules from those other libraries.

One problem is when my CommonJS implementation is overwritten by some library loaded after my Client Express because then I would loose the download and eval capabilities. So for the time being my library needs to either be loaded last, or the other libraries should check for existence of the require functionality and if it exists don’t overwrite it.

I am however thinking about a way that overwrites it again after the Client Express server has started ensuring it will work when during runtime the code requires additional libraries.

Please let me know what you think.

Client Express

2011-06-25

The RSS reader you are using doesn't support JavaScript; because of this you will not see the embedded code gists. Consider opening the post in your browser instead.

I have been very intrigued by Node.js since I heard about; just think about it, using JavaScript on the client and on the server as well. How cool is that. But I have never really done anything with it, that is until Heroku announced their Node.js support. I know it is a weak excuse but I just can’t see myself maintain an other server. I have had my own server from about 2000 till about 2009 which is when I got into Ruby and Heroku. Since then I just don’t see the benefit of doing it myself for my own projects (different for big production systems). It means less freedom but also less headaches.

So Node.js, well it’s freaking awesome I can tell you. In Ruby land I have really enjoyed working with Sinatra, now in Node.js land this is Express. Not very surprising but Express is heavily influenced by Sinatra. Both a real pleasure to work with.

I mentioned Sammy.js before and how cool that was, well there is one problem, you absolutely need JavaScript and while this is pretty ok for most usage’s I still wonder why the h Twitter decided that this was ok. Anyway there is a nicer way and that is called pushState. Not too long ago I found Davis.js which is sort of a copy of Sammy.js but instead of using onHashChange it uses pushState and onPopChange to enhance the user experience.

So the idea is that you implement all client routes on the server as well, then when a route is triggered directly (i.e. not after having loaded the JavaScript code) then the server responds, but when the JavaScript is loaded then the request is being handled on the client instead. But the key is that all routes that are visible to the client are accessible on the server as well. Then you can progressively enhance the user experience using JavaScript.

So I wanted this as well, but I did not want to copy the routes from the server then go and change a whole bunch of stuff in order to make it work in Davis.js, no instead I want something like this: Client Express (http://clientexpressjs.com/).

Server.js

Below here is a very simple server implementation using Node.js and Express:

Client.js

And here is the same implementation on the client but now using Client Express:

Client Express

Now that looks pretty cool doesn’t it? I decided to duplicate the Express API for use on the client and also using the pushState stuff. This enables me to re-use much of the same logic that I use on the server on the client as well. Currently it means actual duplication the code, this is until I find or need a better way fine by me. Once thing is clear, I can duplicate it faster now then I could before ;–)

Let’s take a look at a more complicated example, in fact this is the code that is running on http://clientexpressjs.com/.

Server.js

Client.js

Server_processing_url_and_post_parameters.js

Client_processing_url_and_post_parameters.js

I hope you will take look and provide me with some feedback. This is only a few days of work so there is going to be some code clean-up and making the API more complete, but so far this is working very nicely.

For more details and working examples go to http://clientexpressjs.com/ and browse around (you might notice a big similarity with the design of my blog). Note the upper left corner where it states where the request was being handled.

Hmmm I feel a blog engine rewrite coming up :P

Open Spaces

2011-05-21

I am helping organizing ROOTS this year and part of this years conference is that we will have two sessions with Open Spaces right after a few sessions with lightning talks. And I will be facilitating that; doing my best to make it a highly interesting experience.

Open Spaces are an approach of having a meeting or a conference where there is no predetermined agenda established. Instead the attendees themselves will decide what they want to be discussing and as such it is a self organizing event.

But in-order to make this really successful we do need some guidance; because I fear that if we would truly self organize we would all end up in the pub.

An Open Spaces event has the following pseudo agenda:

Introduction (short)
Market place (about 30 minutes)
Meetings/Discussions (from 30 minutes to an hour and a half)

Introduction

The introduction is basically what I am describing here; it is an explanation for people new to this concept so they can fully benefit from the experience. Should take as little time as possible.

Market place

The market place is exactly what it sounds like; this is the place where you try to sell your idea to the group by shouting about it.

Ok ok we will be a bit more disciplined and give you an opportunity to introduce the group with the topic that you would like to discuss. This is after you have written it down on a post-it and told the facilitator that you want to sell something. Then finally after your introduction you place the post-it on the board.

The topic

It is important to know that the topic can truly be anything that you want to discuss; a help request, something you want to learn about, something you want to discuss, something you want to share or everything else. So if you want to validate a technical idea that you have or perhaps want to hear different approaches on how to deal with specific behavior of people everything is allowed and all you need to do is write it on a post-it and introduce it to the group.

You will be the facilitator of your own topic discussion and it is expected that you are passioned enough about it to be able to start the discussion.

So you feel something is worth discussing, come forward and sell it in the market place! Hint: when going to an Open Spaces event start thinking about a topic well ahead of time.

The post-it

Make sure that the post-it contains the title or very short description of your topic and your name.

The board/Tracks

The board (or wall on larger events) is divided into different tracks, the number of tracks completely depends on the number of attendees, but I think you should have at-least three simultaneous tracks to offer enough diversity.

So once you need to put your post-it on the board, pick a track and time where there is space. If you already see something on a particular time that you want to join yourself then don’t put your topic on the same time, since you will be needed in your own topic discussion.

If there are two topic discussions at the same time that you want to join, you could then ask the two facilitators if either one is willing to move their topic to a different time (if at all possible). If they for whatever reason do not want to do that then you just have to make a choice between the two.

Meetings/Discussions

After the market place you may want to check the board to see when the topic discussion you are interested in is starting and in which track. Once you have decided just go there. If you are a topic facilitator then you have to be there on time for your track to start, other people (hopefully) are joining you, so give them the respect of being on time.

Setup

The different tracks are setup to accommodate discussions, usually the chairs are arranged in a small circle allowing everybody to see and hear each other clearly. Often different tracks will be in different rooms, except for big events where there is no space.

If the circle you are sitting in feel to big or small then change it, don’t keep sitting in a uncomfortable layout.

The law and principles

There is one law and four guiding principles that help you getting the most out of the Open Spaces experience.

^{The amazing hand drawn graphics are done by Claudio Perrone further I am taking much reference material from Harrison Owen’s website about Open Spaces Technology.}

Law – The law of two feet

“If at any time you find yourself in any situation where you are neither learning nor contributing: use your two feet and go someplace else” – Harrison Owen

This law is there to make perfectly clear that you yourself are responsible for your learning and contributing. It means that once you feel that you have nothing more to learn or add to the discussion that you should move on to a better place. This can be a different discussion or even catching some sun outside. No matter what, don’t sit there feeling miserable, because unhappy people are unlikely to be productive people.

So this is expected behavior and thus not impolite, and as a facilitator you should not take it personally.

Principle 1 – Whoever comes is the right people

It is better to have one person to show-up to discuss something with you then to have 20 people sitting there without adding to the conversation. So whoever is there cares enough about the topic to be there to discuss it with you.

Principle 2 – Whenever it starts is the right time

This is just to give a reminder that it is an open and creative process, and nothing will kill such a process more then strict control. The people will come and start the discussion when they are ready.

Principle 3 – Whatever happens is the only thing that could have

What’s done is done, so no need to complain about it, just move on to the next thing.

Principle 4 – When it’s over, it’s over

Once a discussion has come to an end then don’t worry about the clock, it’s better to use that time useful in a different way. Do the work not the time.

Finally

In fact when you think about it an Open Spaces event is not that different from being in a pub with many other geeks/professionals. The satisfaction I get from a good Open Spaces event is the same as that from a interesting discussion during a geek beer.

Now instead of thinking about a good topic to discuss during the market place I suggest that when going to an Open Spaces event you start thinking about a topic well ahead of time.

And in the case of ROOTS you are very lucky to have many lightning talks before the Open Spaces sessions, if there is a lightning talk that you want to discuss further then do it during the Open Spaces sessions.

OSS; my Code, Yes but Not My Secret Stuff

2011-02-01

The RSS reader you are using doesn't support JavaScript; because of this you will not see the embedded code gists. Consider opening the post in your browser instead.

Using Ruby, GitHub and Heroku is absolutely a great combination to work with, the easy way to ‘push’ something into staging or production is just wonderful. GitHub is also a great place to share code and get feedback on it. For this reason I rather OSS something then keep it in a private repository at GitHub.

But I don’t want the whole world to be able to see my database credentials or the different API keys that I am using for my projects.

And for this I found a dead simple solution which I’ll share with you here, and of-course hope to get your feedback on!

Currently

Lets just take a look at a Rails project; there are many different ways to define your credentials and other configurable information. We can for example use YAML::load to load configuration from a .yml file, many gems are familiar with this way of working some even search for a .yml file themselves.

An other way is to assign Environment Variables in the different environment configuration files from Rails, like assigning ENV[‘mongodb_password’] in /config/environments/production.rb. These environment variables are then used in the different places in the code, like my MongoDB initializer does:

And finally you can hardcode them, but I am not going to put any energy into explaining why you might not want to do that.

The Problem

Well simply put if you use either one of these approaches then you cannot OSS your code because well then everybody can look at your code get the credentials and well lets just say that if you do this you would be better of not having any security at all. Saves you a lot of worries.

So you want to keep that type of information outside of your repository if you plan on releasing it as OSS and at the same time use the actual code in production.

But when using Heroku you get yourself a problem, because Heroku receives a push request from your git repository, gets all changes and uses that and only that to build a new instance of your application. Which would then fail to start because it misses the credentials that you placed outside the repository.

The Solution, or well a Solution

First step is to make sure all this secret data is placed in Environment Variables like; ENV[‘mongodb_password’]. Take for example again a look at my MongoDB Rails initializer:

As you can see there are no actual sensitive credentials or other identifiable information stored in the code anymore. Now we only need a way to set these environment variables and for that I use Rake tasks. Lets first look at how I start my development server or run my test, below are my rake tasks:

What happens here is that when I execute “rake run” that before running the actual task it will first run the “set_development_environment” task which as you can see sets the right environment variables. Now I could do this differently by defining these in the different environment config files, but I’ll explain later why I don’t do that.

The Trick

I know, I know, now the secret stuff is still in the same repository. And the dead simple easy trick is to just split this file into two files, one that is outside the repository containing all the secret stuff, like the one here:

And just require this secret rake file inside the public rake file, like this:

Wow that was simple right?

The reason for checking if it exists is that if you want to use Rake and this file is not present then Rake will throw an exception. Now only those tasks that require functionality from the secret rake file will fail. Hint: Heroku uses Rake.

Heeroookuuuu

Yeah yeah, you are right, this won’t work in Heroku because, well Heroku doesn’t have access to this secret repository. But thankfully Heroku has an easy way to set these environment variables using the Heroku gem. Take a look at this additional task which is also located in the secret rake file:

This will set all the needed configuration in the environment variables at Heroku, and is persisted through restarts of the application. So you don’t have to do this after each push to Heroku. The last command just lists all environment variables currently configured. And because I have included this in the project rake file I can within my project structure just type “rake set_heroku_production_environment” without having to navigate somewhere else.

Oh I almost forgot; the reason I don’t use the environment configuration from Rails to setup my test and development environment is that I want to keep all these things in one location. By having them in the same file it is easy to see if I am missing something. Also if I am not using Rails but for example Sinatra then I can keep the same convention.

So there is no reason why you shouldn’t share your code! So share it!

Why Why Why Why Why

2011-01-22

One of my kids: “Why are you going outside?”

Me: “Because I am going to the store”

One of my kids: “Oh ok” “Why are you going to the store?”

Me: “Because I have to buy some food”

One of my kids: “Are you going to buy Pancakes?”

Me: “No, we are …”

One of my kids: “Why not, I want pancakes!”

Me: “… going to eat lasagna”

One of my kids: “Pancakes!”

Me: “We ate pancakes yesterday, we can’t eat pancakes everyday”

One of my kids: “Why not? They taste so good”

Me: “Because they are not healthy”

One of my kids: “Oh ok”

Me: “Bye bye”

One of my kids: “Bye dad!”

… the door closes and I go the the store. After I bought all the things I needed I come home, I open the door …

One of my kids: “Hi daddy” “Have you been to the store?”

Me: “Yes I have :)”

One of my kids: “Did you buy pancakes?”

Me: “No I did not :) I bought lasagna”

One of my kids: “Why didn’t you buy pancakes, I wanted pancakes”

Me: “Well because I didn’t want pancakes since we ate pancakes yesterday”

One of my kids: “But I want Pancakes”

Me: “What did I just say?”

One of my kids: “Ooohhh OK! :(”

How great is this? I mean yes it can get a bit annoying at times, but overall it is just great. Kids are so curious about everything that they keep on asking Why questions the whole time. And they do not yet have the notion of sounding stupid, so when they don’t understand something they will just ask you again, and again, and again.

The question I would like to ask is:

“Why have we stopped asking the question why over and over again?”

It is a good question, don’t stop asking it!

Like Superman, a Hero in Disguise

2011-01-16

When you run into him on the street you wouldn’t know it; he looks be like any other person, perhaps a bit geeky tho. Craftsman is just like Superman, he is a hero in disguise.

It is amazing how many people claim to have seen him and other who are set out to dis-prove his existence. So lets look a bit closer into this weird phenomena called Craftsman.

This is my hypothesis so most likely it won’t exactly fit your theory, but that’s fine.

First lets look at this super hero’s name “Craftsman” I believe that this is a degeneration of the German word “kraft” which means power, so Craftsman is really Powerman in German. This makes me conclude that the origin of the Craftsman lies in Germany.

I guess you didn’t see that one coming?

Ok let the saga continue; Craftsman is an avid crusher of bugs and other unwanted creatures, something eye-witnesses say he may actually enjoy just a little bit too much. But it has been said that he actually believes that prevention is better then providing a cure.

So the story is that he created a small army of digital creatures which he calls “Features” and “Specs”. These creatures have been specifically trained to catch bugs when they are still very small, easy to kill and cannot do to much harm yet.

I believe that these creatures enable Craftsman to catch about 80% of the nasty bugs before they reach adulthood and become very hard to handle. The remaining 20% is caught by continuously keeping a watchful eye on the city.

Craftsman is also big into increasing his knowledge and practicing his skills as he knows that the minute he stops improving himself he is loosing the battle. The bugs he is fighting are getting more and more sophisticated. Rumors are that he has even build his own Dojo especially for the purpose of training himself to get better at crushing them.

Besides constantly improving him self it is also said that he spends a lot of time educating the public mostly in the form of blog posts and other written material under different pseudo names. He may even have a side-kick call it an apprentice helping him out while learning the trade. But for this there is even less supporting evidence then the actual existence of Craftsman himself. All tho I myself strongly believe in both of them.

His hide-out is believed to be some sort of modern cave, he has been heard calling it the “Man Cave”. As far as my studies tell me nobody has ever been inside it and thus we know nothing about what happens there. I can only imagine that he has some special research equipment there to completely understand new species of bugs which helps him finding better ways of beating them.

Personally I think it is extremely ironic that Craftsman’s alter ego is believed to be that of a Software Developer, and that his Twitter description says:

“I am a person working in the software industry and my main responsibility is to put all my power into deliver the highest possible value to my stakeholders”

The sound effect have been recorded by Brian Romeo during one of the bug squashing exercises of Craftsman. Unfortunately it was to dark to get him on screen, so his real identity is as of yet still a complete mystery.

I believe in Craftsman and I think he will be doing great things, I also believe this is still a very incomplete picture of him and that in due time we will learn more about the mysterious forces which drives this great individual to be who he is.

Follow-up Of A Crazy Idea

2010-12-13

After the many responses I got about a crazy idea I had a little while ago with some friends I felt I had to give you a small follow-up on that. Unfortunately this crazy idea won’t fly (phun intended of-course).

There are a bit too many problems to solve, the main one being the noise levels inside the plane which sort of defeats the whole purpose of listening to speakers during the flight.

I did learn a lot from this little exercise; f.ex. I now know that I can charter an A330 (360 seats) for GBP 240.000 and a 474B (581 seats) for GBP 430.000 which if we calculate with a 66% capacity would make the A330 cost GBP 1.000 per person and the 747B would be GBP 1.110 per person. This is including taxes and catering for a round trip from the London to New York. Now to be honest these prices don’t sound too bad too me. Add an other USD 1.000 for two nights at the hotel in New York and we get a total price of USD 2.750 / GBP 1.740 for “accommodation” which is not cheap, but remember this is something really special.

Well maybe we’ll fly together one day in the future, but a.t.m. it is not going to happen, not by me anyway.

Sorry

Power Comes In Different Forms

2010-12-12

I think it is time we get a Noide movement, ahh darn it got my casing all mixed-up; I meant NoIDE of-course. I have been developing on the .Net platform since the first beta’s where released and have been using Visual Studio.Net since it got first released as well (my first .Net coding was actually done in EditPlus). And the one thing that has been a constant during these last 10 years…

Holy crap .Net is 10 years old already… maybe that needs some more reflecting, later perhaps…

Anyway the one constant thing has always been that I always wanted/needed faster and more powerful machines to run my software development environment on. And I am sure you may recognize yourself in this as well; that is why we get Tweets like the one down here:

_{http://twitter.com/#!/unclebobmartin/status/6337116204179456}

And where does this need come from? Well in my case the Microsoft IDE’s got more and more power hungry with every release, we all know this, it is no secrete. Then we also open SQL Management Studio, which slows down the machine even more. And finally some silly developers will start adding plugins to Visual Studio as well, like ReSharper and TestDriven.Net which completely kills the machines performance.

But we say and believe it will speed up development, and I am one of those silly developers, or have been at least. And then there are people like Daniel Cazzuline (@kzu) saying ridiculous stuff like this:

_{http://twitter.com/#!/kzu/status/26143323184}

And you know what, he is absolutely correct! Well let me rephrase that, this is absolutely correct besides perhaps for some people like JP Boodhoo (who is amazingly crazy and you should watch him code, search his publications). If I look at how I use Visual Studio and ReSharper then it is really sad to think about all the wasted effort that was put in these products to create all that functionality that I don’t use. I pretty much only use Visual Studio as a text editor and ReSharper for navigating and some smart code refactorings.

My new girlfriend Ruby

It has been about 6 to 7 months since I have had Visual Studio open, I currently have no machine that has it installed and I truly hope it will be some-time before I have to get into that again (if ever, Hey one can dream). I have since then been working in Ruby and web based technologies, and I am extremely happy.

And please don’t get me wrong, I don’t hate .Net, I just really like working in Ruby.

I started programming Ruby in TextMate, but currently I am getting more and more profound using VIM. And even TextMate starts to feel a bit like a heavy IDE as well (god help me when I do return to .Net development). And you know what; VIM and Terminal are pretty much always really fast on any type of machine. So I started to realize that I stopped caring much about having a fast and powerful machine.

_{http://twitter.com/#!/dhh/status/826554137864192}

Since a few weeks I got myself a MacBook Air 13" and even tho this is not a slow machine by any means, it won’t stand a chance agains my work MacBook Pro that has the same specs as the machine in the first tweet. But it is my little 13" that I am bringing to work each day. Its faster and more powerful brother is quietly waiting for me to get a .Net project. And as long as I am working on these Ruby projects having this faster machine doesn’t bring me anything extra except for some pounds that I have to carry around each day.

The Real Powerrr…

So the power that I rather have is in the tooling being simple and working fast, having a fast machine to counter a slow development environment is solving the wrong problem. This is inline with f.ex. adding SSD disks in your CI server because your tests are running slow, or whatever … people are good at trying to solve the wrong problem, especially developers, just ask Scott Bellware.

So when the next .Net project calls I will have to find a way to get around those stupid project files (generate them) and avoid having to start Visual Studio. Just start coding .Net in VIM and have Mighty Moose, which I understand is a completely awesome auto test runner that also runs on Mono (well awesome except for the name perhaps, ask Greg Young about it). And push it to a CI server to validate it works on .Net as well.

Because the .Net/C# language isn’t a bad place to be and if you need a statically typed language then this is probably one of your better choices. It is just that I really dislike the tooling for .Net, I have always felt it to be wrong, but now I can say with confidence that this is definitely one area where Ruby gives much less friction.

Now after solving the tooling issue lets move on and solve the community issues as well… Ugh…

Good-bye Toto, Hello Dorsey

2010-12-06

The RSS reader you are using doesn't support JavaScript; because of this you will not see the embedded code gists. Consider opening the post in your browser instead.

This post is about something that I hope you didn’t even notice, but I just wrote my own blog engine (again, yes I know…). As you may have seen in an earlier post, but lately I have been using Toto as my blogging engine.

And I was really happy with it, until I wanted more control.

I already had to make some changes to Toto in order to run two different instances side by side. And now I am thinking about adding a few different versions of my blog; f.ex. one mobile version and one that acts more like an application. All for good fun of-course :) And I figured that if I keep using Toto then I will only see obstacles ahead instead of fun.

Because the one thing that makes Toto so great is also the mayor thing that sits in my way; it is build straight on top of Rack and needs nearly no configuration to get boot-strapped. But because of this anything that is different from the default is hard to do. And that is exactly what I want to do, something different.

Sinatra

I figured that in order to do what I want I need more control over what content gets send to the browser and what routes are being used. Think f.ex. about content negotiation to determine whether I need to send the post in HTML or in JSON. I decided to use Sinatra as the base of my new blog engine. Sinatra is a great framework that pretty much doesn’t enforce or prevent you to do anything. It offers the ability to define routes and specify what gets send back to the browser.

If you haven’t seen what Sinatra can do for you then I urge you to take a look, the power is in the simplicity.

Dorsey

The other thing I really like about Toto is the workflow it lets me use:

I create a new text file
Start writing a new blog post
I add this file to Git
Commit when I feel I need to
I push the post using git to Heroku
Viola I just published a new blog post

And I didn’t want to loose this great way of working, neither did I want to convert my existing blog posts again, but that reason is secondary to the first. So with this in mind I started to create Dorsey.

Dorsey is a blog engine that relies on an-other framework to actually publish and drive the content. All Dorsey will do is manage the content, you basically point Dorsey at a folder and it will parse the content (.txt files). Take a look at the setup of Dorsey.

Then I created two routes in Sinatra that will be used to server the content, the first route here is to display the landing page of the blog. It will display the latest blog entries and links to the archives.

The next route is to retrieve the actual content. It uses the requested URL to search for the content. Then it will check how many posts are returned by Dorsey. F.ex. when the URL is /blog/2010/ then it will return all posts from 2010, but if the Url is /blog/2010/11/08/sammy/ then it will return only a single post. Now depending on the number of returned posts I load a specific view and pass in the resulting post(s).

So now we have a blog up and running. I can assure you this is a whole lot easier then writing good blog posts :)

Btw I choose the name Dorsey because it was the band Sinatra used to play in, together they had many great hits, that is also how I see this engine working; together with Sinatra.

Haml & SASS

I also re-wrote all HTML and CSS using Haml and SASS, as I absolutely like that syntax a lot better. If you don’t know this then take a look, absolutely wonderful. More on that in an other post I am sure, but the short story is; Haml and SASS Rock!

Dorsey Internals

Lets take a look at the internals of Dorsey, those who know Toto will see quite a bit of familiarity as I wanted to preserve the workflow of Toto. First up is the Dorsey Server:

Only two methods and two properties exposed, all are pretty self explaining. Now lets look at the configuration options of Dorsey:

You can add different undefined options as well since it is just a hash. They will be accessible via <dorsey_server_variable>.config[:property_name] and this can be useful if you have some additional meta data that belongs to the blog. Next up is the articles collection:

As you can see this collection is actually initialized with the article path and will load the articles itself. I am not to happy with this responsibility here, I think I rather see that in the server instead. Refactoring coming up soon. And when we move that back into the server then articles is nothing more then an array. So most likely bye-bye Articles.

Now finally a bit more logic, lets look at the Article:

A few interesting things here, the famous method_missing is used to enable accessing hash values via non-existing properties. F.ex. you can use post.title instead of having to do post[:title] which enhances readability. Also you may notice a file read, I wonder if I can pass in the stream when I parse the article directory. I am also thinking about making the way the URL (path method) gets build can be configurable. So more room for improvement here as well.

Now all in all this is not a lot of code for a blog engine now is it? Of-course a lot is taken away by using Sinatra, but this just adds lots of flexibility. You can follow the progress on Dorsey here: https://github.com/MarkNijhof/Dorsey.

I also want to thank Toto and the guys creating that, it has been a great source for inspiration, and if you just need a blog then that would be a very good option.

Smidig 2010

2010-11-18

People that follow me on Twitter may have seen my tweets with the #smidig hashtag, so I thought to write a little bit about it so all my non-skandinavian followers also understand what this is all about.

Smidig is the Norwegian word for Agile; and this is also the name of a community arranged conference in Oslo, the conference is 2 days and during these days you can enjoy many lightning talks in the morning, great food for lunch and open spaces in the afternoon. And there was even a 300 person dinner the first day as well.

And unlike many other organized conferences at Smidig all speakers are from the community, so no special invited hotshots. And this big focus on the community makes this event so special, there was a huge amount of interaction going, and many lightning talks kicked-off an open space later.

Lightning talks

Both days there where 3 different tracks with all three time boxes each containing 4 lightning talks, so that made a total of 72 different lightning talks, and that is a lot. All sourced from the community, and they where all professionally done.

A lightning talk was only allowed to last 10 minutes, and this was enforced. But enforced in a good way, once the buzzer goes off you are being silenced by the enthusiastic audience which spontaneously begins to applaud. An other rule was that unless you absolutely didn’t feel comfortable speaking in Norwegian (or a language Norwegians naturally understand like Swedish) that the talk should be held in Norwegian. I think I was the only one that spoke in English, my apologies for that. But what this does is it lowers the threshold for both speakers and attendees.

Below here is my own lightning talk from Smidig, in English.

Open Spaces

Which topics to have at the open spaces was decided half an hour before they where supposed to start. Many of them where triggered by the topics seen at the lightning talks. It was setup in one large room where people could sit in small circles discussing the different topics. Maximum duration was 45 minutes and there where three different time slots. So here again lots of knowledge sharing, learning and discussions.

Thanks

Just wanted to thank the organizers of Smidig for this great experience, will be there next year again for sure!

Sammy.js

2010-11-08

The RSS reader you are using doesn't support JavaScript; because of this you will not see the embedded code gists. Consider opening the post in your browser instead.

If you have some experience in JavaScript development then you know how powerful it can be; for example JavaScript enables us to load additional content after the page has already been loaded by the browser. This can happen either by an user action or perhaps triggered by a timeout. When an user clicks on a link we can trigger some JavaScript that requests some additional JSON data from the server. Then when the data is returned to the client we can use an JavaScript templating engine to combine the JSON with HTML and place it in the appropriate HTML place holder.

And this is absolutely great, now we don’t need to reload the whole page just because a small part of it needs to be updated.

Imagine at a page that lists books, and when the user clicks on one of these books it will request the additional information and displays it in a panel besides the book list. This is a perfect problem to solve using JavaScript, we don’t want to reload the whole page just because we want to show some additional details of a single book from the list. Using JavaScirpt and Ajax will significantly improve the user experience here.

Let’s add one more requirement and that is that the user needs to be able to bookmark this particular book of which the details are being shown, a unique URL is needed.

Now it becomes a bit more difficult, one very common way of solving this is to include the requested book information as optional route parameters. Then when the server gets a request with a specific book provided by the optional parameters it will fetch this books information and embed the information in the correct HTML place holder before returning the page to the client.

And this will work just fine, except that now we have introduced duplication. Duplication in the form of logic to retrieve the requested book (all tho this could easily be reused), the template for displaying the book and the knowledge where this needs to be displayed inside the page. Suddenly we need to make changes in two different places, we need to execute the same tests for two very similar behaviors but that are completely different implemented. And this is not a very elegant way to deal with this.

The easy answer is to get rid of the JavaScript loading and just live with the page reload, but before you do that I would like to introduce you to Sammy:

Sammy

Sammy is a small JavaScript framework build on top of jQuery and gives us ReSTful evented JavaScript. It is based on the same philosophy as Sinatra and allows us to create one page applications that respond well to the forward and back button of the browser and provides direct links within the application. Google’s Gmail but also the latest version of Twitter are using the same technique to create a very nice user experience.

Sammy can solve our problem with the book list, it would do this by providing the ability to execute different JavaScript code triggered by HTML anchor bookmarks. You know when a URL contains a #some_name then the browser automatically navigates to the location in the page that refers to some_name. Well instead of doing that it will trigger a specified JavaScript function instead. We can now have the following URL’s /book_list.html and /book_list.html#/Test_Driven_JavaScript or even /book_list.html#/Test_Driven_JavaScript/Details and Sammy will trigger the correct JavaScript function with the right parameters to load the correct book in the information panel.

Take a look at the code below where we define the two different routes, home, and book details:

As you may have noticed this looks an awful lot like how routes are defined in Sinatra. You can see that we have two GET requests here, but Sammy enables us to use the different ReST keywords GET, POST, PUT and DEL (delete is a reserved JavaScript keyword) in our page routes as well. So a form on the page (or a different page for that matter) that uses POST to /book_list.html#/Test_Driven_JavaScript/Details will result in different functionality being triggered then when you make a GET request to the same URL.

So after we have implemented this logic for our book list we have solved the last requirement without having duplication on the client and server. The only thing that we now also want to do is change the way we used to have our book information loaded via JavaScript to just be a link with the correct hash route and then we are completely DRY again.

There are many more gems build into Sammy and some as extra libraries, like the eventing part which I’ll discuss an other time. In the meantime take a look at the Sammy website and play with it. If you haven’t already them you will soon discover the amazing power this little framework provides. My own website is using Sammy as well to show the different information connected to the menu, just take a look at the source.

A Crazy Idea

2010-09-22

Some time ago after the NNUG yearly meeting I was talking to Sondre Bjellås and Lisbeth Skogland and one thing led to another and suddenly we were talking about arranging a conference.

But not any conference may I add:

Air Conf

The name may have given you a clue, but this is a conference held in the air. Imagine getting on an Airbus A380 in London Heathrow with destination New York; together with 350 – 400 other highly motivated craftsman? We would take-off late in the morning, and flight time would be around 8 hours.

Instead of the boring old stuff that is normally provided on a flight, we have speakers presenting their ideas and experiences about software development to us. Now that is inflight entertainment if you ask me.

On a plane, really?

Well think of it; what is a conference to employers?

Probably most of them see it as something they can do for their employees. A nice treat as it where, but something that actually adds value as well, in the form of knowledge building / sharing.

So what Air Conf offers it exactly that, one 8 hour flight with the best inflight entertainment solution ever. Then one full day in a great place; like for example New York. And finally an other 8 hour fully information packed flight back home. How amazingly cool would that be?

Really, You mean on a plane?

Well an Airbus A380 holds about 525 passengers; which means that there is enough space for about 350 – 400 craftsman and still have some place for a more quite area.

Looking at the seating layout of a normal configured A380 I can clearly imagine that we will be able to have four simultaneous tracks and have a common area. And of-course the speakers area is taken care of on the first class deck.

Most planes actually offer the ability to reconfigure the layout somewhat if needed, so this will probably be the case for Air Conf, but not like this VIP layout:

Four Different Tracks

We will have to make sure that once people pick a track that they would be happy most of the time there, as I am sure that the pilots don’t want massive groups of people moving in the plane. Also a clear maximum number of people per track would be something that is enforced, we can’t all sit in the back.

Making the speaker properly visible for all attendees in the track will be a bit more of a challenge, I am thinking about removing the middle seats of the first few rows in each track creating enough space. Also the choice of airplane is important. The Boeing 777 offers more head space then for example the Boeing 747. The Airbus 380 offer lots of head space and can carry more people, but can we get one?

Half way into the trip there will be food, and I am sure we will be able to get something better then that awful airplane food :)

Agile Train

This is sort of done before, but then it was on a train, this was Agile Train from Oslo to Trondheim going to XP 2010 and I believe it was a great success. A train is of-course more limited in respect to its destinations, and a plane is just so much cooler!

The Speakers

There are many speakers who travel between Europe and the US for work; so I figured is that some speakers perhaps only want to be on Air Conf one way. For example someone from Europe would be at Air Conf on the first leg, but will stay in the US to do some consulting. In the US we would pick-up another speaker for the second leg back home.

But realistically will they join such an endeavor? I think they will, it is going to be super exciting and not done before (exactly the right ingredients).

Kevlin Henney said: “That’s insane! Which means that, with some ironing out of details, it could in theory work.”

Robert C. Martin said: “<gasp> <gulp> <eye-bulge-choke> If you think you can pull it off, I’m game”

But can it actually be done?

Yes!

I have already been talking to an Air Charter and the first response was that it isn’t all that crazy after all.

Your Feedback

Yeah I would really like to hear what you have to say about this!

Would you sign-up for this conference?
Would you want to present at this conference?
Why is it completely ridicules and will it never work?
Why is this perfect and amazing that it took so long before someone thought of it?

So I like to hear positive and negative feedback!

In The Enterprise

2010-09-01

I quite often see comments like; “We are only allowed to use Microsoft Technologies” and every-time I think “Why?”. What is the reason behind such a standpoint. What is won by limiting their choices in such a radical way?

Customer Support

Support from Microsoft is often the reason that I hear when I ask these questions, but is this really that valuable?

When you are a gold partner or something then indeed you can ask Microsoft to help you out with some difficult issues that you are experiencing, and they will help you as much as they can. I also understood from Kenneth that when indeed a bug is found in the Microsoft code then that they can create a temporarily patch which will also end up in a next release. And besides this that it will also be made available to others in the form of a KB article. This is however not how I have experienced it.

One other thing is that Microsoft will not accept patches from us. For most of their software this is not a problem because it it closed source anyway, but they won’t even accept if for the “Sorta Open Source” software they have.

Now all tho it sounds pretty great that you can ask Microsoft to help you out, is this something that you would be missing from using other solutions like Open Source Software?

The OSS Alternative

When we for example take a look at NHibernate (just to stay on the Microsoft platform) then you have a great community that can help you out with any sort of question you may have. The contributors can point you to the right recourses and even provide example code. In my experience the help you get is also very fast, but it is true, you can’t just call a guy (all tho I have actually done that a few times as well) and tell them that you need help. You will need to help them to help you by providing good information and means to reproduce the problem.

And you also have full access to the source code, so you can look inside to see what is going wrong. After you have identified the problem you can fix it yourself as well. And when you submit a patch then they happily accept it from you. And when done properly it will be applied in the next release. Releases are most often also more frequent so the wait won’t be that long.

And finally, the community actually cares about your input, if you have an issue with the framework and have a great idea on how to do something different then they are more then happy to listen to you. And if it makes sense then it will be incorporated as well.

So what did you really win here by skipping the alternatives?

Framework Support

Ok here is another reason; Microsoft will keep supporting their software and keep improving it.

Well this is just not true, sure for many software solutions this is true, especially the ones you have to pay for. But there are also many cases where Microsoft suddenly decided to not continue developing the solution. There have been many people that have lost a lot of time in for example; Linq to SQL, M and the last one I heard about IronRuby. Now true IronRuby will hopefully be picked up by the community and Microsoft will release the source code, but this is not what happened with the other products.

And even if Microsoft releases the source code and the community picks up the work then you are perhaps not stuck anymore, but you won’t be using Microsoft technology anymore either. But history has proven us that most of the times this won’t be a concern anyway because the code will never be released.

The OSS Alternative

For this let us take a look at the Ruby community, this community is to my knowledge 100% Open Source. There are now 75.000+ Gems stored on RubyGems.org now that is a community. And of-course there will be many gems that are discontinued or not supported, but you will always have the source available. So you will never get stuck with some software that is not supported and you cannot touch.

Oh and framework support is pretty impressive on some of these solutions, take a look at the most well know framework; Ruby on Rails. The first version was released on 13th of December 2005 and it has steadily gone from 1.0, 1.2. 2.0, 2.1, 2.2, 2.3 till now 3.0 just released on 29th of August 2010. These are 3 major released in 5 years, and the minor releases in between where significant as well.

Another good thing is that there are usually different approaches to solving the same problem, which means that if one approach doesn’t really fit your needs then you can try to find a different approach. Think for example about Rails, Padrino and Sinatra all are frameworks to provide web content, but where one is very complete there the other is very bare bone. So depending on your needs you choose one.

So what did you really win here by skipping the alternatives?

It Is What We Know

All our infrastructure and existing software is .Net and the guys building and maintaining it don’t know anything else, so your new solution has to fit in there as well.

This is a scary argument, but most likely the most valid one (it is so scary because it is valid in many places). This is why you see that each and every project is using Microsoft SQL Server and that everything is build up from ASP.NET. This “It is what we know” argument even prevents many from using something like ASP.NET MVC, which compared to ASP.NET is a major step forwards.

And it is not only management deciding that this is how things work, I have heard developers as well choosing a technology that wasn’t the right tool for the job just because they felt comfortable with it.

The OSS Alternative

The alternative is that you actively encourage your employees / college’s to learn other things. Because they will learn different things and start understanding that the de-facto is not always the most appropriate choice, you will end up with better software. At least there is a good chance that your software solutions will be better. Even if only because by exposing them to the OSS world they also get a feel of something that Microsoft is very good at hiding from you; proper software design and testability.

At the very least by having more versatile people adapting to a new Microsoft technology will be much easier, but I am hoping for more.

So what did you really win here by skipping the alternatives?

Finally

I am really hoping for your input here; how do we get to open up the “enterprises” to accept that there is much more then only Microsoft. I am not even saying we should all be using Ruby, Closure or Scala to solve our problems (all tho that would be great). But even on the Microsoft platform there are many good alternatives to solve business problems in a better way.

Also if you have reasons valid or not of why an organization would be staying with purely Microsoft software then I would love to hear about them as well.

The Web is all about Composition

2010-08-23

With new technologies we also get new ways of solving problems, and of-course this very much applies to the web. In the good old days we had static web pages that contained all the information we wanted to display. Web browsers would do one request per page and several parallel or sequential requests for images and style sheets. But time has changed this.

Now we have browsers that know how to execute JavaScript and are able to make sequential requests triggered by JavaScript from the original request. This can also be triggered by the user; for example by clicking on a link or button. This additional requested content can then be placed anywhere on the page, even after the page has already been displayed to the user.

Composition

So this means that we can start composing our web pages out of multiple resources, each resource responsible for one part of the web page. Do I hear someone say SRP?

Let’s take a quick look at one very well know website: Amazon.com

On this page I count at least 11 different resources or components, each component has one single responsibility. And perhaps even more important, the different components on the page are not depending upon each other. I highlighted them for you:

Now when we for example look at the “Inspired by Your Wish List” item and “Netbooks Resellers” item, these two different content items do not relate to each other, at least not directly. There may be different rules at play that do take into account what content you are looking at, but these rules would mostly be affected by the main component of the page i.e. what the visitor is actually interested in, and who the visitor is.

Layout

So what would happen to the page if the rules engine decided that I should not see the “Inspired by Your Wish List” item? Well nothing drastically the “Netbooks Resellers” item would just move up and all other content would not be affected by this.

This means that the layout of the web page needs to be flexible enough to display different configurations of different components. If we look at the Amazon site we clearly see three different columns, the two on the side always pretty much stay the same size. Content in these columns can be pretty fixed for this size. But the middle column scales to the width of the window, and Amazon is smart about it. When the window is small it only shows for example 3 items in the “Inspired by Your Wish List” item, but when you make it wider it will start showing more items, dynamically. This makes me think that Amazon has decided that a particular component can be as wide as it gets space, but can only be a certain height.

So the layout of the web page is important, but it is also important that the different components either know their configuration or can adapt to the environment.

Small Applications

The next step is to think of each one of these components as their own little web application, they do one thing but they do them well.

Why is this interesting, well it lets you iteratively add new functionality to your website without it affecting other parts of you site. It also lets you display different configurations to different visitors. For example when you know who is visiting you, you may want to show him/her some custom content.

Now this would be very hard or become extremely complex if there was one single process that builds a web page, but when we split all these components into their own little applications then suddenly it is only one part in the code that is affected.

Also all these components can also be tested independently from each other!

JavaScript

I really like to use JavaScript to load the different components onto the web page, the actual requested content is just a place holder. On the Amazon site this would be the top bar and the three columns and the footer. Then the web page will start requesting the different components from the server and adding them into the correct placeholders. For the end user the page is nicely responsive, content just starts popping up on the page. You would prioritize the main content of-course.

Then there are two ways that you can request the components, either request full html and load that into a placeholder or request Json and combine this with a template and load that into the placeholder. It would depend on the content which would be preferred.

For example take a book description on Amazon, considering how often this is shown it might be a good idea to only request Json for each specific book, and make a separate request for the template. The template can be cached for a long time, so after the first book this piece of html is already on the client. And with a popular site as Amazon, it is very likely that this template was already at one of the nodes close by, so it wouldn’t even have asked the Amazon servers for it the first time anyway. Then combine this cached template with the unique data and you are done.

On second thought a book can probably be cached quit long as well, so perhaps a template is not the preferred option here. I think the deciding factor would be how often the content would change independently from the way it is presented. It all depends, just keep in mind that there are multiple ways and each way has different justifications.

Now I know Amazon doesn’t use JavaScript to load the different components on their website, instead they let the web server aggregate the different components, more about that here, but I do believe that most of us could just use JavaScript to accomplish the same flexibility.

HTML 5

I am just going to say here that we are facing interesting times when this becomes common ground!

Shown; To be or not to be

So who decides? Well first of all if the main request doesn’t request the other components then they won’t be shown, simple, but each component can also decide that it doesn’t want to be visible to the particular visitor. It can for example return nothing and in that case the layout will just move the next components up. It can also decide that instead of displaying the actual component content it redirects the request to an ad service or something else.

I like to have the main request be responsible for the different components it should be able to show, usually this depends very much on the main content of that web page. I would keep this process as dumb as possible, one exception is whether or not the user is know. Because this is usually a drastic change of content. But if there is a beta program then this beta component should just always be requested.

Then the different components themselves can decide if they should be visible or not, in case of the beta program maybe it is only shown for 5 procent of the users in Norway, that complexity I like to have at the component itself.

Then again, a site like Amazon probably has some sort of standardized logic for this, a pre-filter as it where to deal with this stuff.

Finally

Whether or not JavaScript is a good choice here really depends, but I am convinced that it is a good starting point. The beauty of this all is that you have started to think about your website in a different way. Instead of pages you start thinking about resources and aggregators of these resources. Then if later you need to move to aggregating these on the server instead of on the client it will not be such a huge job.

Personally I think that the JavaScript approach can be pushed for a very long way. Maybe next time I’ll show some code :)

Importing Comments into Disqus

2010-08-22

The RSS reader you are using doesn't support JavaScript; because of this you will not see the embedded code gists. Consider opening the post in your browser instead.

I wrote before how I moved my blog to Heroku using Toto for the blogging engine and Disqus for providing the comments functionality. And I have to say so-far Disqus has been great, of-course I haven’t received that many comments yet that I would have encountered problems ;)

But I already had a few comments on my previous two blogs; I did manage to port the posts without any problems, but I also would like to port these old comments. Thankfully both my custom solution and Wordpress offers full access to the comments, so I had all the data.

Importing into Disqus

So I looked at the different options Disqus offers to import existing comments and I went for the import using the IntenseDebate format. And this worked pretty well. Here is the format for the import file:

isAnon – I don’t know the meaning but I can guess, an IntenseDebate thing
score – I don’t know the meaning but I can guess, an IntenseDebate thing
url – can be left empty
ip – can be left empty

Also note that the format of the comment is like plain text formatting, all html formatting is stripped. So you would want to convert your formatting to plain text formatting, like the example above here shows as well.

Issues

Well once I knew how to do it, not many actually. One problem I have is that you cannot override existing comments by re-importing them again, even not when you delete the wrong ones from the Disqus website. Disqus has some algorithm to determine if a comment already exists and if it does it won’t re-import them. In my first try I messed up the date format (I had some whitespace before and after the date, silly me) which meant that they where all dated on import. I couldn’t import these again without the white space. I only managed by actually changing the time of these posts. All second are now 00 :) So that was a bit weird and annoying. There are 4 comments I am not able to re-import, I even tried moving them one day ahead in time.

Threads are not working either as this information does not exist in the import data, but I can live with that, order is preserved by comment data.

Suggestion

I would suggest to make one test post using a new guid when that all looks ok you change the guid to the one matching the actual post and import them again. That way you don’t associate them with the actual post and if you messed up then no harm is done. You can even fake the test post, but that is just an easy way to access the comments.

Gemsets, Bundler and Versioning

2010-08-19

One of the things I like about Ruby versus .Net is how easy it is to work with different versions of a specific Gem, for example you wanted to upgrade a specific gem in a project from version 1.2.0 to 1.2.3, then all you needed to do is gem install the_gem -v=1.2.3. This command will just install the new version side by side with the previous version. Then in the project you only need to specify that you want to use the new version of the gem. If you don’t specify a version then the latest installed version will be used automatically (so you want to make sure you specify a version). For example in Rails you can specify the version by adding the line: config.gem ‘the_gem’, :version => ‘1.2.3’ in the environment.rb file, but there are also other ways.

So far this looks pretty much like how it is done in the .Net world, where we can have multiple different versions of a single DLL side by side in the GAC. But as many .Net developers already know, the GAC ain’t a great place to be. And the same applies to the Ruby world, you would for example end up with many different Gems which each have many different versions. A complete nightmare to maintain, what project is using which Gem of what version? To solve this problem I want to introduce you to Gemsets and Bundler.

Gemsets

We will start with exploring Gemsets a bit, this is a solution that comes with RVM and works at the lowest level. What Gemsets does is that it creates different folders for different named projects and installs the gems inside there. Then when you switch projects also the gems folder is switched. Thus nicely separating the different dependencies of each project you have on your machine. A great way to automatically switch to the correct project is to add this rvm 1.8.7@project_name to your .rvmrc file.

You can also add rvm —create use default@project_name > /dev/null which will create the Gemsets project folder if it doesn’t already exists, which is great for sharing your projects with team members. Or export rvm_gemset_create_on_use_flag=1 to the root ~/.rvmrc file, but that won’t help team members that don’t have that :) take a look here for some background on this.

Now even tho this is very project orientated this is actually still very specific to the machine it is setup on, meaning that the project dependencies are very isolated from the other projects, but they are not managed by the projects themselves. In other words, if I get a new project from someone then I still have to setup all the Gems myself in the specific project location.

You can import all declared dependencies from the .gems file by using rvm gemset import, but unlike Bundler this won’t resolve dependency conflicts.

Bundler

And that is where Bundler comes into play, but before looking at the dependency resolution I like to talk about the fact that by using Bundler you have now also moved the dependency declaration towards the project. All you need to do is gem install bundler and create a Gemfile file in the root of your project. Inside this file you declare your project dependencies like gem ‘padrino’, ‘0.9.10’. There are more cool options like directly pulling the source from a GitHub repository, but you can read about that on the Bundler site. Now once you have declared all your dependencies you need to run bundle install which will start downloading all the required Gems and it will place them in a global cache or a project specific folder.

Now so far it is not to different from how plain Ruby works, except that now all Gems are actually managed by the projects themselves. You could just delete all the Gems on your system, install Bundler and then let Bundler get all necessary Gems again.

Except there is more, you can and should now also run your Ruby code through Bundler. When you for example want to execute Cucumber then you would normally run cucumber but this will just load the gems from you gem location without looking at the specified dependencies. But when you run bundle exec cucumber then Bundler will create an isolated environment in which it will run cucumber, meaning that if you have forgotten to add a Gem to the Gemfile then it won’t run, even if this gem is installed on your system.

Rails 3 will by default use Bundler, other frameworks may need some configuration in order to use it directly.

Now before you start saying what a pain this is, think about the scenario where you need to deploy your application to a server, and instead of having to manually check all the different Gems needed, you just execute bundle install and you are done. And because you run your development environment also through Bundler you won’t forget to add this needed dependency.

Heroku will do this all for you, if it detects a Gemfile in your GitHub repository then it automatically uses Bundler to resolve the dependencies, how cool is that!

Dependency Resolution

And now it is getting really interesting, we continue with Bundler. Let’s say we have two different Gems, gem_a and gem_b now both of these Gems depend on gem_c but they both have different restrictions on the actual version of gem_c. gem_a can take any version of gem_c above 1.0.0, but gem_b only works with versions 1.0.* of gem_c. Now lets say gem_c has a version 1.1.0, this will work nicely for gem_a but not for gem_b.

Bundler will resolve these issues nicely for you. Much more including this example can be found at Yehuda Katz blog who is the creator of Bundler.

Combining Gemsets and Bundler

I think that everything except the Bundler gem should be defined in the Bundler configuration (i.e. Gemfile) and that the Bundler gem itself should be managed by Gemsets. Because now all dependencies except for the Bundler version are managed by the project it self, and there is no chance that you forget to add a dependency because it was already installed on your system, this also includes Gems needed to execute the tests. But would of course exclude any System Gems.

Reflecting on .Net

There are currently some great efforts going on in the .Net world to solve the problems that the above solutions have solved for the Ruby world, think about OpenWrap, NU and Bricks which is absolutely great!

But I feel that what needs to be done first is that all .Net projects start implementing a good versioning policy, because without that nothing will solve this issue. And I truly hope that these three initiatives can at least come-up with one versioning scheme, but so far I haven’t heard too much about that.

The .rvmrc File

2010-08-18

So after you have read my last post I am sure you have been looking into RVM and if you haven’t perhaps you should. And you have seen how much power and simplicity RVM can bring to your development workflow. Take for example my current situation, I have some projects in Ruby 1.8.7 and also in JRuby 1.4.0. Now switching between these is still a little bit of a pain, because I have to remember to tell RVM that I need a different Ruby version depending on the project I am working in.

So that is where the .rvmrc files come into play. Just add on of these files to your root directory of the project and specify the rvm command inside this file (f.ex. “rvm 1.8.7” or “rvm jruby-1.4.0”). Then every-time you hit the root of your project RVM will notice the .rvmrc file and execute the commands inside. Thus switching to the correct Ruby version for you.

RVM also allows you to group installed Gems together based on a tag, so if you f.ex. have two different projects that need different Gems but both of them are on Ruby 1.8.7 then you can specify a sort of group to each of these projects and then “gem install xyz” will install this gem only for that particular group. I’ll go a bit deeper into this in a following post looking at both Bundler and Gemsets.

My Terminal Prompt

2010-08-10

The RSS reader you are using doesn't support JavaScript; because of this you will not see the embedded code gists. Consider opening the post in your browser instead.

I want to share something that I learned from Rolf Bjaanes not to long ago and that is that the Terminal command prompt can give me a lot of useful information. Below here is how my Terminal command prompt looks by default:

Now my command prompt displays: [machine name]:[current folder] [user]$

But when you look at the second screenshot you see much more useful information, it starts with the user @ the machine then follows the complete path instead of just the current folder. It will tell me that a folder is a Git repository including which branch I am currently working in. And finally it shows we which Ruby version I currently have selected using RVM (Ruby Version Manager, which is something cool as well). And, there is color :) I know color is not for everybody, but I like it, it clearly separates the different types of information for me.

And finally instead of closing with a boring $ sign I now have a three dotted triangle ∴ how cool is that?

So how did we get all of this added information? Well you need to tell the Terminal that you want this and for that you would use a profile, there are apparently multiple ways of defining a profile, but I did it by creating a .bash_profile file in the user root, just type mate ~/.bash_profile if it doesn’t exists TextMate will create it else it will just open the existing file. Then near the end you add the following line:

Excuse me!

Yeah I said that as well when Rolf send me the string. So I went on and Googled the different parts, and really it isn’t that hard. Lets go through the string and discover what is what:

export PS1=

This is the variable that the Terminal will use to build the command prompt, we are just going to assign a new value to it.

\[$(tput setaf 2)\]

Ok this is a bit special, the \[ and \] indicate to the Terminal that between these brackets there is something that should not be displayed directly. Then we have the $( and ) and this means that what ever in inside of this will be executed. Now finally we get to tput setef 2 and what this does is tell the Terminal that from now on it should color the output green. The number corresponds with a color. I won’t take away the fun you’ll have discovering what it what.

\u, \h and \w

These are commands to print certain system information, for example \u is the current user, \h is the machine name and \w is the current path. But there are many more commands that you can use.

`if [ “a” = “b” ]; then echo “A” ; else echo “B”; fi`

Then we have two if statements inside the configuration string. Please note the weird ` single quote surrounding the if statement, without these it won’t work! But assuming about everybody reading this blog will know what an if statement is I am not going to go deeper into this, except that you see [ “$(vcprompt)” != “” ] more specifically the variable (or for those paying attention this is actually a command) is enclosed with quotes as well, and this is because when they are not there and the variable is empty then the executed command would be [ != “” ] which would fail.

Now the only reason that I have these two if statements is because I want to prefix the printed information with a white space if it is not empty, I could ‘hard code’ this but then when it is empty I would end-up with multiple spaces. Yeah call me crazy!

vcprompt

This is a utility that provides you with version control information, and we use this to print the Git information. You can install this using for example HomeBrew.

~/.rvm/bin/rvm-prompt i v

And then finally this command will print the current version of Ruby that is selected by RVM. This script is installed when you install RVM so you don’t need to worry about it too much.

.bash_profile

Below here is my complete .bash_profile file where you will also find two functions that are used to automatically resize the Terminal window. I just found that today, so I don’t know if there is a better way to accomplish this.

Here is also a good blog post about 8 different setups. Any feedback you may have is appreciated!

Blog moved, Once again...

2010-08-04

The RSS reader you are using doesn't support JavaScript; because of this you will not see the embedded code gists. Consider opening the post in your browser instead.

Ok it happened again; I started blogging on Blogger, then created my own blogging engine using FubuMVC and moved to the Fohjin domain. Then I got asked to join Elegant Code which I was very flattered by and accepted right away. But now I feel that this is maybe not the right place for me to blog after all.

So I setup a new blog under http://Cre8iveThought.com/blog/index and will be blogging there from now on. And this is where I will stay :)

Toto & Heroku

So did you create a new blogging engine, well no, I grabbed Toto and it is running on Heroku and that was done in 5 minutes, copying most of the previous content took a little bit longer tho. Now my blog posts are on GitHub and they are pushed to Heroku when-ever I have a new post (or change an existing one). The comments are now managed by Disqus and the code examples are provided by GitHubs Gists. And finally the layout was mostly copied from Rob Conery’s blog as I think this is extremely well readable.

I only added by not adding, so I feel my blog is now super clean. There will be some small features added later I am sure, but content is what it is about. Please let me know what you think of the new style?

My apologies to all of you who are still following my FeedBurner feed for all the duplicates, and for all of you on ElegantCode not following my personal feed here is the link: http://feeds.feedburner.com/fohjin, if you want to then please do.

And finally, I want to stress that me leaving ElegantCode has nothing to do with ElegantCode, but more about me wanting to take my own steps.

I'll be BEKK

2010-06-24

Just a little note; I am very happy to announce that I’ll be joining BEKK from August on, I’ll be working mostly on .Net projects I am sure, but I have the intention to squeeze some Ruby projects in there as well. And perhaps this can become the majority soon as well?

Anyway it will be great fun and many cool college’s, so looking very much forward to it.

Ruby Heroku

2010-03-02

When I started looking into Ruby more seriously I wanted to have an actual hosting provider that would host the site that I am working on. A friend of mine recommended me to look into Heroku and I have to say that I am very happy that he did.

Heroku offers a unique way of deploying your Ruby web applications, you push them! git push heroku master

Even do it is the first thing you see when visiting the site I still want to walk you through the process of setting up Heroku and deploying your Ruby application.

$ sudo gem install heroku
$ heroku create
$ git push heroku master
$ open http://.heroku.com/

Oh did I mention yet that the initial plan is free? Free as in you don’t have to pay anything! Of course when your application starts to attract visitors and you need more performance then you will need to start paying. But you have full control over this and I believe you can push it pretty far.

There are also many addons that you can enable (for some you need to pay for some not) providing a lot of functionality.

Now this looks like a commercial, but until someone told me about it I didn’t know, so now I am telling you. If you are doing some playing around or are actually working on a serious project. If it is in Ruby then this is worth checking out.

They are also offering some beta Node.js support!

More Ruby and other web stuff coming soon!

The Ubiquitous Language is not Ubiquitous

2010-02-18

I attended an interesting Domain-Driven Design talk today given by Janniche Haugen talking about why you would want to use Domain-Driven Design in a project, and this presentation is what triggered this post.

My statement is that the Ubiquitous Language in Domain-Driven Design is not Ubiquitous.

Lets first look at the definition of ubiquitous: Being present everywhere at once. Hmm that sounds a bit vague; here is a definition of ubiquitous language: A language structured around the domain model and used by all team members to connect all activities of the team with the software. Ah that is better. So the ubiquitous language is a common language shared by the domain experts, developers and the code.

But it is not a common language throughout all of the domain and code, this is one of the reasons why we have different bounded contexts. Because the same language may mean different things to different domain experts. Think for example about a shipping company, the meaning of the word ship is completely different when talking to accounting versus maintenance. For accounting a ship is an asset that degrades in value over time, but for maintenance a ship is an object that needs service every x nautical miles.

The same word has a different meaning when used in a different context, that is not ubiquitous.

Lets continue the small example, domain experts from maintenance also talk about an engine and rotor blades. But the domain expert in accounting don’t use these words at all, they have no meaning to them.

Some words even have no meaning at all when used in a different context, this as well is not ubiquitous.

So the Ubiquitous Language is only Ubiquitous within a given Context. What do you think?

CQRS Scalability

2010-02-09

Scalability is one of the several different benefits you gain from applying CQRS and Event Sourcing to your application architecture. And that is what I wanted to take a closer look at in this post.

One of the first obvious ways to increase the performance of your system is to split the Command from the Query side by using a service bus or just a simple queue. So instead of one machine being responsible for both responsibilities you now have 2 machines.

Then you also get the ability to measure more precisely what the actual bottleneck is, normally your application queries for data many times more then that it is executing behavior. And there are already many well known approaches for scaling your reporting database so that is not what I would like to talk about, but I guess it is clear that this side can now be scaled-out individually from the command side.

Now the command side; because even if this is used many times less then the query side, the actual behavior may take much more time and or processing power.

There are two natural ways of splitting up the domain, the first one is by Aggregate Root type, so you can choose to place a single (or multiple) Aggregate Root type on a different server. The second way is by splitting-up a single Aggregate Root type by the Identity each individual instance has. Think of for example about splitting them up depending on the last number of the Id, even versus un-even.

So who decides? The decision needs to be made by the process that accepts the commands and passes them on to the command handlers, these command handlers can then be running on different machines and each machine may even have it’s own Event Store.

This is a very flexible way of splitting-up your system into different components, and because of the Event Driven Architecture basically build-into this approach you will be able to trigger other behavior on different Aggregate Roots without added trouble.

CQRS Event Versioning

2010-02-09

The RSS reader you are using doesn't support JavaScript; because of this you will not see the embedded code gists. Consider opening the post in your browser instead.

When using Event Sourcing you store your events in an Event Store. This Event Store can only insert new events and read historical events, nothing more nothing less. So when you change your domain logic and also the events belonging to this behavior, then you cannot go back into the Event Store and do a one time convert of all the historical events belonging to the same behavior. The Event Store needs to stay intact, that is one of its powers.

So you make a new version of the original event, this new version carries more or less information then the original one. Lets take a look at a very simple example:

This to me looks like a natural evolution for this type of event, so how do you deal with this. Because after having used the system, before adding this extension there have been many cash withdrawals. So all these events are in the Event Store, they cannot be altered, and when you retrieve an Aggregate Root from the Event Store all these historical events need to be processed in order to restore the internal state.

Now what you don’t want is to maintain code in the Aggregate Root that knows how to handle these old event versions, sure one version is ok, but what about one hundred different versions? Also we are not just talking about just in the Aggregate Root, also the different event handlers need to be kept and maintained.

The better approach is to have a mechanism that you can hook-up with different event convertors. Then when an event is retrieved from the Event Store it first goes through this pipeline of convertors to be converted to the latest event version.

Now I wanted to do this properly and write some actual code for this, and then blog about it, but someone kept nagging me about it, so here is a very rough spike instead, first some tests:

So basically some tests to confirm the correct conversion from one event to another event, now below here is the full implementation:

This implementation is definitely not very elegant (so it doesn’t really belong on this blog) but hey it does show you how a possible solution would work. When building this yourself you might want to use conventions to auto register the convertors and chain them together during configuration so there is no need for the recursive functionality.

Also look at the jump from version 1 to version 4, this is an optimization to speed up the conversion. You would do this after a few versions, not for each version.

I’ll be adding a proper solution to the example in the near future, something that you would just plug the convertors in and the system would figure out how to handle them itself.

CQRS Event Sourcing

2010-02-05

So after reading this blog post by Rob Conery about Reporting In NoSQL where he explains very well what the problem is when using a RDBMS for persisting the state of your domain, or really anything that is written with Object Orientation in mind.

His solution to the problem is to use a object database or a document database for persisting the state of your object structure. And I do agree that this is a valuable approach to solve the problem.

But I would like to talk about a different approach, which is called Event Sourcing; a pattern thought of by Martin Fowler that, “Captures all changes to an application state as a sequence of events.”

Hey that is interesting, so instead of trying to store the object tree in a whole, we just store all the individual state changes that the object tree encapsulates; meaning all the state changes that have happened during the complete lifetime of the object tree. These state changes are being represented in the form of events. And such an event is nothing more then a Plain Old CLR Object, so not an actual .Net event.

And the objects are also re-constructed from the same events by applying all the state changes that they represent, and thus coming back to the previous state in the identical way that it came there originally.

Now the interesting part with respect to persistence is that these events are being serialized using a technique you like (Binary, JSon or custom) and this serialized event (object) is persisted in an event store, and this event store will threat all serialized events equally.

This event store can be based on an object database, document database, file system or even a RDBMS, you basically need to have one collection that describes all the different objects that the event store has persisted this includes the Id and the Version. Then another collection will contain all the serialized events for each different object and they should be retrievable by the object Id ordered by their Version. So to simplify this, in a RDBMS this would mean 2 tables, in total.

So there is no impendence mismatch between the domain (object structure) and the persistence layer anymore. Which would mean that it is should pass Rob’s criteria’s as well.

But hey it doesn’t stop there, you get a real audit log for free a well. And you have the ability to replay all the events to abstract new information about certain state changes. For example a web store is in business for 6 months and now they would like to know when and where an item is being removed from the shopping cart. With an event store you will have this information from the start of the system, and you didn’t have to think about it straight from the start.

And this also enables an easier swift to an Event Driven Architecture as well, because you can start publishing the events outside of the domain and have different behavior or processes react on them.

In my example I use a combination of CQRS and Event Sourcing and this makes a very powerful solution. I would recommend that when you are when applying CQRS you would do that in combination with Event Sourcing to get a very flexible system without much more complexity then just applying CQRS.

Slicing the day

2010-02-01

I have noticed with myself (actually I know this for a very long time now already) that I am much more productive in the evenings then I am at the end of a working day. I also just finished reading Programming is all Design by Joakim Holman, Why Design Cannot be Billed by the Hour by Steve Zelle, Software on the Cheap by Uncle Bob Martin and The Design Dilemma by Kent Beck which made me want to put my thoughts into writing.

Now the main topic in these blog posts is design, writing Software is about design and not about producing code. And that design is an easily overlooked topic within the software industry. Sure for us developers our end product is the functionality that our code delivers, but the way we chose to get there makes a difference like between night and day. In the end choosing the road that leads through design is the more optimal path for creating software (I am aware that there are exceptions). Also note the interesting move from calling TDD Test Driven Development to Test Driven Design and even Design by Example.

Now design means thinking, because we need to really understand the problem space and find a solution that we think will solve a particular problem the best way possible given our constraints without eliminating the possibility that we were wrong. And also that it is a continuous process that will drive software development throughout the whole lifetime of the project.

So it is fair to say that this takes effort? I would actually say that it takes a lot of effort. So considering this assumption I can also safely assume that we would do a better job in the morning then in the afternoon? Right? Well I know I do, that is until I take a break and do something completely different and then after that continue to work on the problem. After such a break I have been able to recharge my internal battery and my mind is ready to do some more serious work.

So my ideal day is 4 hours in the morning and 4 hours in the evening. The time in between is my spare time where I can enjoy the weather (if only I didn’t live in Bergen), get the kids early from kindergarten or even do some shopping with my wife. Then after this change of focus I would continue to work on the problem space, but now fully recharged.

This brings some obvious problems along the way, one of them is that pairing (a practice that I value very much, read more about that here and here) becomes very difficult because most of your college’s would not be slicing the day like this. Remote pairing would solve some of these issues I believe.

One minimum step that I believe we should all consider is to move any meetings you have to the second part of the day, giving your full focus to the more difficult parts of the job, Designing software.

Now what I like to know from you is; what are your thoughts about this?

Being provoked by your uncle

2010-01-24

So there was a lot of commotion about two blog posts by Uncle Bob Martin; Dependency Injection Inversion and Mocking Mocking and Testing Outcomes. With replies from people like Jimmy Bogard “Poor use of DI versus need for DI” and Ayende “Rejecting Dependency Injection Inversion”. And also on Twitter there were many discussions going back and forth about it.

I would like to leave the discussion for what it is and focus on another aspect of the story. Apparently Uncle Bob was trying to provoke a reaction from his readers, but this was not very obvious in the blog posts themselves.

And that is something which I didn’t expect from my uncle. Usually my uncle is the one that provides very good insights and advice. Yes perhaps controversial and challenging but with a clear message. I mean normally it is the neighbor that is very provoking, I expect this from him so I read his posts in a completely different light.

I guess what I want to say is that if you have a certain established role within your community or team and then suddenly change the intent of your messages then you will mostly get confusion instead of anything else. In this case I believe my uncle would be able to send out the same message but in a way that is clearly provoking you to think. And thus achieving the wanted results I am sure.

Because there is absolutely nothing wrong about being provoked into thinking. On that note I am sure my uncle will continue doing so, in new ways perhaps.

Pair Programming Continued

2010-01-16

In my previous post I talked about how I experience a really good pairing session, and in this post I like to discuss what I believe is needed for this to succeed.

Respect

As with any team activity you will need mutual respect for it to truly succeed. But with pair programming this needs to by lifted to a new level, just because of the very nature of developers and the intellectual type of work that they do. To much pride for your code is a bad thing.

We have all experienced the developer that gets personally offended when commenting on their code. This is a bad characteristic for a pair programmer because he will constantly be commented on his code. So respecting each other enough to know that the comments are never personal and are only made to improve the solution, or at least to discuss if it can improve. So you need to be very open to learning new things as well.

Honesty

One of the things that I tend to do is always ask people what they think about a particular thing I have done, this can be code, a presentation or my cooking skills; anything really. The reason why I ask this is so I can improve myself, but the problem I encounter is that there are very few people that will tell you that something is not done properly. People are always happy to say; yeah that was good, or I liked it. But I don’t learn from that, and I am not so self centric that I believe I don’t make mistakes.

So next time some one asks you for your opinion, tell them the truth, hey if they don’t want to hear it they should not have asked. But seriously if you see your pair make a mistake or just know a more elegant way to solve the problem then you should say so. Not doing so is showing lack of respect to your college, you are basically denying him an opportunity to improve. And you deny yourself this opportunity as well, because you will notice that you will be wrong at least half of the times.

If you feel that you cannot be entirely honest with each other then you should discuss this.

Communication

A constant dialog is how pair programming should feel like, you are continuously discussing the small problems that you are solving. While keeping an eye on the larger problem space. You are not pairing to have someone tell you when you made a typo and things like that, you want to have a sparring partner to discuss what would be the best solution to solve the problem at hand. You cannot do this without talking about it, so talk, and talk a lot.

Nothing is more annoying than pairing with someone who just types, and where you have to guess by reading the code what goes on in his mind. If this is what you want then you don’t need to pair program. Just hope that the other party makes his code descriptive enough so you can read it. This btw is still a strong requirement for code written my a pair.

Experience

And here I mean experience in pair programming, this is not something that you will just pick-up and be good at, this is something that you will need to learn. And the only way that you get this experience is by doing it.

But it also helps a lot if you are pairing with someone that is similar experienced in programming as you are, but I don’t want to make this a very important thing as you know how the saying goes: “A single idiot can ask more questions then all the wise men in the world can answer”. What this means is that as a very experienced developer you can still learn form the fresh insides a junior has to offer you.

A junior might be less eager to ask for the keyboard, but by showing him the proper respect this should be an issue of the past fairly quickly.

Fun

Fun! Pair programming should be fun, not a burden. I know that in the beginning it can be uncomfortable, but this should be improving quit rapidly. If you keep feeling uncomfortable then you should try to figure out why this is, and if you conclude that this is something that just doesn’t work for you then perhaps you should not be doing it.

But there is no way you can make that decision after only one month continuously pair programming with different people.

Equipment

Good equipment like a dual screen or a very big single screen (30 inch or so), a proper desk that allows you to sit side by side. Perhaps even two keyboards connected to the same machine so you don’t actually have to move the keyboard over.

Without these simple things pair programming can be very annoying. Think about it, you are equally involved so you should have equal access to what you are doing. If every few seconds you have to move your chairs because the other person is having the keyboard and needs to correctly see the screen, then that won’t work.

Finally

Just to recap, the single most important thing that should be present is respect, all the other things will then be worked out in someway or another.

Happy Pair Programming everybody!

Pair Programming Session

2010-01-15

Today I had a really enjoyable pair programming session with my college Øyvind. It was quite some time ago since our last session, which was when we were both working at a customer together.

We were there for a few months and during this time we experimented with different ways of pair programming. It was also the first time we where pairing together, and all do it went very well, and we both learned a lot from (and about) each others, by the different ways of working and thinking, it felt unnatural to me. Or perhaps I should say, not completely right.

It was too much guided by rules, we for example tried ping pong; which means that you change from being a driver to a navigator between each and every test. But we tried other approaches as well to determine when to switch between roles.

But today was different, completely different I have to say. Today we were not driver or navigator, today we were both at the same time and constantly. Sure only one of us had the keyboard at the same time, but the switching was so rapidly and irregular that there was no real pattern to discover (not that I was looking for one). One only had to give a slight hint that he wanted to try something and the switch was made, and the same applied to passing back the keyboard.

If I had to compare it to something else I would say that it felt like we where both standing in front of a whiteboard heavily discussing the design and what the next steps to take would be. Both equally involved and drawing together to make the picture complete.

Needless to say that I felt very energized during the session and drained from energy afterwards, in short this is for me what pair programming is about.

I also recommend you to read this great explanation on how pair programming really works (PDF).

You are responsible

2010-01-14

Many have said it before, and I’ll repeat it once more; as a developer you, yourself are the main responsible person for your own education. So not your employer, your mentors or anybody else. You are. This is one of the principles that is central to the software craftsmanship ideas.

So now we got that out of the way lets look at a particular way that I find very effective.

Software Conferences.

Software conferences offer the unique opportunity to first of all listen to some great people in the software industry talk about the topics that they are passionate about. Secondly you can have great discussion with the speakers, get more details about the topics that interested you.

I think that this might be the best way of introducing new ideas, you won’t know all the ins and outs, but you will know enough to determine whether or not to continue investigating them.

But

Lets go back to that it is your responsibility to educate yourself; so you want to go to one of these great conferences like for example; QCon, NDC or Øredev. These conferences are very focused on software craftsmanship and learning, but when I look at the costs of these conferences then I realize that it is almost not possible to be responsible for going to these events yourself. It is easy to spend 1.500 to 2.000 Euro on a three day conference including travel and hotel.

So these big conferences are really targeting companies, and they don’t really consider that there are many developers that cannot go because their employer is not willing to pay for their education.

Now I completely realize that having these conferences costs a lot of money, and also that if they would give some sort of discount for developers that pay their own ticket it would be misused by more then a few companies.

But honestly paying the 1.005 GBP for the QCon London event does hurt my wallet, I hope it is worth it :) Please note that this post has been tagged as a rant, but I hope it was an ok rant.

Snow

2010-01-10

You wake-up in the morning open the curtains and look outside, and there it is; everything you see is covered in a nice layer of fresh snow. Beautiful if I may say.

When you go outside you realize you are the first person to go outside; there are no existing tracks except from maybe a few deer tracks. Then when you start making your way out, every step you take leaves a new footprint in the snow, it is lovely, but it takes quite a bit of effort. Sometimes while making your way through the snow you have to take a few steps back and try a new route until you reach your destination.

Other people that woke-up after you, or just didn’t feel like being the first to go outside will follow the initial trail, after a little while a clear path is made. Walking becomes easier, but you still have to watch your steps a little bit, and there is plenty of room to explore other paths.

Then the majority of the people come outside, and they just follow the path created by the those before them, all goes well and walking is relatively easy. But this is also where things start going wrong, because it seems easy they don’t look where they walk, and stepped on snow gets very slippery, so they fall.

Now this is going to be a small stretch, but there is something to be learned here:

It is great to create your own tracks in fresh snow
When things seem easy that is when you should be alert

I for one love to walk in fresh snow, but unfortunately I am not a morning person, so usually I am not the very first, but I am very happy being the second person outside.

Test Driven Spikes (TDS)

2010-01-04

I have had a discussion with a college of mine about spike code and how to bring the chosen solution into production code and quality. Jeremy wrote about it a while back stating to never check-in spiked code, and I was wondering if I could challenge that a little bit.

Now guys, don’t look at me in that way, I mean it well …

What is spike code?

A spike solution is a very simple program to explore potential solutions. Well I like to add that I think a spike is relatively small, if it becomes very large I am wondering if we are not talking about prototyping instead? In my experience a spike is perhaps a days worth of coding, but then that it is rather much, more normal is an hour or two. Within this time there could be several attempts to get the desired result.

There is spike code and there is SPIKE code

Jeremy is talking about spike code to learn an API or something like that, in that case you don’t have a real use case that you need to solve so there I completely agree and it should be deleted as soon as you have learned what you needed to know (or perhaps checked-in into a special spike area). I would not really consider this writing spike code anyway.

Also code that is really hacked together and not following any good design principles, there adding tests afterwards would not give you anything, if at all possible. Just throw away that code and start over again.

Test Driven Spikes (TDS)

First of all sorry for another three letter acronym.

When I write a spike I almost always drive the execution of this code through tests, tests are dead easy to fire-of and they give you instant feedback. It is also very easy to have several tests next to each other exploring different angles or approaches. So while creating my spikes they are sort of driven by tests. And I use tests to verify that my spike code does what I expect from it. It has been a very long time ago that I created a console application to test or try-out some code.

But those are not spikes!

Well what is different here from actual Test Driven Development Design (can we just start calling TDD; Test Driven design?) is that I do take shortcuts, not all angles are covered correctly and my tests are not very descriptive and readable. So it is not production ready code. Also the actual code should be cleaned-up, different naming and removing the new keywords if they are any at all.

For example: I would write a specification (or test if you will) that dictates that I want such and so to happen when doing this and that. I do this from a fairly high level, so most likely not directly on the class that does the specific work, but it could be. Anyway while trying to get this to pass I run into a little problem. The solution is not very straightforward and I am stuck.

So for me that is when the spike begins, the existing test remains, but I ignore it for now. I probably create a similar test but now much closer to the actual problem itself. And from there I start playing with the code.

I would also try to take small subsets of the problem and try to fix that independently from the bigger problem area, again using tests. So when have a few of these smaller subsets working I try to put them together. I refactor the code and tests to get my code working the way I need it to work.

When the spike goes bad I sometimes completely delete all the spike code and start over again, but more often I delete subsets of the spike and redo those. Then when I get the functionality that was needed in the initial specification I hook it up, and verify that it works.

When it does then I go back to the spike code and start cleaning it up as mentioned before, then I could also add some more tests for some edge cases, but more likely is that those edge cases would be discovered by sequential specifications, meaning actual use cases that are needed.

Also I am a bit weird

I am really bad at really hacking code together or I am just too stupid, but if I don’t clean-up after myself I lose the bigger picture or overview. Even in spikes, if I have the same code in a few places it annoys me, so I refactor it into a single method or into its own class. The same goes for naming, I don’t use variable names like a, b and c that makes no sense to me, also ReSharper makes it a no brainer anyway. Having unclean code just feels wrong to me.

Its the same with other things in my daily live, some stuff just needs to be true, like the car keys need to be on the desk. I can place them somewhere else, but next time I see it I move them to the right place. The volume knob in my car needs to be in a certain position, if it is not I use the volume on my steering wheel to adjust the knob without the volume. So yes I might have issues!

I think good design in spike code is important as well because it gives a better overview of what is going on, and if there is a place where you really need that then it is in code that you don’t really understand yet, i.e. spike code.

So in the end my spike code might actually get a much fine-grained test suite then the code that was not spiked. And this might actually make the test suite more brittle around this code, so it is not always a good thing, but for the time being I leave them where they are. Which is in a different place from the actual specifications so they are recognizable as spike tests.

From memory

Update: This paragraph came to me after having slept for the night.

I often hear that when deleting the spike code and starting over again you come-up with a much nicer solution then you finished with in the spike.

Now lets say you work like this and delete the whole spike code and started from scratch, don’t you think that the tests you will write are very much towards the already thought of solution? Can you really disconnect yourself from your previous attempt enough to come-up with better tests and potential better solution? I doubt it. Perhaps this will be true if you indeed really cut al corners, but even then the solution will be to close in memory.

Conclusion

So maybe we can conclude that I don’t know how to write spike code? Or that I am doing it all wrong? I would love to hear your opinions about this.

Specifications

2009-12-22

The RSS reader you are using doesn't support JavaScript; because of this you will not see the embedded code gists. Consider opening the post in your browser instead.

I received a couple questions about the Specification Framework that I use in the CQRS example and thought lets talk about that for a bit. The first thing that should be underlined is that this is not a framework, they are a few classes and extension methods that rely on NUnit for the actual assertions and and Moq for mocking of the dependencies. I got the initial bits from Greg Young at his DDD course which I extended a little bit for my specific needs.

I have the following base test fixture classes:

BaseTestFixture
BaseTestFixture
AggregateRootTestFixture
CommandTestFixture<TCommand, TCommandHandler, TAggregateRoot>
EventTestFixture<TEvent, TEventHandler>
PresenterTestFixture

These different classes are all very specific towards a specific need, which is a direct opposite from what a framework usually provides.

Black box

I try my best to make my tests treat the subject under test (SUT) as a black box, meaning that in my tests I don’t directly interact with the actual class that I am testing. Instead I want to trigger the behavior by executing behavior that lies further outside. The behavior that triggers the behavior on the SUT may be an actual implementation, or it could be a fake.

The same applies to the result of the behavior that gets tested, instead of verifying some state in the SUT I want to verify what happens outside of the SUT. So what matters is that I test the behavior not the state of the subject under test.

I also try to get further away from the SUT then its immediate usage. Doing so makes the tests less brittle for change. This in itself is not always an easy task, but I recommend you try it anyway.

The BaseTestFixture

This is the simplest test fixture class that I have and I use this for and I actually don’t use this anywhere in the example code, but it serves a really good basic overview of the semantics that are shared among the other test fixture classes.

It is following the Given When Then (GWT) approach, and as you can see it is really simple. Also note that I introduced some other named attributes by simply inheriting from the default NUnit attributes, this was purely done to stay with the GWT approach.

Below here is an incredible KISS example of how you would use this BaseTestFixture, which I believe doesn’t need further explanation. (I know I am misusing the term KISS here, but I thought if was fitting anyway).

The BaseTestFixture

Now we are getting into a more interesting case because now my subject under test is actually provided by the generic parameter of the base test fixture class. And to be honest this class is only used with 12 specifications out of the 122 specification classes. This is mostly because it is still a very generic solution, but again a nice way to ease into it.

Wow there is a lot more going on here! You are right, because here I make the base test fixture responsible for instantiating the subject under test, including providing mocks for any dependencies that it may have. So it is an auto mocker as well, but the interesting part here is that it puts a reference of the injected mocks in a collection that you can access inside your tests by using the OnDependency method that returns a Moq object.

Take a look at a specification using this base test fixture class:

So the first thing you see is the method SetupDependencies that is requesting the mock object for injected type IReporintgRepository and it is using the Moq way of setting up the Mock object. This is only needed when in your specification you need the mocks to be setup in a specific way. I intentionally separated the SetupDependencies from the Given as they may be two different things. And in the actual test you see the usage of the OnDependency method again where its being used to verify that something was indeed triggered on the injected class.

Now indeed this is not really treating the subject under test as a black box, for example in the When I make a direct call to a method on the subject under test. So my test knows about this now, meaning when it changes I need to change this test as well. So lets take a look where I go a bit further into the black box mentality.

The PresenterTestFixture

Here I am not going to show you the code of the PresenterTestFixture implementation as it is almost identical the the previous base test fixture. So lets go straight to an actual specification:

Again there is the setting up of a dependency in the beginning and then there is the Given method, Presenter in this case is the subject under test, so you can see that the specification still knows about the SUT in the Given, but you will notice that this is not used in either the When or the Then.

Hey what is that On thing in there? Well that is a very small DSL wrapping the Moq API. I did this to make it slightly more readable, and in this case it is very adapt towards working with a view and presenter. In the Given I am setting up my IClientDetailsView with the correct data, but I am also simulating that an event was triggered. This is not logic that the test is concerned about, all we do here is bring the view and the presenter in the correct state for this particular specification. So instead of setting these things directly on the presenter this will all be directed from the view.

Then in the When we again fire an event from the view, but in this case it is the going to trigger the behavior on the presenter that we want to verify. And in the Then methods we verify on the view again that the presenter actually did the correct things, but we also verify on other dependencies that the correct methods where called, in this case the IBus.

I am not completely happy with the mini DSL yet, but I think it is cleaner then the default Moq API. Just for those that are curious, here is the mini DSL which gets returned by the On method:

CQRS and Event Sourcing

By combining CQRS and Event Sourcing we get an architecture that is very suitable for black box testing its behavior, which was a real eye opener when Greg demonstrated this to me. He says that the way to bring your aggregate root back into the desired state is to playback the events that are needed to do so. Then you can execute the behavior on the aggregate root, and finally to actually verify your behavior you retrieve the published events and verify that they are as expected.

Now the beauty of this is that the setup and the verification this will work on any aggregate root because we are using the IEventProvider interface there that they all implement. The only actual knowledge about the aggregate root that remains is the specific behavior that you trigger.

But I went a little bit further then what was shown during the course, I am saying that instead of executing the behavior on the aggregate root we could also just provide the command that should trigger this behavior to be executed. Because the command would be handled by a specific command handler which in turn would execute the domain behavior.

Now below here is the command test fixture that allows me to do just that, I need to provide the actual command, command handler and aggregate root types that are to be used in this specification.

Please note that the Given method now returns an IEnumerable this is to be used to provide the events that are needed to bring the aggregate root into the correct state for this specification. This is using the exact same mechanism as the actual code uses to make state changes in the aggregate root, so there cannot be a case that you are testing your aggregate root using a state that it cannot get into.

The When method returns the expected command, so all you do there is create the command with the correct information and return it to the specification.

Then in the try catch block you may have noticed that a command handler is executing the provided command and that after that the events are being retrieved from the aggregate root. These events are what you would verify to make sure your domain behavior is correct.

But this may all sound very abstract, lets look at a simple specification and see how clean and readable is really is:

So you provide historical events to bring the aggregate root into the expected state, you fire of the command, then you verify the published events to ensure your domain behavior is correct. If you choose the correct naming for your events and commands, then a business person would be able to understand the specification. Especially if you parse the text and do a little bit of formatting.

Where is Should?

Hey what are all those WillBe and WillBeOfType things that I see in your specifications, should they not be ShouldBe and ShouldBeOfType? Well I used to think so as well, until I attended a presentation by Kevlin Henney at NDC where he explained that Should is not specific enough. Should indicates that it might not. I like to use the example; “I should really do the dishes, but I won’t”. By using Will Be and Must you are much more dictating what will or must happen, its not a question anymore.

Finally

I am completely taken by this approach and as you see you don’t need a big BDD framework for this. I think using something like this gives a good learning experience before going towards an actual BDD framework like MSpec. Also Uncle Bob just wrote a good post about to not abuse the Given When Then approach and also take a look at the Mocks aren’t Stubs article by Martin Fowler.

By now you must understand that I like to throw with code examples so yes the post is very long, but I hope it provides enough value instead of just confusion.

Using conventions with Passive View

2009-12-19

The RSS reader you are using doesn't support JavaScript; because of this you will not see the embedded code gists. Consider opening the post in your browser instead.

I was reading Ayende’s blog post about building UI based on conventions and thought; hey I have something similar in my CQRS example. And since this is the least interesting part of the whole example I guess it will be missed by many, and I can’t let that happen.

Passive View

The example has a Win Forms application in there that is build accordingly to the Passive View pattern, so my actual forms are being dumbed down to simple views without any logic in them, well almost no logic. Then I have presenters that have the actual logic in them, or delegate the logic to other responsible entities (services or whatever).

The reason for doing this is because you want to be able to test the behavioral parts of your code as simple as possible, and nothing is more simple then being able to unit test your behavior, hey even better drive your design / behavior through Test-Driven Design (TDD). Imagine how hard that would be to do when not abstracting these different responsibilities from each other (hehe I am sure some of you don’t even need to imagine this ;) ).

I realize the Win Forms is so not in anymore and that nobody uses them, but perhaps what I show you here makes you think about other areas that this could be applied to.

The View

The view is responsible for displaying information to a user, capturing user requests, and … uhm no that’s it. Let’s take a look at a simple view.

So what is happening here? As you can see there are two public events declared and there are two properties that provide access to some form controls. The interesting thing here is that the two public events are wired to two events from two form controls. But hey nothing is happening; no behavior, no calls to services or anything. Below here is the declaration of the interface that the form implements.

Here are both the two public events and the two properties that provide access to two form controls, of course the implementation could be anything. This interface is used by the presenter to control the view.

The Presenter

So the presenter is responsible for controlling the view; which includes setting and retrieving data and executing behavior that is triggered by the user of the view. Below is the presenter that is responsible for managing the previous mentioned view.

For simplicity I removed any other external dependencies and replaced the behavior with a comment “Do something”. Talking about external dependencies; the interface that is implemented by the view is injected into the presenter, this is interesting.

Look at the Display method, there you see that the view that is injected first gets its data loaded and then actually gets activated. This means that the view will not get its own data, but that the presenter will provide it to the view, for this the presenter will use the two properties that where declared in the interface. And it means that the presenter is responsible for activating the view. This is different from how Win Forms works out of the box.

For this to work I had to change the Program class, here take a look:

Look at line 17 in there, the method Display is called on the presenter, which then prepares and activates the view.

But I can hear you think; what about these two events that where declared on that interface as well, you know “OnCreateNewClient” and “OnOpenSelectedClient”? I can see some obvious candidates “CreateNewClient” and “OpenSelectedClient” but they are not wired-up together. What gives?

The Magic

You must have noticed that the names are very similar and that they follow a certain pattern, this is the convention that I have chosen to use. Basically I call my event handlers the same as the events without the “On” prefix. Then I have a base presenter class without the word “Base” because that would be EVIL. And this class will wire-up the events with the event handlers for me.

The abstract base class Presenter needs the view interface as the generic parameter of the view that the presenter controls. A reference to the actual view will be injected into the presenter class. Then the first thing that happens is that I get all the events declared from the provided interface. Then I get the actual events from the provided view, but only those that have been defined on the provided interface. This makes it possible to define multiple interfaces on the same view that get controlled by different presenters. After this I get all the public methods from the presenter.

Once I have the events form the view and the methods from the presenter then I can start matching them together. As I mentioned before in my case I use a very simple convention where the event is prefixed with “On” so in order to get the event handler I only need to search my event handler collection for a name of the event minus “On”. Then finally the event gets the event handler added to its collection of event handlers.

When there is no event handler for a provided event then I throw an exception, because I consider this to be a bug. There might be event handlers that does not have an event associated with it, but I consider that less harmful since this logic would not be called anyway. This exception will be visible in the unit tests for the presenter.

Reflection

Yes this relies very heavily on reflection, but for this scenario I don’t mind. Indeed it is slower, but the question is; will you notice this when displaying a form, and I don’t think you will. You could improve this code by for example making the WireUpEventAndEventHandler an action and cache those for the combination of the presenter and view interface, but I don’t think that is worth the effort.

The Specifications

I am going to leave you with the specifications that I have for the presenter that I used for this post. The whole presenter is being tested by setting data on the view and triggering events. I am not calling the methods directly on the presenter itself. And if you want to see more, then get the code from GitHub.

CQRS Domain State

2009-12-08

This morning Aaron Jensen asked a really interesting question on Twitter “Should Aggregate Roots en Entities always keep their state if it is not needed for business decisions? Is firing events and relying on the reporting store enough?”. He really made me think, Aaron thanks for that!

So for example you have some behavior on your domain that gets called when a customer moves, this behavior will publish a Customer Moved event containing the new address. If the domain does not use the address information for any decision making, does it then need to be persisted in the aggregate root?

I guess it would greatly depend on whether or not you are using Event Sourcing to persist your published events (this is what I would suggest). If you do then I don’t see a problem in skipping storing the address information in the aggregate root. Because if at a later time you need the address information to make some decisions then it is easy to retrieve this from the events when reloading the aggregate root from the event store. You just create the address property and an internal event handler to process the Customer Moved event. Just make sure you delete the snapshots first.

But if you do not store all the events that you publish, but instead store only a snapshot then I would not skip storing the new address information, because in the end the domain is responsible for the state, not the reporting side. In other words, you can rebuild the reporting side from the domain, but you cannot necessarily do the same in reverse. So if you don’t store the address information and you need to rebuild the reporting store then you cannot do this.

So my conclusion is that the information should be available on the domain side. And when using CQRS and Event Sourcing to store the events instead of the internal state of the domain then that makes it possible to skip having the information in the domain structure, else not.

CQRS Trying to make it re-usable

2009-11-28

The RSS reader you are using doesn't support JavaScript; because of this you will not see the embedded code gists. Consider opening the post in your browser instead.

If you have been following the source code changes on GitHub you may have noticed that I renamed the folder Fohjin.DDD to Fohjin.DDD.Example, my intention is to not make anymore changes there. Instead I have created a new folder next to it and in there I am rebuilding the same components but now with re-use and ease-of-use in mind.

The first thing that is very obvious in the example code is that the domain is not very persistence ignorant, something that is valued a lot in Domain-Driven Design. So this is something that I wanted to try to address first. I really like the way NHibernate makes our code persistence ignorant and am attempting to solve this in a similar manner, using Castle DynamicProxy.

Since I am a complete Castle DynamicProxy noob I asked the help of Krzysztof Koźmic who is an active committer on the Castle project. Krzysztof was gracias enough to get me through the basics in .Net proxy-ing. For those interested in this technology I would also recommend reading his excellent blog series about the same topic.

What I want to achieve is that you get back your poco and I am going to show you one suggestion that I have working now.

For comparison I’ll show you here how an aggregate root looks like in my initial example code (it is stripped down to only contain one behavioral method):

And here is the same aggregate root but now more persistence ignorant:

Ok this may seem a bit strange at first, I mean we are sending an event into nothing and we don’t set our state anymore. Let me try to explain :)

Convention over Configuration

Well in order to make this work I am going to insist on some conventions, in fact it is going to be very opinionated. If you want to learn more about convention over configuration then I would suggest reading Jeremy Millers article on MSDN. Personally I am all in favor for some strong guidelines so lets just go over them and see if they make sense to you as well.

Mandatory methods

The protected virtual method Apply and the protected method RegisteredEvents are both mandatory if you want to instantiate the aggregate root using this approach.

In the RegisteredEvents you return all the events that the aggregate root can publish, this is to let the persistence code know what it can expect. So when you add some new behavior that will publish a new event then you just add it to the list here. I think that you can sort of think about this as the mapping classes in NHibernate.

The Apply method is used to publish the events from the domain behavior, but as you can see it has an empty method body, so nothing will happen, right? Here we use Castle DynamicProxy to intercept calls to this method and replace the behavior (non-existing) with our persistence logic. Where this logic comes from I’ll explain in a minute.

Internal state needs to be protected virtual properties

One other difference with this approach versus the example code is that instead of private fields or private properties for the internal state, it now needs to be in the form of protected virtual properties. The reason for this is because state will not be managed by the aggregate root anymore, but instead by something that we add using Castle DynamicProxy and we will be using interception to map the properties to the internal representation of the state. The code even throws an exception if the aggregate root tries to set the internal state directly.

The event store

I haven’t done anything with the event store yet, I don’t think the functionality will change much from the code in the example project. One thing that will remain the same is the dependency on interfaces, and not on specific implementations. This is obviously because it is good design practices. And it will enable us to mix this new approach and the approach using the BaseAggregateRoot class together with the same event store. The BaseAggregateRoot is discussed in detail here.

Mix-in magic

So after reading the previous chapter you would probably say, but this aggregate root doesn’t implement anything? This is true, but when our repository is instantiating the aggregate root it is also adding or injecting two more classes into it. One class that implements the IEventProvider interface and an other class that implements the IOrginator interface. This is called mixing-in classes and by doing so the aggregate root will implement both these two interfaces and it will re-direct calls to it to the mix-in classes. So as far as our event store knows it just is a class that implements the needed interfaces.

Why have everything protected?

Well in order to be able to intercept members using Castle DynamicProxy these members need to be visible to the interceptor. The most private way that is available to us is protected, and the reason why I want this to be as private as possible is because this is not domain logic, and thus should not be visible to the users of the domain. For the same reason you may notice that for example the IEventProvider implementation has all the members implement the interface explicitly thus hiding them when the domain is used and only showing them when the domain is explicitly casted to the IEventProvider interface.

I have been thinking about providing a basic interface that dictates that the Apply and RegisterEvents methods needs to be present, but I like this approach better as from the outside these implementation details would not be known, but I could imagine being able to support both approaches.

Reflect only once

There is quite a bit of reflection going on to discover and the properties of both the domain events and the aggregate root internal state. But once this is done once the results will be cached and re-used for each sequential need. But I am sure this can improve quite a bit :)

Conventional limits

So what if I run into the limits of this conventional approach? Well the Event Store only cares about that the different interfaces are implemented, so you will be able to switch back to the BaseAggregateRoot solution at any time. This does mean that you will have to do some more coding, but you will have all the freedom to do what ever you need to do. And the different approaches can be used side by side.

One other limit is that if you instantiate the aggregate root yourself nothing will be intercepted and mixed-in so no state will change nor will the events be published.

Current state

Currently the code only supports very basic usages and there are some challenges ahead of it being truly useable. Think for example about other entities in the same aggregate managed by the aggregate root and collecting their events as well. Also think about adding and removing entities from a list which may need some configurable conventions so you can decide when something needs to be added or removed.

I am thinking about the ability to provide adapters to be able to handle certain state changes differently from the conventional way. One that I see a need for already is the ability to set the aggregate root Id, since this is not managed by the aggregate root but by the event provider mix-in.

Finally

Currently the functionality is very limited but I have tests passing proving that this is working :) so what I would like is to get some feedback on this approach. And yes I realize that this is very opinionated, and will not fit everybody’s approach. Having said that, keep the suggestions coming anyway.

Convention over Configuration

2009-11-28

Just a quick note on Convention over Configuration, I believe this is one of the more useful practices that you can apply in your codebase. So what is this all about then? Well think of about everything you do in life; think for example about “opening a door” or “turning on the water” we all exactly know how to do those things, and because of that it is a fast action. It is something that we don’t have to figure out just before doing it, again and again. These are conventions.

When we start applying these sorts of ideas to our code base this would mean that certain operations need far less thought as they will be the same each and every time. A good example here is mapping the controllers and view together and extracting the URL from the controller name in MVC. If you are a developer that knows these conventions then you know exactly where to look for the code of a certain URL. No need to figure this out each and every time.

Conventions also bring an opportunity to automate things within your system, because you are following certain rules in your code, you can write some other code that does something accordingly to these same rules. Configuration is a area that greatly benefits from conventions. Instead of having to manually configure how each different part works together you can write some code that configures this for you. You can also write some tests that verify your conventions.

Configure your Conventions

Many frameworks apply conventional thinking in their code, making it easier for their users to use it. There are frameworks like Ruby on Rails that rely very much on conventions, the only thing here is that these conventions cannot be changed very easily. This makes the framework very opinionated. This is not necessarily a bad thing in fact this means that any Ruby on Rails developer can jump to any other Ruby on Rails project and start working on it (sort of).

But frameworks that allow you to configure your own conventions are much more powerful, because not all conventions make sense in every scenario. FubuMVC is one such framework where we try not to be to opinionated (we are) so that you may configure your own conventions.

Not just for framework code

I try to figure out what conventions that are in a system that I am working on and make them explicit as soon as it makes sense. The benefits are clearly there in my eyes, being able to test them and apply them is great. This is not something you would only want to apply in framework code, for example the win forms application that I created in my CQRS example also uses conventions to hook up the views with the presenter. Very easy and I don’t have to configure it manually ever again.

Now adding a new event to the application is like turning on the water or opening the door.

CQRS Domain Events

2009-11-20

The RSS reader you are using doesn't support JavaScript; because of this you will not see the embedded code gists. Consider opening the post in your browser instead.

As you may have seen in my previous post “CQRS à la Greg Young” now our domain aggregate root is responsible for publishing domain events indicating that some internal state has changed. In fact state changes within our aggregate root are only allowed through such domain events. Secondly the internal event handlers are not allowed to have any sort of business logic in them, they are only supposed to set or update the internal state of the aggregate root directly from the data the event carries. Using these rules you completely separate the business logic from the state changes. This separation enables us to replay historical domain events without any business logic being triggered bringing it back to the same state as the original aggregate root.

Why is this important? Well now we can use these same domain events for our persistence using an Event Store, this pattern by Martin Fowler is called “Event Sourcing”. Obviously you don’t want to process a credit card or send an e-mail every time you load an aggregate root from the event store. Also from the time the original domain event was recorded and when the aggregate root is loaded from the event store the business logic that decided a state change was needed could have changed, this should not affect the actual historical state change. So this separation is to be taken seriously.

Domain events can also be used to signal something to the outside world (taken from the aggregate roots view point) that something has happened without having an actual state change. When persisting our domain events we would not differentiate between those two different domain events.

All domain events should be named with the ubiquitous language in mind, meaning that they should closely represent what the user intended to do in the same language as the user would use to explain it to you. By keeping all these domain events we gain a huge amount of knowledge about what happened and why it happened.

This means that our aggregate root gets the added responsibility of tracking these domain events, but I don’t see this as being any different then for example the proxy that NHibernate generates for you, except perhaps that it is not a proxy and that you have more control over what happens. But it is true your aggregate root has these added responsibilities.

So let us take a look at how the aggregate root provides this functionality, for obvious reasons we use a base class for this, but really all that is needed is that the aggregate root implements the following two interfaces:

The IOrginator interface is for the snapshot functionality which is an optimization technique for speeding up loading aggregate roots from the Event Store. As you can see it is using the “Memento” patter from the Gang Of Four book. I wanted to get this interface out of the way first; it is not needed, but does provide a good optimization for loading aggregate roots.

The IEventProvider interface is the most interesting interface of the two, this one defines how domain events can be retrieved from the aggregate root and how historical domain events can be loaded back. It also defines that each aggregate root must have an Id and a Version, both of these are used by the event store, the Id is obvious so I won’t go into that, the version on the other hand may not be that obvious. The version is used to detect concurrency violations, meaning this is used to prevent conflicts that occur because between the time the command was send and the aggregate root was saved an other user or process has updated the same aggregate root. In this case we would throw an Concurrency Violation Exception which currently results in a rollback. In a future post I plan to look into how you could try to deal with these concurrency violations automatically.

Using domain events for state change

Now we will take a look at how these interfaces are implemented and how the implementation is used. The way I am going to go through the code is as if you where using R# going from method to method. So in order to get a more overall impression I would encourage you to look at the source code. The source code can be found here: http://github.com/MarkNijhof/Fohjin

Each aggregate root has to register the domain events and the internal event handlers with the base class. I am working on getting this as static information for the type since this will not change between different instances of the same type. As you can see I am registering the domain event type and an action to handle the specific type. As you can see the actions have the specific domain event as an input parameter.

The RegisterEvent method is defined in the BaseAggregateRoot class, here is a little bit of interesting logic going on where basically a new action is defined that has an IDomainEvent as an input parameter, that is how they can all be stored in the same Dictionary. Then inside this action the provided action is invoked where the input value is cast from an IDomainEvent to the actual expected type TEvent.

So if we would write the example out of what is actually happening for the Client Created Event then it would look like this example below, and this is what is being stored in the _registeredEvents.

And below here is the private apply method that is being called from two different methods in the aggregate root base class. The method retrieves the action that is registered for the provided domain event, and than it invokes the action with the domain event as the input parameter. The apply method takes an IDomainEvent instead of the specific domain event and because of that we have the above mentioned logic.

Let us take a look from where this apply method is being called, first we will look at some actual domain behavior in the aggregate root.

First we execute all our valuable domain logic, the business behavior. Then when we are satisfied that everything is ok and we know what type of state change we need to execute and we Apply a new domain event with the new internal state. The Apply method used here is a protected method on the BaseAgregateRoot. Btw there is nothing stating that there can only be one outcome, i.e. only one type of domain event being Applied.

When we Apply a domain event we will first assign the aggregate root Id to the event so that we can keep track to which aggregate root this event belongs to. Secondly we get a new version and assign this to the event, this is to maintain the correct order of the events. Then we call the apply method which will make the state change to the aggregate root. And finally we will add this domain event to the internal list of applied events. This is very similar with the dirty check of NHibernate (the idea, not the actual implementation).

This is all what what is needed to execute domain behavior and keep track of the domain events that have been used to update internal state of an aggregate root.

Getting the state changes

So now we have build up some internal state changes and we want to persist them to some sort of medium. I am not going to discuss how to actually persist these state changes, that is for a later post, all I want to show you now is how to get them out of the aggregate root. How about we start with the method GetChanges :)

Here we simply return all the applied domain events, by tracking these domain events we in effect track all the state changes that have happened since the aggregate root was instantiated. Here we also request all the applied domain events from all entities. Than new order the domain events by version. After having received and processed all the applied domain events we should Clear the aggregate root from all applied domain events so they won’t be persisted again.

Just a quick word about the entity event providers, in some cases you want to have domain behavior inside entities that are part of the aggregate but are not the aggregate root. Well in this case you want to have those entities generate domain events as well, and you want to get those as well when getting all changes. Same when clearing the domain also the changes inside each entity event provider should be cleared.

Before saving the changes in the aggregate root is finalized we also need to update the version of the aggregate root. This version will match the version of the last applied domain event.

Loading historical domain events

In a next post I’ll dig deeper into the event store but for now lets just assume you have a very nice way of storing these domain events and have the ability to retrieve them again.

As the IEventProvider interface nicely dictates there is a LoadFromHistory method that takes an IEnumerable below here is the implementation of this method.

When you take a look in the for each loop you will notice that here we are calling apply again, please note that this is the private variant which is responsible for updating the internal state of the aggregate root and that these events are not added to the _appliedEvents collection, nor is the Id or Version updated. We don’t want to save these events again. After applying all the historical domain events we update the aggregate root version to the version of the last event.

The base class

I am using a base class to provide this functionality to all the different aggregate roots using inheritance and I know there are different opinions about this. So I wanted to highlight that you can achieve the same results by using composition. Your aggregate roots still need to implement the interfaces, but the implementation can be provided by using composition.

An other thing is that all the public methods in the BaseAggregateRoot are explicate interface implementation. I do this because I want to hide these details for any piece of code that is using the aggregate root as is, only when dealing with persistence we use the interface and get access to the explicit interface methods. Nice and clean.

Aggregate entities

Ok I have already mentioned the term entities and entity event providers, and I would like to focus on them for a bit. An entity is an domain object that is part of the same aggregate as the aggregate root, but is not the aggregate root it self, is however it is managed by the aggregate root. We manage state changes in these entities in the exact same way as we do this in the aggregate root, so each entity is also an event provider. There is a different interface for the entities then for the aggregate root so they can not be confused among each other.

You may have also noticed the hookup version provider method, remember it :)

The problem is that we should deal with the whole aggregate as a whole so all state changes need to be persisted within the same transaction. So we want to have a single point to access the internal state of the whole aggregate as well as a single point to load the history back into the whole aggregate. In order to achieve this the aggregate root needs to register all the entity event providers so it can track them. To enable that we have an additional interface for our aggregate root to implement.

Lets look at the implementation right away

Now this is only a very simple collection that hold references to IEntityEventProviders which is used in the previous shown GetEntityEvents method. So getting out the changes is relatively simple this way. I also created a special collection that will automatically register entities when they are added to the collection.

Remember I mentioned the HookUpVersionProvider method, well this is used to get a reference to the method from the aggregate root that deals with assigning a new version to each event. We want a reference to the same version generator that the aggregate root uses because then all the versions of each domain event will be in sequence independently is the aggregate root or an entity created it.

The collection takes a reference to the aggregate root it is part of in the constructor to be able to add each added entity event provider to the collection in the aggregate root.

Currently there is some duplication between the BaseAggregateRoot and the BaseEntity because they are inherited from different interfaces, I am sure there is some optimization possible there :)

There is finally one more thing to cover about this and that is how historical domain events are being passed into the correct entity, currently this is a bit ugly but I am working on a more elegant solution. Take a look at the code.

So each entity domain event will be registered here as well and are all passed to one specific event handler (one specific event handler for each different entity type). What happens is that we check if the specific event provider is present in the collection by looking for the Id, when the requested event provider is found the historical domain event will be loaded using the load from history method on the event provider.

The problem here is that the information which events the entity can produce is already registered, and that is in the entities them self’s. So by making that information statically available I should be able to auto register the entity event handlers.

Finally

I hope that this was a useful explanation of how the aggregate root works with the internal domain events, and that you would agree with me that it really is not very difficult. Next time I want to discuss the event store so we can take a look at persisting the domain events. I will also post a blog about making the entity registration more elegant and less manual work.

CQRS à la Greg Young

2009-11-12

The RSS reader you are using doesn't support JavaScript; because of this you will not see the embedded code gists. Consider opening the post in your browser instead.

I have had the pleasure of spending a 2 day course and many geek beers with Greg Young talking about Domain-Driven Design specifically focussed on the Command and Query Responsibility Segregation (CQRS) pattern. Greg has taken Domain-Driven Design from how Eric Evans describes it in his book and has adapted mostly the technical implementation of it. Command Query Separation (CQS) was originally thought of by Bertrand Meyer and is applied at object level

Bertrand defines CQS as: every method should either be a command that performs an action, or a query that returns data to the caller, but not both. In other words, asking a question should not change the answer.

Greg however takes this same principle but he applies it to the whole architecture of a system, clearly separating the write side (Commands) from the read side (Queries) of the system. The write side is what we already know as the domain, containing all the behavior which is what makes the system valuable. The read side is specialized towards the specific reporting needs, think for example about the application screens that enables the users to execute domain behavior, but also any traditional reporting needs are provided by the read database.

So lets take a quick look at the overall architecture before we dive into the details:

You have to excuse me for not using Visio to make this drawing, but I really didn’t feel like tackling yet an other layer of complexity tonight. At least I did the labels in typed text, so it is readable. I will discuss this architecture in 4 phases, the order I choose is what I believe is the natural way your would think about this. In the end you will see that the whole principle is rather simple. Really.

Queries
Commands
Internal Events
External Events (Publishing)

In order to understand this type of architecture better I decided to build an example application; I will use this example in this post to demonstrate the different aspects of the architecture to you. The example has already been released in the wild (ok I meant the DDD group on Yahoo) and can be found here: http://github.com/MarkNijhof/Fohjin

Queries (reporting)

The first part I would like to discuss is the reporting needs of a system, Greg defines any need for data from the system as a reporting need; this includes the various application screens which the user uses to base his decisions on. This seemed like a strange statement at first, but the more I think about it the more sense it makes. This data is purely used to inform the user (or other systems) about the current state of the system in the specific context of the user so that they can make certain decisions and execute domain behavior.

These reports will never be updated directly by the consumer of these reports, the data represents the state of the domain, so the domain is responsible for updating it. So all we really do on this side is report the current state of the system to who or what ever needs it.

When an application is requesting data for an specific screen than this should be done in one single call to the Query layer and in return it will get one single DTO containing all the needed data. Now because of this specific use of the data it makes sense to order and group it in such a way that is determined by the needs of the system. So we de-normalizing the data trying to make a single table reflect a single screen or usage in the client application. The reason for this is that data is normally queried many times more than domain behavior is being executed, so by optimizing this you will enhance the perceived performance of the system.

Here you may choose to use an ORM like NHibernate to facilitate the reading from the database, but considering that you would only be using a very small percentage of the capabilities of a proper ORM you may not need to go that way at all. Maybe going for Linq2Sql of even as Greg suggested using reflection to assemble the SQL statements directly from the DTO’s (using reflection and Convention over Configuration makes this rather simple) is perhaps a better solution for the problem. This will be up to you and probably depends on specific scenario and what you feel comfortable with.

In the example that I created to demonstrate this type of architecture I choose for using reflection of the DTO’s because I wanted to put the emphasis on the CQRS implementation and not on the ORM implementation.

The more traditional reporting needs will also get its own database schema and the data there will be optimized for that need as well, so in the end we will end up with quite a bit of duplication of the data, but that is all right. The process that is responsible for updating the data in the different databases will make sure that this happens in the correct way, we will go over this in a later part of this post.

Commands (executing behavior on the domain)

Lets first consider what would normally happen after receiving a DTO; the user would make changes to the data and save this back on the DTO. This DTO then gets shipped back to the back-end, converted into an entity and the ORM will make sure that the changes are persisted into the database.

This would result in loosing some very valuable information; the Why did this change happen? You completely loose the intent the user had when he changed the data, and this is one of the things Greg’s implementation of CQRS is solving.

CQRS as the name indicates uses Commands, these commands are created on the client application and then send to the Domain layer. Lets take an example: A customer from a bank comes in the office and tells the person behind the desk that he needs to change his address. And instead of just getting the customer information and making the changes directly in the address fields, the bank employee firsts asks the question; Why would you want to change your address? The most likely response would be because the customer has moved, but it could also be because there is an error in his address and that all his mail ends up with his downstairs neighbor. Now these are two completely different reasons to update a customer address. Why could this be important? Well granted the example is rather silly, but lets assume the bank wants to know how many customers go to a competitor after moving? How loyal are our customers, should we keep sending them specific information after they have moved x miles away? Right this information is completely lost in our original way of working, but when using commands and events (more on events later) we maintain the original intent of the action. So after asking the question the customer answers that he has indeed moved and the bank employee selects the “Customer has moved" in the application and gets the ability to change only the address. When clicking save a CustomerMovedCommand will be created with only the changed address and is send to the domain.

We also get one other great benefit from using these commands and that is that these commands are easy to communicate with our client while building or working on the system. Because our clients would most likely use these types of behavior when explaining what they want to accomplish. Al do Greg thinks times have changed, “Our grand failure", but it should really be the case that our clients talk their own domain language. When using these commands we can start talking the same language even in the code.

That is what Domain-Driven Design is all about, instead of doing something technical like update client, it is actually describing the process that the user uses into the code like; the client has moved.

All these commands will be send to the Command Bus which will delegate each command to the command handler or command handlers. This also effectively means that there is only one entry point into the domain and that is via the Bus. The responsibility of these command handlers is to execute the appropriate behavior on the domain. Close to all of these command handlers will have the repository injected to provide the ability to load needed the Aggregate Root on which then the appropriate behavior will be executed. Usually only one Aggregate Root will be needed in a single command handler. Later we will also take a closer look at the repository as it is different from your ordinary DDD repository.

Command Handler

As you can see a command handler has only one responsibility and that is to handle the one particular command by executing the appropriate domain behavior. The command handler should not be doing any domain logic itself. If there is a need for this than that logic should be moved into a service of its own. An example of this is in my example code is an incoming money transfer, more about that later.

Internal Events (capturing intent)

So finally we have arrived at the actual domain, the client has requested a certain view of our domain, has received the appropriate report DTO and has made a decision which resulted into a command being published. The appropriate command handler has then loaded the correct Aggregate Root and executed the appropriate domain behavior. So now what?

Now we are going to separate the domain behavior from the state changes resulting from this domain behavior including the triggering of external behavior. This would not be much different from how you would do this now, first verify the normal guards, do what you have to do, but don’t set any internal state, and don’t trigger any external behavior (Ok the last part is more an optional thing to consider, state is the key here). Instead of writing these state changes directly to the internal variables you create an event and fire it internally. This event as well as the method name of the behavior should be descriptive in the Ubiquitous Language of the domain. Then the event will be handled inside the domain Aggregate Root which will set the internal state to the correct values. Remember that the event handler should not be doing any logic other then setting the state, the logic should be in the domain method.

Domain Behavior

Domain Event

Internal Domain Event Handler

The reason why we want these events is because they now become part of our persistence strategy, meaning that the only information we will be persisting of an Aggregate Root are the generated events. So if every state change is triggered by an event, and an internal event handler has no other logic then setting the correct state (and that means not even deriving other information from the data in the event), then what we can do then is load all historical events and have the Aggregate Root replay them all internally bringing back the original state of the Aggregate Root in exactly the same way it got there in the first place. It really is the same as replaying a tape.

One thing to note is that these events are write only, you will never add, alter or remove an event. So if you suddenly end up with a bug in your system which is generating wrong events, then the only way for you to correct this is to generate a new compensating event correcting the results of the bug. Of course you want to fix the bug as well. This way you have also tracked when the bug was fixed and when the effects of the bug where corrected.

By having this architecture we now basically solved the problem of loosing original intent, because we keep all events that have ever happened and these evens are intent revealing. An other very interesting thing is that now you have an audit log for free, because nothing will ever change state without an event and the events are stored and used in building up the Aggregate Roots they are guaranteed in sync with each other.

The Domain Repository

I mentioned before that the Domain Repository would be completely different from one that is normally the result of practicing DDD. Normally you would end up with very specific repositories allowing the request of all kinds of different information from the domain. But when using Greg’s implementation of CQRS your domain is completely write only, so the repository only has to be able to Get an Aggregate Root by its Id and it must be able to save the generated events. You also completely get rid of any impedance mismatch between the domain and the persistence layer.

Domain Repository Contract

The reporting repositories on the other hand would probably look much more like the traditional repositories from DDD.

So what happens when you have 100.000 events that need to be replayed every-time you load the Aggregate Root, that will slow down your system immensely. So to counter this effect you would use the Memento pattern to take a snapshot from the internal state of the Aggregate root every x number of events. Then the repository will first request the snapshot, load that in the Aggregate Root and then request all the events that have occurred after the snapshot, bringing it back to the original state. This is only an optimization technique you would not delete events that happened before the snapshot, that would pretty much defeat the purpose of this architecture.

Data Mining

One other really interesting fact about storing all the events is that you can replay these events at a later date and retrieve important business information from them. And you get this information from the start of the system, instead having to build-in some extra logging and wait a few months for reliable data.

External Events (publishing, letting others know)

Finally we are getting to the fourth part of this explanation (and I am saying this more for my own sake, pfff). So what happens here, looking back at what we have so far, we can read domain state and we can execute behavior and update the internal state of the domain. The obvious thing that is missing is a way to bring the reporting database in sync with the current state of the domain. The way we will be doing this is by publishing the internal domain events outside the domain. Then there are event handlers that pick up on those events and bring the reporting database in sync. This is a place where you could use an ORM, but in fact it is very easy to just generate the needed SQL statements and execute them.

Greg actually mentioned a really nice way of caching these SQL statements, he would batch them in a single batch and execute the batch if it gets older then x seconds, or (and this is the interesting part) whenever a read request came in. So when a read request comes in this SQL statement is appended to the batch and the whole thing is executed, ensuring that the read request will always have the latest data available to that part of the system. More on this below here.

The domain repository is responsible for publishing the events, this would normally be inside a single transaction together with storing the events in the event store.

Events are also used to communicate between different Aggregate Roots, I my example I am using a transaction from one account to an other account. Here I generate an event that money is being transfered to an other account, this event will reduce the balance of the current account. Later an event handler will make the same change in the reporting database. An other event handler will actually forward the transaction to a service which checks if the target account is a local account, else forwards a money transfer to a different bank (in my example the different bank is actually the same, but via a different route). But lets assume that the money transfer goes to an internal account, in that case the service will publish a money transfer received command in the command bus and the whole process continues in a different Aggregate Root. So in this case the command is not triggered from a GUI but from a different part in the system.

There is an other interesting fact in the scenario when money is received from an other bank, because a money transfer only has the account number to identify the target account and not the Aggregate Root Id (you can’t expect foreign systems to know this, yes I know you can make this a natural Id as well) the money received service first needs to do a query to the reporting database requesting an account DTO where the account number is the same as the target account. When this is successful it will use the Id from the account DTO and put that in the command to be published.

But it doesn’t stop there, as I mentioned before you could also have events that don’t have any state change information but for example indicate that a message (Email, SMS or what ever, depends on the event handler) needs to be send to an user. And because you are using Domain Events for all this, everything will be stored in the Event Store, so you keep your history.

Eventual Consistency

Normally when beginning to implement CQRS you would start with a direct publishing mechanism so that storing the events and updating the reporting database happen in the same thread. When using this approach you have no problems with eventual consistency.

But when you system starts to grow you might get some performance problems and then you could start by implementing a bus disconnecting the publishing of the events and handling of these events. This means that it is possible and likely that your event store and reporting database are not completely in sync with each other, they are eventual consistent. Which means that it is possible that the user sees old data on his screens.

Depending on how critical this really is you can have different counter measures for this, which I will be going into in a different post as this is also not something the example provides.

Specifications

The specifications that you can write using this architecture is something that I really like, what you would do is talk to your client and ask him things about how he wants his process to work. So a possible scenario could be:

Withdrawing money from an account

So how would that go? Well the clients needs to have opened an account with our bank and he needs to have some money transfered on it in order to be able to withdraw money from it. And when this happens the balance of the account must be lowered to the correct amount. Ok so, given an account was opened, and a cash was deposited, when making a cash withdraw then we get a cash withdrawn event.

Ok ok, but now we have the same story, only now there is not enough money on the account, then we should give an exception.

The cool part here is that the whole domain is seen as a black box, you are bringing it to a certain state exactly the same way as it is used, then you publish a command just like your application would, and after that you verify that the domain publishes the correct events and that their values are correct. This means that you are never testing your domain in a state that it cannot naturally get to, which makes the tests more reliable.

Now imagine a parser that takes all the class names, underneath each class name it would print the events that occurred to get into the current state. Then you print the command that we are testing. and finally you print the method names that test the actual outcome. This would be a very nice readable specification that the client at least can understand.

Some other benefits

One last benefit I would like to highlight from using this type of architecture is how easy it is to split the work load between different team, more specifically between team with different hourly rates. The domain logic is something that needs to be right, this is where you would put your more expensive developers on, the ones that understand the business, understand good coding practices, you know what I mean. But the read side is not as important, sure it needs to be correct as well, but this is not where the value lies, this can be done quickly and in a year or two again. This is something you let the cheaper developers create, it doesn’t require much domain knowledge all that is really important is that they need to know how the GUI needs to work, what commands they can use and what events to expect.

I think this is of great value for the business, something that is easily overlooked.

Finally

As you can see this is all very simple and straightforward. It is a different mindset, but once you enter this mindset you will notice that your applications will be much more behavioral versus CRUD. And hopefully our clients can move back into thinking about their business logic instead of the thinking about the CRUD way we have forced upon them. Also I would like to thank Greg Young for providing me with so much information and putting up with all my weird questions, and thank Jonathan Oliver and Mike Nichols for some improvements on the technical side.

CQRS à la Greg Young example code

2009-11-03

I have had the pleasure of spending a 2 day course and many geek beers with Greg Young talking about Domain-Driven Design specifically focussed on Command Query Responsibility Segregation (CQRS).

And because this interests me very much I thought about writing an example, now this example is already well on its way. I am also preparing an actual blog post describing the example in great detail, but I am running a bit out of time so I decided to publish the code even without the supporting blog post. So the DDD Yahoo list has already seen the URL which is: http://github.com/MarkNijhof/Fohjin

Also Jan van Ryswyck has asked me to do a talk about this example on the next E-VAN (18 November 2009), more details can be found here: Europe Virtual ALT.NET So if you have any questions then just submit them to the E-VAN Google group as the blog mentions.

TDD Naming Styles

2009-05-22

The RSS reader you are using doesn't support JavaScript; because of this you will not see the embedded code gists. Consider opening the post in your browser instead.

Just recently I have been discussing a different approach to our naming convention for our test projects at work. A commonly accepted way is to append the project names with “Tests” or “Specifications” suffix. Like the following example shows us three different projects:

But I don’t really like how the namespaces look like when you are appending the Tests or Specifications suffixes to the base namespaces. Take a look at the following namespaces in the three different projects:

I would like to suggest an other approach, instead of appending a suffix I would like to prefix the project names with “Tests” or “Specifications”. Because this will result in the following namespaces:

As you can see the namespaces in the Test and Specification projects are not interrupted with the words Tests or Specifications anymore. I know it is not a huge deal but I like how this reads just a little bit better. Also if you have multiple test or specification projects they are nicely grouped together, something I used to do using solution folders. Well actually still do that :)

TDD class names

Now if we have specific projects for our tests do we still need to append our classes with “Test”? I mean it is obvious that anything in these projects are related to testing. So I would say that that is not needed anymore.

So what do you say?

From liquids to solids in baby steps

2009-03-07

The RSS reader you are using doesn't support JavaScript; because of this you will not see the embedded code gists. Consider opening the post in your browser instead.

This post is about refactoring your legacy code towards better designed code, but first let’s define what I think legacy code is. For me legacy code is code that is not properly testable. I intentionally say testable instead of tested, because I believe that some code although testable does not necessarily need to be tested. Having said that, I am an opponent for having plenty of proper tests; they ensure proper design and enable change. One golden rule applies here; when in doubt write tests for it.

I want to go over a few scenarios on how you can steadily improve legacy code to make the code more testable and improve the design. The biggest problem with legacy applications is that you cannot fix it all in one go; you will have to take baby steps in improving the code. And this can be quite daunting, so where do you start?

The Boy Scout rule

This is one that I got from Uncle Bob’s presentations; it basically means that when you work in some code you should leave it behind in a better state then how you found it. This is a very effective way of gradually improving the quality of you existing code base. Improvements can be anything, from renaming method names and variable names to completely refactoring classes.

Separate your Concerns

When you need to make a change in a class or method and you notice that it has multiple responsibilities; then separate them into different classes and or methods. This will greatly improve the readability and reuse of your code. Yes you will get more classes, but that is not a bad thing.

No more new keywords in the code

When you see a new instance being created in a class you should see warning lights and start to think what the reason is that is happens inside the class. Is it something that can be injected or is it some sort of factory (i.e. like the creation of entities or DTO’s). I have one basic rule; if it contains logic then it should be injected. So either inject the dependency or create a proper factory for it (and inject that into the class). So do I also have to change the calling code, because my code just news this up somewhere as well? Well that depends on the situation, but you can always use constructor overloading. Constructor overloading keeps your existing code working, but enables you to properly test your class without the dependencies. And when you have refactored a whole isolated part of the code you can easily introduce an Inversion of Control container.

From:

To:

Don’t Repeat Yourself

When you see or know that you are implementing something similar for the second time, go and look at the first time you did it and check if you would be able to reuse this functionality. If you can then you should; extract the functionality into a new class and use it in both places.

Write tests

Changes

When changing code you should first write a test or multiple tests that verify the current behavior of the code, make sure they pass. Then decide what you want / need to change and change the tests accordingly. After that make you tests pass again by changing the code. The reason of first writing your verification tests is that you want to make sure that your change is not affecting any other execution paths that shouldn’t be affected by the change.

Bugs

When fixing a bug, first write a test to verify the bug, and then fix the bug and the test should pass. If the bug ever decides to come back you will be notified of it immediately.

Make it testable even if it is not

When you use some existing code that you cannot touch, this could be in the .Net Framework or another third party tool, and you cannot create a mock for it. Then you should wrap this functionality in a wrapper class so that you can mock this wrapper instead, this way you can test the class using the dependency in isolation from the dependency. A mock is basically an object that acts like it is the real thing, but in reality it is a fake. The Timer class in the .Net Framework could be an example of something you cannot mock and would want to create a wrapper for.

Show intend

Your code should be understandable without the need of comments, it should be self explanatory. So that means that it should be readable, use variable names, method names and class names that clearly describe the responsibility they represent. When you are browsing the code and you see something you don’t understand just by reading it, then you should rename it. There is nothing wrong with long names; that is why we have intellisence.

From:

To:

Abstract implementation details

When you see that a concrete class (actual implementation) is directly being used by another class or method you should refactor it to use an interface or abstract base class instead. You don’t need to replace every usage of this concrete class with the interface or abstract class, just the code that you are currently working in. Later when you see another occurrence then you can easily change it there and then. Keep this up and soon the concrete class won’t be referenced directly anymore, which means you can freely replace it by a different implementation.

From:

To:

Use the right tools

Ok refactoring without the use of a tool like ReSharper is going to be painful. Imagine for example when creating an interface for an existing class would only take a few keystrokes. Renaming a method, class or variable name with one shortcut and all usages will be changed as well. I could fill several blog posts about all the refactoring options that these kinds of tools make possible, but I won’t, just click the link.

Don’t make it worse

Any addition that you make to the code should be accordingly the proper principles and practices, so do TDD, SOLID and the other principles, and do them well. There is no excuse for writing crap code! To quote Uncle Bob one more time:

Bad schedules can be redone
Bad requirements can be rewritten
Bad teams can be reorganized
Bad code stays

Keep thinking

Don’t just do something, know why you do something, only then you can decide to perhaps not to do it.

So just do yourself a favor and stop writing bad code, start doing the right thing and you will get to enjoy it later on.

What I mean when I say, This is Bad code

2009-03-03

Well of course I mean that the code is bad. But this has nothing to do with the person who wrote the code. This small but so important detail is not understood by the majority of software developers. Actually it is something people in general find it hard to understand; the first reaction is often a defensive one.

But that way nobody learns, a golden opportunity for the author of the bad code to learn a better way and improve his skills, forever gone, out the window. Hey and also for me, I am often wrong, so explain it to me.

I am not saying you should confront the person in an offensive way, or in front of many others, that will never work. Ask if you can discuss a certain part of the code, because you have some questions / concerns about it. Then you can discuss the code and offer your advice and show what can be done to improve the code. Especially showing what to do is extremely important, because that is how someone can improve.

I for one want people to tell me when and where I can improve, I often ask what others think of certain solutions I created and about the presentations I did. Often the response is positive, now this is not because I am so good (I wish), it has to do with the fact that people expect that that is what you want to hear. I sometimes actually have to say, so where can I improve? I want to hear what I do wrong, that is how I learn.

So when somebody actually comes to you with some concerns about a solution you created, try to see this in a positive light. Embrace it and look at it as a moment to improve yourself. As you have already read there are not many of such moments, so use them well. And whether or not if the person is right you should be thanking him for bringing it up, encourage him to do it again. And return the favor.

Because trust me, when someone takes the time and energy to come to you and talk about his concerns it is because he is generally acting in goodwill. If he wanted to make a fool out of you then he would have done so publicly perhaps even submitted the code here.

I hope that from now on you will be happy when someone tells you that you wrote bad code. Because you know that this is one of those rare opportunities that you can use to improve your skills to become even better at what you do. Next time I do something wrong, please tell me!

Software Development for Dummies

2009-02-14

Why do so many people and even actual developers think Software Development is easy, something that everybody can do? Do we really have to look back and see how many projects have failed? Or how many are a maintenance nightmare? Let me tell you by approximation:”Way too many”.

So can we maybe conclude that Software Development is not that easy, in fact let’s conclude that it is hard, very hard actually! Sure everybody can write some code that does something, even businessman can do this, be it in excel but still. Proper Software Development, software that will stand the test of time, or in other words software that is easy to change and extend, that is not so easy. That requires the developers to actually think about their design, anticipate change.

Anticipating change is hard; change is something humans by nature don’t handle very well. So anticipating to something like that is even harder, perhaps we don’t even want to think about it. Now let’s at least try to make this anticipation a little bit easier shall we?

That’s where principles like the SOLID principles come into play. These help us to anticipate change, enable us to deal with change much easier. They help the developer to properly design the software for the future. Uncle Bob has just written a nice blog post “Get a SOLID start” I recommend everybody to read this and follow his advice.

As a conclusion I would like to go back to the title, and I want to state that Software Development is not for Dummies, it is for developers that consider themselves craftsman’s, developers that take pride in their code.

Keep learning, always!