Thomas Ferris Nicolaisen's blog

Git-SVN mirror product: SubGit

2011-12-26T01:18:00.001+01:00

Just a quick mention for you people who can't be bothered with my somewhat intricate Git-SVN mirror setup, there now is a now is a solution packed into a product: SubGit.

I tried it out on a little local SVN repo recently, and it worked just fine. It delivered pretty much perfect two-way syncing/bi-directional mirror (which I've earlier deemed to be very impractical with git-svn).

The bad parts are:

It's new (use at your own risk, in other words: DO NOT install it in your main-line subversion repo used by a 100 devs).
It's closed source, so you're at the mercy of the SubGit devs for fixing any feature that you need in your special repository.
It requires instrumentation on the Subversion repository installation. If the repository is beyond your control (hosted externally, or by some separate department), you're stuck with my old setup.

The good parts:

Super-sweet functionality! I mean, bi-directional Git/SVN is highly sought after.
Seems to have a pretty high *just-works* factor, easy setup.
It's still free (as long as they're in the EAP phase)

Google Guava's Predicates

2011-12-15T13:37:00.003+01:00

So the other day I was working away on some rather tricky rules for validating an incoming identifier code. Like

999392304II+TJAJEIAJTI901119

or

EPFOOMNNVLLALOWEIITOE

Now, the system where this code is going is rather picky about what these codes should look like. (For the sake of this blog-post, I'll invent some business rules here on the fly, in reality they are a bit different, but the nature of them are the same):

A code cannot be null, or an empty string (of course)
If there are only letters (A-Z) in the code, it must be longer than 10 characters, but fewer than 20
On the other hand, if there are letters mixed with numbers, it must be exactly 10 or 15 characters
Numbers only are not allowed
No other symbols are allowed, but + signs are allowed, if the code starts with 999. In this case, the max length is 10.

Instead of just going straight for the if-else'ing my way through these rules, I implemented it using Predicates from Google Guava. I was pretty happy with the result, and wanted to put it up against what the classic solution would look like.

I've created a little project on Github that demonstrates the above rules. There are two branches:

ifelse - The old imperative way, using if's and else's

predicate - The new functional way, using predicates

Both branches pass the same test suite. I won't swear on the correctness of the implementation, as it's currently 01:59 in the night, and the tests aren't really extensive.

Performance is not a big thing in this validation function. We're dealing with codes that come in every few seconds, so we've got plenty of milliseconds to spend on making the code look as nice as possible. Also, my regexp sucks, I know.

Here's a quick look of what I ended up with using if's and else's:

(Source on Github)

And here's using predicates:
(Source on Github)

In order to really appreciate the differences, it's best to have a look at the whole classes on Github. It's a Maven project, so you can import it into IDEA or Eclipse, and use Git to jump between the two branches for comparison.

I'm not saying that the predicates way is easier to read or write, but there is something functional and declarative about it that feels kind of.. powerful. A nice thing about those predicates is that they can easily be combined with eachother, and re-used in other places, like in filtering collections using other Guava utils.

Note that the full predicate style solution is much larger, but much to blame are those base-predicates I'm defining for saying matching, shorterThan, longerThan and exactLength.

If we were dealing with a language that properly supported functions (a predicate is a function that evalutes to true/false), it would probably look a lot nicer. Perhaps with Java 8, I can slim down the code quite a bit..

I just read that the Guava developers are going to include new fluent interfaces for combining predicates in a more elegant way, than the prefix style I've used above. Maybe something like

new FluentPredicate().not(NULL_OR_EMPTY).and(ONLY_LETTERS).build().

Read more here about FluentPredicates, etc. They can't be too far away, release-wise.

For other Guava resources, check out my collection here.

Some thoughts on Git vs complexity

2011-11-06T23:51:00.002+01:00

I originally wrote this in the Git For Human Beings mailing list. The thoughts are stolen from Rich Hickey's Simple Made Easy talk. (Matthew McCullough commented the same parallel the same day, but I think his timestamp was a few hours afterwards). I wanted to tweet about it, but it ended up being a whole post, as I'm trying to gather my thoughts on it for my next Git talk.

There's simple stuff, and there's easy stuff.

Simple means the opposite of complex. Easy, on the other hand, means it's very close to the stuff you already know. Git is "simple" but hard. Subversion is "easy", but eventually complex.

Git is *a lot* of features in one tool (think of the 100+ git plumbing commands). Each feature is simple, but learning to use them together is good bit of work. As soon as you've understood the model and you get that Eureka-moment, the tool never fails you, and you find it more and more fun to use the more you learn. (This is why there are so many git enthusiasts, I reckon.)

Subversion has a very limited set of features.It also turns awfully complex when you want to do stuff like merging. Actually the more I learned about Subversion, and the more I used it, the more frustrating I found it.

Why is Subversion complex?

- Because the stuff on the server is something different from what you have locally. Your local checkout is just a thin skin. All the interaction goes over the network with a really thin and crappy API. It’s like reading a book using binoculars. The same information is there, but it’s horrible to get at it.

- Everybody is tangled together using the same repository. People start making mistakes like committing without updating first. Everyone mis-use trunk as their own sandbox. This is complexity too (everything in one bucket).

- It mixes together committing, and pushing the changes to central. A commit should do one thing, and one thing well, and the reasoning should be in the commit message. Some are refactorings, others are feature-changes.

- Because branches are completely disconnected - there’s nothing that ties them together, except being a lot like each other. That’s how merging in Subversion works: “I assume that these two directories will be very similar”, and if they aren’t, BOOM.

Git, on the other hand, is simple: It all boils down to being three kinds of objects in a graph (commits, blobs, trees), and the rest of it are algorithms that work this data-structure.

Presenting Git on Windows at the next Bonn-to-Code.net meeting

2011-11-05T21:48:00.002+01:00

Update 18.11.2011: Due to Gad J. Meir visiting Bonn-to-Code.net all the way from Israel, we've postponed our talks that were planned for the 28th of November. Most likely I'll be doing the talk near the end of January next year instead.

It's been a while since I ran Visual Studio on my laptop. In my early university-college days, around ten years ago, .NET was just coming out of the oven. We learned algorithms and data-structures doing Java, but when it came down to actual application development, desktop- and web, connecting GUI's with databases, .NET was the thing.

I liked it. It was bit of an eye-opener for me, that I could actually use programming for building useful stuff, and not just fantasize about being a game developer.

Later on, I fell back into the Java/J2EE world, and rode there for a while. While working for Objectware's Java department, it was always interesting to see what the .NET department was up to, and while they were part of them evil scheme of Microsoft's technology lock-in, they were mighty productive in certain domains.

So, the reason I'm installing Visual Studio again is this: The Bonn-to-Code.Net meeting on the 29th of November. I'll be showing off how to get Git working in the hands of a .NET developer.

My employer, Viaboxx Systems was kind enough to sponsor me with three German Git books I can give away at the event. Hint: Asking questions might result in a free book!

Those who follow this blog know I'm actually a Mac/Linux user, so it's actually been quite a lot of research for me into finding the right Git tools to use on Windows and how to use them. Here are the things I hope to show off (with actual live demos!):

msysgit (of course) and some Git basics
PowerShell integration with posh-git
Visual Studio integration with GitExtensions
TortoiseGit (quick mention)
GitHub interaction
Gitblit (Windows-friendly git repository serving)
git-svn (for those who want to convert)

If you're planning on attending the talk, I'd love to hear any background or expectations you have. If you're undecided on attending, be aware that I just finished Scott Berkun's Confessions of a Public Speaker, so I know all the tricks of the trade now! Additionally, I've spoken a bunch of times before, some of which are recorded if you're curious what my talks are like. I promise an action-packed and dynamic talk with zero slides.

Be a Good Audience

2011-10-23T21:59:00.001+02:00

So I'm sitting here reading Scott Berkun's Confessions of a Public Speaker, pretty good book by the way (from the Viaboxx library). I'm just through the chapter about "Working a hard room".

And then it hits me: I actually spend an effort being a good audience at talks. I go to the odd conference and user-group meeting, and whoever is speaking, and even if the talk is not hitting home with me, I always try to...

Sit down in the middle, close to the speakers where they can see my face
Focus my full attention on them - no smartphone/laptop
React to what they're saying with my face, friendly smile when they say something smart, raise my eyebrowsand smile when they say something surprising
Laugh at their jokes (I mean, not fake laugh, but be open to laugh)
Ask questions when I wonder about something
Hold my criticism till after the talk, if any, and give it in person
Think about and note any questions I'd like to ask in the Q&A

A good speaker/talk summons this behavior in the audience naturally, but I also give an effort in the not-so-good talks.

I do this because when I'm up there speaking myself, I would like to get the same behavior. A more involved audience makes my talk better, and gives more value back to them in the end.

PS: My next talk is presenting "Git for Windows users" at the Bonn .Net user group, Bonn-to-code.net, 29th of November.

Why Releasing More Frequently is Good For You

2011-10-03T23:39:00.001+02:00

So I was thinking a bit about frequent releases. There are many agile books and articles that explain how more frequent releases are a good thing. However, to many people in management, this is counter-intuitive. They say "Slow means safe. Slowing down means more time to improve quality, more time to test, and more time to fix bugs. Also slow is cheaper, because it's less overhead costs." I've seen a lot of projects where release frequency slows down, especially after the initial development burst and launch of a product, and I think this is a shame.

So how do I go about explaining people that the slow-means-safe line of thought is wrong?

I've come up with a little model I'd like to go through here.

I start off with defining a Rate of Development, which we'll assume is constant throughout the model (leaving out factors as motivation and skill).

Now, having a high rate of development is not worth anything if we're not Doing the Right Thing. This symbolizes working in the right direction, and our Performance ultimately is decided by Doing the Right Thing at our Rate of Development. Performance represents the long term success of our organization.

So so far we've got RoD x DtRT => Performance

Now we won't pick at the RoD in this model (assumed to be constant), but rather look at DtRT: Doing the Right Thing encompasses all the hundred little decisions we make every day, from whether or not we should rename this method, which OS we choose on our servers, to which feature we choose to develop. So what tells us what is the right thing to do? Answer: Feedback.

Feedback => DtRT

Feedback comes from customer (support, sales, social media), developers (retrospectives, standups), monitoring metrics and logs on the product, doing user experience testing, market response and stuff like that. This feedback gives us the intelligence we need to Do the Right Thing.

How can we increase Feedback? Answer: With more Frequent Releases. This is fairly intuitive, releasing more frequently will increase the mass of Feedback in most channels.

Frequent Releases => Feedback

At this point traditional management will cross their arms and say hold on, it's not that easy: We can't risk releasing more often, it's too dangerous. So, let us consider Safety as a parameter for that.

Safety => Frequent Releases

Safety means no nasty bugs or deployment botches. The problem with management is that they mix up what is the cause of the effect here. They see Frequent Releases as a driver for Safety going down, while in reality it is on the other side the factors lie.

So let's dig a bit deeper and see what leads to Safety. Here are the typical factors:

Tests (automated tests, unit-, integration-, as well as manual testing where necessary)
Good Code (fewer unexpected side-effects from making changes)
Small Feature Set

The first two there are fairly obvious. The last one is a pill management has a hard time swallowing:

Releasing a Smaller Feature Set means more safety, because there are fewer features to figure out, develop, and to test in parallel. Fewer moving parts that can malfunction, so to speak.

Now the model is complete. Have a look at the complete thing:

(You can also draw a line from Doing the Right Thing leading back up to the factors increasing Safety.)

Typical objections from management that object to this model (exercise for the reader: are these fallacies or not?):

Doing the Right Thing is better decided by planning/strategy/architecture, than by Feedback.
Safety increases linearly with QA: 10 times as many features is just as well tested by 10 times the QA.
Good Code is irrelevant to Safety. (Refactoring is actually regarded as a minus to safety in some places).

While Frequent Releases are the result of Safety and the drivers behind it, traditional management unfortunately sees it as a lever they can turn down to increase safety.

So, I'm not sure if the model will be of any help to you. For me it's just a nice way to explain the benefits of frequent releases to non-developers.

Git-SVN Mirror without the annoying update-ref

2011-09-25T01:07:00.000+02:00

This post is part of a series on Git and Subversion. To see all the related posts, screencasts and other resources, please click here.

So no sooner than I had done my git-svn presentation at JavaZone, I got word of a slightly different Git-SVN mirror setup that makes it a bit easier to work with:

In short, my old recipe includes an annoying git update-ref step to keep the git-svn remote reference up to date with the central bare git repo. This new recipe avoids this, so we can simply use git svn dcommit directly.

So, longer version, with the details. My original recipe is laid out in five steps:

Clone a fresh Git repo from Subversion. This will be our fetching repo.
Set up a bare repo.
Configure pushing from the fetching repo to bare repo
In the shoes of a developer, clone the repo
Set up an SVN remote in the developer's repo

In the new approach, we redefine those last two steps:

(See the original post for how to do the first three.)

4. Clone the bare repo

Note the -o parameter that specifies that we want to call the origin remote "mirror":

tfnico@flint:~/sources/git/>git clone -o mirror ~/git-repos/websites.git/

We can now track the remote branches, and pull the latest changes as they come in to the bare repo.

tfnico@flint:~/sources/git/websites/>git checkout -t mirror/yksi
tfnico@flint:~/sources/git/websites/>git pull --rebase

(Remember to use --rebase to avoid merge commits in case you have local commits.)

When the time comes to push commits back to Subversion, we have to complete the last step.

5. Set up an SVN remote in the developer's repo

We'll do an svn-init with the same parameters as we cloned in the beginning, adding a prefix that matches the name of the remote "mirror":

tfnico@flint:~/sources/git/websites/>git svn init --prefix=mirror/ -s file:///svn-repos/company-repo/websites/

The point of the mirror/ prefix is that whenever we do a pull, the git-svn reference will be updated along with the normal remote reference. So now you can do git svn dcommit without any further ado.

My JavZone git+svn talk is online

2011-09-09T01:19:00.001+02:00

This post is part of a series on Git and Subversion. To see all the related posts, screencasts and other resources, please click here.

Update: I've also shared my notes from the talk. Here they are.

So, the Cisco/JavaZone folks were lightning fast and uploaded the recording of my session at JavaZone the day after I held it. Here it is:

Living with Git and Subversion in Parallel from JavaZone on Vimeo.

You'll have to watch it fullscreen and in HD to follow the command line action, I reckon.

I think this marks the climax, and probably the end of my adventures with git+svn. I mean, we're still using it at work, and I'll still help people with it when they ask on the "Git for human beings" mailing list. However, I won't submit it to any more conferences or user group meetings.

It's been a really interesting topic, and an important one. The resources-page has gotten 6000 views (plus a lot more on the blog posts), and the screencasts on YouTube have received nearly 4000 views. It's been adopted by at least one open source project, two companies where I set it up, and I've seen some incoming links from some big companies' intranets. I still hope more people will find them and make use of the knowledge.

One problem is that if you google "git svn", my resource page is nowhere near the first page. The top results are littered with typically 3-4 year old posts with really trivial git-svn stuff. So if you want to help others find this stuff, you can help out by linking to it like this:

Check out this git svn page!

So, google-bomb away :)

New job

2011-07-24T16:58:00.001+02:00

In the end of June, I left IP Labs to work as a developer at Viaboxx Systems. I had a great time at IP Labs, and learned a lot there, but it was time to try something different.

Viaboxx is a small company, much like a startup (although a bit more established). It's a small team of excellent professionals, and they are agile to the very core, much to my liking.

I'm learning a lot: Groovy, Grails, with options of doing JavaScript and even some native platform dev in the future. But my favorite part is that I get to work on a whole product. From the user experience in the front-end, through software to the hardware components, the whole team is responsible for making it as good as can be.

And just to mention a couple of perks: Waiting for me on my first day was a brand new top-o-the-line 27" iMac. We've got 20% innovation time (FedEx days). We work in a villa. We cook and eat together every day. And the company is sending me up to Oslo for the JavaZone X conference in September.

For more insight into what Viaboxx is about, have a look at our blog, where I'll probably be writing some entries in the near future.

Speaking at JavaZone 2011

2011-07-24T16:14:00.000+02:00

Some weeks back I got word that my Git-SVN talk got accepted for JavaZone!

Last time I attended JavaZone was in 2008, where I did my talk on Web Testing (video available). Since then I've moved to Germany, and haven't really had the chance to attend the conference. However, I've heard its kept getting bigger and better every year, and I have no doubt this is the best Java conference in the world. So I'm really looking forward to coming back there, meeting a lot of old friends, and learning a whole lot of new stuff.

This year marks JavaZone's 10th anniversary, and as usual their marketing is hilarious:

So, see you in Oslo, early morning Wednesday the 7th of September!

2nd day at GearConf

2011-06-13T15:55:00.000+02:00

I was gonna post this Friday evening, but I got knocked out by a cold over the weekend. Anyhow, here's Friday's highlights from GearConf the way I remember it:

Stephan Hochdörfer did a talk on building software using generators and DSLs. The message didn't quite reach me, but the idea of generative programming was interesting enough. Some concrete examples and demos would have helped.

Afterwards, Martin Geisler presented Mercurial's Query Language (PDF link) - which was a really impressive array of features. As is obvious of this blog, I lean towards Git as a favorite SCM, but through this talk I feel I really *got* Mercurial. With bookmarks and queues, it's basically has the same firepower as Git. The only aber aber about Mercurial I find to be the GPL licensing. I reckon this will make it trickier for 3rd parties to implement tool support (example: FogBug's Kiln uses Mercurial).

Onwards to a double talk with Hans Dockter, the Gradle chief. This was another eye-opener. I always thought Gradle was something of a Maven clone in Groovy, but turns out the Gradle philosophy is a bit different: Bend and adapt to the requirements of the build, rather than imposing standards and conventions.

Now, I'm a big Maven fan, and I know the old "You're doing it wrong!" argument used facing people with Maven issues. The way I see it:

Many projects try to do stuff wrong because they don't know any better.
Some projects try to do stuff wrong because they have to.

The problem with the Maven fundamentalists, is that they put everyone in the first category. I suspect Gradle honors all users by putting them in the first. While this is very nice, it gives validation to a lot of sub-optimal builds out there.

At the end of the day, I think most Java projects (components might be a better word) out there are very simple: Java files go in, JAR-file comes out. For these projects, Maven works great. Even for the average webapp, Maven will work fine. However, for our main product, with all its special packaging, shrink-fitting and so on, I might consider Gradle a better tool for the job.

So quickly now the last two talks:

Stefan Glase present CodeNarc, static code analysis for Groovy. I might be going a lot deeper into Groovy soon, so this was a nice tip on a tool to use for keeping my code in check.

Last presentation was Thomas Koch presenting Gerrit. I've played around with Gerrit a lot the last weeks, and it's a great tool - it deserves a lot of presentations. However, in this talk I would have liked to see less screenshots of Gerrit, and more demoing of the software itself. Thomas did have a running Gerrit on his machine where he showed one thing and the other, so why not use through the whole presentation? Cut down on the slides, folks!

In conclusion, I really enjoyed the conference. I think it is probably a bit too small (60 attendees I reckon) for the organizers, but the content is great. Some innovation into the shape of the conference would be nice though: How bout doing a bit less of the 45 minute presentations, and more stuff like lightning talks and panel debates? Also, get active on Twitter and social websites (like SpeakerRate and Lanyrd).

Git-SVN talk at GearConf 2011 done!

2011-06-09T23:15:00.000+02:00

So, just a quick post to announce that I did my Git+SVN talk today at GearConf. I feel that the talk went pretty well. There were only 45 minutes at hand, so I focused on setting up a Git-SVN mirror with branches and tags. I also set up Jenkins to drive the git svn fetch+git push in the fetching repo, and in the end put the bare repo on GitHub (for full buzzword compliance).

If you attended the talk, I would really appreciate to hear any kind of feedback! Either comment here, or rate my talk at SpeakerRate.com.

Otherwise, 1st day of the conference was great! I got an introduction to Chef from Martin Eigenbrodt, heard an eloquent Maven rant from Halil-Cem Gürsoy, and Michael Leibfried talked about how to introduce Scrum in hostile environments. All really interesting talks.

Also speaking in the revision control department was Stefan Lay presenting the awesome upcoming EGit 1.0 release, and Martin Geisler, a fellow Scandinavian, presented Mercurial, where he's one of the core committers. Tomorrow Martin is doing a talk on some more Mercurial stuff, so I'm really looking forward to that, and maybe some more interesting Git vs. Mercurial discussion ;)

PS: For those who liked my Git shell thingie in the screenshot above, the receipe is here, and you can see my PS1 variable (with the git branch name) in my .profile here. And again, if you want to learn more about Git+SVN, my resources are all collected neatly here. All feedback is welcome!

English Talks from the ROOTS 2011 Conference

2011-06-07T12:50:00.000+02:00

Great news! The recordings from the recent ROOTS conference are now available online!

Apart from the keynote speakers (including Jurgen Appelo and Dan North), there were a myriad of more local speakers doing lightning talks. Now, in spite of speaker "quality" on average being a bit below the keynote-level, I find these lightning talks are particularly refreshing. There something very real, and honest about them. Most of them are just regular developers sharing their good and bad experiences from stuff they've tried out.

Unfortunately, it's a right mix of Norwegian and English talks, and the good ROOTS folks haven't done any particular tagging in regards to language.

I therefore had a quick skim through all the talks noting down which ones are English. Here they are:

Keynotes:

Lightning talks (in the order I found them on vimeo):

If I left out any English talks, or if any of the ones above switch to Norwegian mid-talk, leave a comment and I'll update the list.

Added some ads..

2011-05-31T22:01:00.000+02:00

I've just added some ads to this blog as well as my homepage. This is more out of curiosity than it is for making money, but maybe it'll be enough to pay for the domain ;)

I'll keep them for a few weeks, and then take them away if there are any complaints. Please give me a tweet or a comment if you think they have a detrimental effect on the content.

AgileCoachCamp: Germany

2011-05-28T16:49:00.000+02:00

In a few weeks I'm driving down to the AgileCoachCamp: Germany (#accde11). It's taking place in a place called Johannesberg, near Frankfurt am Main, from 17th till 19th of June.

I'm not sure what to expect, but I'm sure there'll be a lot of eye openers on how to help agile adoption.

My personal pain-point is how to avoid agile saturation, meaning where an organization comes to the point where they have "enough agile", or too many non-movable obstacles are in the way of further change.

I'm really looking forward to finally interact in-person with other people on the German agile scene, as well as some from other countries. It's been a while since my last agile gathering/conference (apart from the Bonn XP meetups), so this will be a blast!

Git-SVN at GearConf 2011

2011-05-26T23:00:00.001+02:00

It's that time of the year again! Well, actually last year GearConf took place in October, and this year it's June 9-10th. Well, anyhow.

Again I've gathered up some premium octane Git-SVN knowledge, a bit like the stuff I did at the RuhrJUG in Essen, but this time I have to fit it into a slot smaller than one hour. At least I hope to fit in the interaction with Jenkins and GitHub this time, and do a bit less of the git-svn basics.

So, if you live around the Ruhr area, in NRW, or otherwise nearby, get your GearConf ticket now!

Agile Game Development: Magicka

2011-04-30T00:19:00.001+02:00

Update 2011.05.09: Arrowhead posted a reply to this post.

This post is a tribute to a company which, judging by the looks of it, is kicking ass, agile style.

"This sprint is really going down the drain" from the first scene
of Magicka in adventure mode.

What's this about
The company is called Arrowhead Game Studios, and they've made a game called Magicka.

Now, I'm not going to write so much about the game itself, but it's awesome. I used to play a lot of computer games growing up, all the way through my studies. Since then, it's been pretty sporadic. Until I tried out Magicka. I've been playing it for over 50 hours, which is pretty good value-for-money, considering it cost 10€ (plus DLC's, which are a point I'll come back to later).

How successful is it?
In the first 17 days it was for sale, it sold over 200.000 copies. At times it topped the stats for most-selling game on Steam, at one point selling 30.000 copies in 24 hours. Its gotten pretty good reviews all round, and some rather high ranking game reviewers have completely fallen for the game.

Take into account that this is an indie game which was/is developed by eight Swedish students turned game developers. The only other game I've heard of that comes close to this phenomenon is Minecraft.

Why is it agile?
They ship early and they ship often.

Early because they launched the game while it was still early alpha and crap-full of bugs. Often, cause after they went live on Steam they were patching on a daily basis (the Steam automatically upgrades the game, a bit like the Windows update). Since then they've eased down to more of a weekly release cycle.

Now for those of you who haven't been playing a lot of computer games lately, to me this is pretty radical. A big "Hollywood" game usually takes weeks to produce their first (well needed) patches, some times even months.

Lots of people have been complaining about the instability of the game and the bugs from early on. Then again, so do many big budget games. For a small company of eight developers, it's hard to keep coverage of all possible hardware drivers and configurations, so this is understandable.

But they fix the problems continuously. These releases are pumped out so regularly that I hardly notice them anymore. They've gotten really good at not letting new bugs slip through. This is continuous delivery.

Humble, Open and Honest
The developers seem incredibly involved with their users. As an apology for the early buggy releases of the game, they released a new avatar in: The Mea Culpa Wizard (granted some new magical powers: summon bugs, and the spell Crash To Desktop). By now you can probably guess that there's a lot of geek humor in there.

They communicate frequently with the world through twitter and forums. They often take in suggestions from players into patches and new changes. They post release notes and weekly community updates.

They are honest, continuously maintaining a list of things they know are still buggy. They admit it when they mess up and they apologize. They say thanks when they get feedback.

Even though I've never met them, it feels like the developers are practically my buddies, eager to understand and help me have the best possible gaming experience.

I really had to smile when I saw the Scrum board from the first scene in the game (see screenshot in the top of this post). I mean, how many gamers out there know what Scrum is? That is clearly a wink to other devs out there :)

Pay per use
Its important for the developers to keep the players happy, and keeping them playing, also after that they bought the game. They've shipped some downloadable content, some of it is free, others cost a few euros.

I bought the game for 10€. I then bought the Vietnam extension for 5€, and then I *had* to get the latest level as well for a meager 2€. Later on they are going to release Player-vs-player mode (PvP) for free, due to heavy demand among the fans.

They are tip-toeing the fine line of keeping players paying for more, while not giving the impression that they are exploiting (like some other games have tried).

Courage
Arrowhead have really challenged a lot of the existing conventions in the gaming industry. They heavily under-priced (games are usually 30-50€). They launched the game without a 6 month QA period. They did no traditional marketing (apart from getting on Steam, which was probably a big part of the lift-off). They sit down to play and discuss the game, while streaming it online. It's a spelled-based game where there is no mana-bar, for crying out loud!

Rounding off..
The XP values are Communication, Feedback, Simplicity, Courage and Respect. I think these really ring through every time I have read on the forums. Just to paste some quotes from their change log announcements:

From early on:

Right now we've got so many players posting on the steam forums we're having a hard time keeping up answering posts. But we're reading all of them and are adding stuff to our "to-do"-list. [...] We'll monitor your response closely and keep patching the game as often as possible. Please let us know if the patch helped!

Later on:

We (devs and publisher) are still super committed to fix stuff that's broken and make sure you guys can have a great time with the game. Additionally we've also have a bunch of improvements planned and will be patched in as soon as the major issues are out of the way! These are based directly off your input.

Later:

As we've released new patches we're seeing much fewer reports of the game malfunctioning. More and more players are reporting that the game is working nicely for them. We're also noticing that the problems that exist are A) known and being worked on and B) centered around fewer different things.

And then things are really becoming stable:

So the last patch squashed a lot of bugs and we're glad to report that we're getting fewer error reports and the reports we're getting are about a smaller number of problems.
So we're definitely getting where we want to go! BUT that doesn't mean we're done. We'll keep updating the game and improve it as we go along. We will also try to add additional features that you guys are requesting.

And since then they've done another ten patches. This is agile straight out of the book (or into it, depending on how you look at it). Really inspiring to see how successful they are.

Update 2011.05.09: Arrowhead posted a reply to this post.

Agile Lean Europe: Some thoughts

2011-04-23T20:25:00.000+02:00

Just jotting down some thoughts..

A couple of weeks ago, I drove up to Düsseldorf to attend the Scrumtisch Rhein/Ruhr meeting, because the topic was this new Agile/Lean Europe (ALE) Network. Olaf Lewitz facilitated the discussion ( credits to CodeCentric for the free beverages/food).

I first heard about this network in this discussion in the Norwegian agile forum, initiated by Sergey Dmitriev and followed up on by Johannes Brodwall.

Olaf and Deborah Preuss were both involved in the Agile Coach Camp in Norway earlier this year, and I suspect that they already discussed some of these things there. This makes for a firm connection between the German and Norwegian agile scene, and I hope to be able to support this bridge somehow.

Now, why does this European interconnection matter anyway?
I'll give you three reasons why it matters to me personally:

1) It's no fun doing it aloneAgile is deeply nested with social drive. While it's perfectly fine to drive on the social forces within a closed community (say, the German agile scene), it gets a lot more interesting if we know that this is an international effort. Knowing that other countries are interested adds motivation.

2) A mirror for our culture
We make changes based on feedback and inspection. For the sake of fresh perspectives, we often use externals (consultants, coaches) for gaining insight. Agile depends heavily on, and influences, culture. Ergo, for a fresh perspective, we might benefit from using externals from other cultures/countries.

(One thing is organizations of business, another thing is getting feedback on how our communities are operating.)

3) Synergy, or re-using knowledge and experience in other countries
If I spent hundreds of hours creating some crystallized knowledge on how to deal with some particular problem (like introducing agile in public sector), it would be an awful shame if it only came to the benefit of the few thousand potentially interested in my country. Letting other countries benefit from the same knowledge is win/win.

A few thoughts on Germany/Norway in particular

There are limits to how much we can absorb from American (and even British) experience reports. Germany and Norway are both social-security oriented economies, with conservative innovation habits, well on the way into privatizing a bunch of former national institutions (rail, telecom, post/logistics, health).

So we have a lot of things in common. And if I may say, people are very similar in regards to personality and behavior (the way we eat, drink, talk, politeness, etc.). But Norway is tiny and Germany is huge, which makes Norway a bit like a miniature test-lab for Germany. What works on in Norway could work in Germany on a larger scale.

To name one example: The traditional rejection of agile projects in Norway's public sector projects was overcome some few years ago by (among other things) the creation of an agile contract standard, the PS2000 agile. Since then, several successful projects have made use of this contract, both in public and private sector.

Is this a lesson we could port to Germany somehow?

Also check out Kurt Häusler's thoughts in this post and Olaf Lewitz's summary from the Düsseldorf meeting. Both of them, as well as Johannes Brodwall are attending the ALE Network events at the XP Madrid conference in a couple of weeks.

The Dreaded Service Locator Pattern

2011-04-03T18:49:00.000+02:00

Torbjørn Marø recently blogged about Dependency Injection, due to Mark Seeman visiting the Norwegian .Net User Group in Bergen. This triggered my thinking about the dreaded Service Locator.

I have worked with several teams that favored a home-made ServiceLocator class, a static component referencing a set of services, typically chunks of functionality that are singletons that interact with something external like database, filesystem, or web-service.

My beef with the Service Locator is that you can put it in, and use it from anywhere: It can be used to grab services in a controller/action component, inside a service, in a domain object, inside a for-loop, anywhere. This sounds pretty powerful, but ends up bringing in a lot of maintenance problems.

Now, in spite of my troublesome experiences with it, I keep finding myself being pretty lousy at explaining the disadvantages of a ServiceLocator to my peers.

I therefore hunted through Seeman's blog for some better explanations, and here's what I found:

He totally nails it in the first post:

Service Locator is a well-known pattern, and since it was described by Martin Fowler, it must be good, right?
No, it’s actually an anti-pattern and should be avoided.
Let’s examine why this is so. In short, the problem with Service Locator is that it hides a class’ dependencies, causing run-time errors instead of compile-time errors, as well as making the code more difficult to maintain because it becomes unclear when you would be introducing a breaking change.

Also have a look at the comments for some more discussion and affirmation.

Seeman has also written a book on the subject of DI, and Service Locator is discussed within. I haven't read it, but it sounds pretty good, especially if you work with .Net.

In case you got a little lost in his C# examples, here's my own take on it:

Let's say you want to test a CustomerRepository (an already initialized field in this test class):

@Test customerRepositoryHasCustomers {

assertTrue(customerRepository.hasCustomers());

}

Bang! This explodes in a null-pointer because you haven't injected the proper services that are used inside the hasCustomers method (via ServiceLocator). So you try again:

@Test customerRepositoryHasCustomers {

ServiceLocator.setRemoteCustomerService(new MockCustomerService());

assertTrue(customerRepository.hasCustomers());

}

Bang again! This is because there is another service which is used inside the hasCustomers method a little later.

As you can see, once you know what you need, the ServiceLocator is pretty straight forward to use. And you don't notice this need during runtime, because the ServiceLocator is fully populated during startup.

(This explains why the Service Locator being a perfectly fine pattern for those who don't enjoy writing tests.)

Then there's the maintenance issue: If you change the hasCustomers method to make use of even more services, you won't discover that the tests are broken until you run them again. Also the other way around: If you remove use of services in the method, you aren't reminded to remove this superflous setup from your tests.

In total, Service Locator removes a whole lot of compile-time verification that would be nice to have. Again, this doesn't matter much for those who don't write tests.

But, it does matter for the over all drive towards good code and architecture. Quoting Mark Seeman again (from the end of the third post):

Refactoring from Service Locator to Abstract Factories make it more painful to violate the SRP.

Using Service Locators breaks the window that usually stops you from giving a class too much responsibility. Usually, when you see the number of constructor, or method arguments are towering past a handful, you start thinking "refactor?". But with the Service Locator in use, you don't get this reaction.

Git+SVN at the Java User Group in Essen (ruhrjug)

2011-03-26T14:36:00.001+01:00

Update: There's now a Xing signup for the event: https://www.xing.com/events/living-with-git-and-subversion-parallel-710010

I will be heading up to Essen next month to present Git+SVN at the ruhrjug event. The arrangers (Infaktum) are also behind GearConf, where I presented on the same topic last year.

The event takes place on 14th of April, and starts at 18:00 in the Glaspavillon Campus Essen:

View Larger Map

If you want to know the tricks of living with Git and Subversion in parallel, be there!

I'll be showing off all the steps, from getting started with git-svn, through the obligatory Git-Subversion mirror, to the final-migration-away-from-SVN. Hopefully there'll be time for showing off stuff I didn't get to at GearConf, as well as some good discussion.

Oh, by the way, if you've got any reference cases where you are also using git-svn, I would love to hear about it. I recently noticed that Gnu Cash are using it, and it would make me happy to hear if others have had success with it (or if you had troubles, drop by the Git-for-human-beings mailing list and ask there).

The Dream of a Bi-directional Git-SVN mirror

2011-03-18T20:00:00.000+01:00

This post is part of a series on Git and Subversion. To see all the related posts, screencasts and other resources, please click here.

I just got an email asking me how one can set up a bi-directional Git-SVN mirror. It ended up being quite a long answer, so I'll post it here for the benefit of other Git-SVN readers with the same idea.

As you may know, I'm a proponent of my own Git-SVN setup.

I remember trying to go down the path of a bi-directional repository, but always ran into problems. Here is how it could work:

However nice this would be to have, it can be very hard to achieve in practice:

Git-svn requires working in a non-bare repository, so pushing to it is by default refused. You can work around this by doing this in the target sync repo:

git config receive.denyCurrentBranch ignore

You also have to automatically perform a git reset --hard in the syncing repo after each push (by some git hook?), because the work-dir is considered dirty when it is out of sync with the repository (a consequence of the previous workaround).

And now comes the really tricky part. Consider the following example:

Bob commits change A in his local git repository, and pushes this to the sync repo.
The sync repo does a git svn fetch (automatically to retain linear history).
It then performs a dcommit, but in doing so, Bob's commit is re-written (with SVN meta data), therefore the commit changes into being A´.
The central git repo dcommits the change A´ to the SVN repo.

You see what has happened now? Bob's history has diverged from that of the sync repo:

Bob's history: X <- Y <- Z <- A
Central history: X <- Y <- Z <- A´

So the next time Bob does a pull, Git will see that history has diverged, and automatically create a merge commit so Bob's history looks like this:
   A
Bob's history: X <- Y <- Z <- / \ <- merge commit
   \ /
   A´

You don't want this to happen, because next time Bob pushes, the merge commit will be dcommited, and so an evil circle is created.

You can avoid this by getting Bob to do a git pull --rebase, as Git (at least now when I tried it out) automagically figures out that A and A´ are actually the same commit, and discards the former of them appropriately.

You still will have to deal with some other problems:

* Concurrency between commits - basically if somebody commits to SVN before the git repository has a time to sync a push from a git user, your syncing-repo gets messed up and needs to be manually reset (and probably the git pusher will have to repeat his push). This can get ugly, and it would happen relatively often in a busy repository.

* In case you have set up a physically split bare-repo and fetching-repo (like I've recommended on numerous occasions), the concurrency issues get slightly worsened, as you need to implement some locking mechanism across both repositories. Perhaps the best thing to do here is to go back to one single git/svn syncing-repo instead.

In summary:

Your developers will have to always do pull --rebase before and after pushing.
It has to be a low-traffic subversion repository, preferably all committers sitting co-located.
You have to be ready to dive in and fix any problems quickly (or have several people who can).

So, I hope you have enough information to properly consider whether this is worth going for. If you do, I would love to hear how it works out.

For the record: I do not recommend the bi-directional syncing repo approach. I still prefer my original setup where each developer is responsible for dcommitting back to Subversion. It's not so much more complicated for the Git users, and it is much safer, concurrency-wise.

A small Subversion guide for Git users

2011-02-13T02:54:00.000+01:00

A recurring problem for us Git users is that we tend to forget the good old Subversion tricks. We want to do some patch on some old code in a Subversion repo, and suddenly we've got no idea how to work around.

Here's a quick guide:

> git pull
> svn update

> git add new_file

svn add new_file

> git add changed_file

N/A: SVN automatically adds all modifications to the index. If you don't want to commit it, don't change it.

> git commit; git push (you always have to do these together):

> svn commit

> git revert [SHA]
> svn merge -c -[R] .

> git branch branch_name
> svn copy url_to_project/trunk url_to_project/branches/branch_name

> git tag tag_name
> svn copy url_to_project/trunk url_to_project/tags/tag_name

> git checkout branch_name
> svn switch url_to_project/branches/branch_name/

> git merge branch
svn merge -r[start]:[end] url_to_project/branches/branch_name .
(Note that you have to keep track of at which revision you made the branch, and that the result will always be squashed. Renames and moves will create conflicts).

> git rebase ...

Forget about it :)

Please add other neat SVN tricks to the comments if you can think of any good ones ;)

XP-Meetups

2011-02-12T17:43:00.001+01:00

Kent Beck wrote a small chapter about Communities in his book Extreme Programming Explained: Embrace Change (2nd ed):

Participate in communities, local and global. Look for communities that encourage you to be your best self. If you can't find such a community, start one yourself. If you are wrestling with difficult questions, you are not alone. As a community we can accomplish more than we ever could in isolation.

When I lived in Oslo, I was a member of the local XP-meetup group. It was awesome. In the beginning, I was still a CS student, and I felt privileged I could attend these meetings. These experienced people were dropping serious knowledge about how to work and develop software in practice.

It was so refreshing compared to the artificial subjects at university, and it just made more sense. At the same time, the speakers were so humble, you could ask them anything, and they would try to identify with you, draw parallells based on experience, tell stories..

Over the years, the meetup grew into being the largest XP-meetup in the world, and they attracted big jolt speakers like Tom Gilb, Michael Feathers, Jeff Sutherland, the Poppendiecks, Kent Beck and Uncle Bob. The organizers started doing an annual conference, now gathering 500 participants. Other meetups popped up as well, about Lean, Coding-Dojos, Agile Offshoring and more.

I think these communities have made Oslo a vibrant city for software developers. There are numerous advantages to this:

* Knowledge is shared, which benefits the community as a whole. People become more happier and better at their jobs. Projects become more successful. Old inefficient companies die out, and new companies that provide value to society are the ones that thrive.

* There is synergy between communities. Oslo's Java User Group, javaBin, got more traction thanks to the XP-meetup, and vice versa. JavaZone is arguably the best Java conference in the world, and every year it's sold out. There even is an umbrella organization for the communities in the city, and they're talking about raising a building for hosting communities.

* Networks between software developers are extended. I think this actually sharpens the competition for creating the best place to work for the developers. Communities give a lot of insight on how it is to work for another company, which in turn makes it more tempting to switch ships. The companies who want the best people have to make themselves more attractive, which again means more budget for salary, conferences and courses.

In the end, developers get better treatment, become better at what they do, and enjoy life more. Society gets more value from the software produced. Everybody wins.

----

Bonn, with a population of 320.000, is not as big as Oslo, but it's certainly big enough to maintain its own community of XP-practitioners (aka Extreme Programmers). We'll do our first XP-meetup on the 21st of February, in an a bar 5 minutes from the central station. I hope you'll join too!

Sonatype and Hudson/Jenkins: An Analysis

2011-02-11T21:40:00.001+01:00

I'm dare say I'm a seasoned Hudson user (admin) and proponent. For over five years I've been introducing different kinds of CI at whichever place I worked at. When Hudson entered my radar, this work became a lot easier. I'm also a heavy Maven user, and Nexus did for Maven repositories what Hudson did for CI.

The Plot
Over recent weeks, the community around Hudson has been shaken by a fork: The main developers (Koshuke and crew) have renamed Hudson to Jenkins, while Oracle have kept on developing Hudson in partnership with Sonatype. Here's a simple illustration of what happened:

These are my pure external speculations about what has happened recently in the forking process, with regards to Sonatype's involvement in particular.

So who is running this show?
I noticed that Sonatype (usually personified by Jason) have gotten their fingers pretty deeply into the Hudson infrastructure already:

Like this tweet noted, it appears they are controlling Hudson's Twitter account. The @hudsonci account tweeted about something Maven-related, and it appeared it it was indeed meant for a Sonatype account.

And here it appears Jason is administering some of the Hudson infrastructure.

It's no secret that Sonatype and Oracle are tightly partnered on this. But I think the relationship is more like one of a smart trickster sitting on the shoulders of a dumb giant.

Backstabbers and Benson
A lot of people felt betrayed when Sonatype chose to side with Hudson/Oracle in the fork. Heck, even I was disgruntled that my Maven heroes were going with the big evil anti-open source Oracle..

I've been pondering on why they did this unpopular move, and here's my rough guess:

The reason is Benson:

Benson is Sonatype's Maven-focused distribution of Hudson taking into account everything that we've learned about Maven 3.0 during its development and how Maven should operate optimally within a CI environment: there will be no better way to work with Maven and Hudson. Benson is Hudson's brother from another mother. (source)

As with any fork, maintaining it is expensive. I think Sonatype found it hard to keep Benson up to date with Hudson and all its plugins. Or maybe they wanted more changes in direction Nexus/Maven than the Koshuke crew were willing to go with.

See more about Benson in Jason's presentation from JavaZone 2010 (time 39:20).

A Genius Take-over

Had Hudson stayed under Koshuke's rapid development, Sonatype would've had to keep on maintaining Benson, merging in conflicting Hudson changes as they grew ever more distant from the fork.

Now, they can achieve the same with Hudson itself, and in the process they gain a huge brand by sticking to the Hudson name, and they also stay under Oracle's wing (this has its own con's, of course), with infrastructure and paid coding contributors.

So in a way, Hudson is the new Benson.

You can actually see this manifestation taking place right now. I guess you could simplify this to mean that Sonatype are now merging back the development they made in Benson. What the community is wondering, is why they didn't offer to do so before the fork (see the discussion in the comments), but that doesn't really matter anymore. Sonatype came in at exactly the right time, and did what was best for their company and customers. Nothing wrong with that.

My two biggest questions of this are:

Will Oracle put up with Sonatype's strategy over time?
Will Sonatype put up with Oracle's inherent bureaucratic inertia?

Over time there's nothing stopping Sonatype from forking out Benson again, if it turns out Oracle's wheels are winding too slow. Another possibility is that Oracle takes a liking to Sonatype's infrastructure tools and proposes a buy-up.

This is a Good Thing

I think Sonatype is one of those rare companies that are able to work the open-source ecosystem, making a profit while at the same time donating a huge amount of great tooling back to the community. Some times they have to make unpopular compromises in order to stay profitable, and this is what happened with Hudson/Jenkins.

Nexus is an awesome product. It shares many of Hudson's qualities: It *just works*, it looks nice, it's easy to upgrade and maintain. I hope Hudson will retain these qualities. I'm sure Jenkins will.

What path will you take?

Personally, I think I'll go with Jenkins for now. I'm tempted to go with Sonatype/Hudson because we extensively use Nexus and Maven, and we're not paying Sonatype-customers. At the same time, I have only moderate needs for stability, and from experience I'm very happy with the features vs stability rating of the Koshuke crew.

I think (but might be wrong), that Jenkins will outrace Hudson in terms of features and usability.

I fear, that Hudson will get tangled down with Oracle stuff (register here to download, etc).

I hope both will continue to exist and fulfill two different needs in the market. Best of luck to both of them!

Google Guava r08 is released

2011-02-07T00:09:00.000+01:00

A few days ago, Google Guava r08 was released.

"This project contains several of Google's core libraries that we rely on in our Java-based projects: collections, caching, primitives support, concurrency libraries, common annotations, basic string processing, I/O, etc."

I had a little look through the API changes to find some neat new details, and put together a couple of JUnit tests that show them by example here.

That whole GitHub project is a Guava showcase, so if you can think of some more examples that should be in there, please fork! Especially the IO, Net, and Concurrency bits could need some examples.

I've also taken my old blogposts about Google Guava and ported them to tfnico.com/presentations/guava. After noting this, the Guava guys honored the effort by linking the page from the Guava homepage, which gave me quite the traffic bump! Luckily, my pages are all hosted on Google servers, so it's their own bandwidth bill ;)