The Build Doctor

Unclean. (does your CI server have an IDE installed on it?)

2008-05-14T16:08:00.006+01:00

(image taken from SubFlux's photostream)

Your build shouldn't depend on an IDE. I've been saying that for a long time. It doesn't matter that all the developers use the same IDE on your project. At least in the Java world that I have inhabited for most of the past 8 years, you absolutely should not need an IDE installed on your build server.

Yesterday I installed an IDE on the build server.

Casey Charlton and I both agree that in an ideal world, there's no connection between the build and the IDE. I've been trying to find a reference to the "thou shalt not install an IDE on the damn build server" rule. I've found a pretty authoritative quote from Paul Duvall's book:

You should avoid coupling your build scripts with an IDE. An IDE may be dependent on a build script, but a build script shouldn't be dependent on your IDE. .... Creating a seperate build script is important for two reasons:

1. Each developer may be using a different IDE, and it can be difficult to to account for configuration differences in each IDE.

2. A CI server must execute an automated build without human intervention. Therefore, the same automated build script used by developers can and should be used by the CI server...

As usual, Paul is bang on. But my issue at the moment is Visual Studio. Not that I can't write code in it (I haven't really tried), but the fact that it's actually more than an IDE in the traditional sense. It's also the container for the testing framework. It's almost the entire stack. It comes with Crystal Reports (which I'm happy to say I didn't install), and other stacks of middleware. Which you need to build your app. Ever tried to build a VSTO app? You need Visual Studio.

In theory, you don't need it, because the build tool for Microsoft Products (MSBuild) comes with the .NET framework. But it seems like in practice, you do.

So I installed it. And the build works, without me fiddling with the GAC. I can live with that. What works for you?

Links for 2008-05-13 [del.icio.us]

2008-05-14T00:00:00-05:00

Ron Jacobs : Living through "The Change"
Dealing with change. Planning for it, etc.
Channel9 Wiki: EquivalentTasks
nant vs. msbuild:
MSBuild Community Tasks
The MSBuild tasks that Microsoft didn't make.

Links for 2008-05-12 [del.icio.us]

2008-05-13T00:00:00-05:00

ant [thecrumb wiki]
Jim Priest's collection of Ant resources, including his own blog posts.

Ant Best Practices: Define Proper Target Dependencies

2008-05-12T22:59:00.003+01:00

(Image taken from Nick Sieger's Photostream)

Wondering what this post is about? Have a look here.

Last time I wrote, it was about reusing paths. Tonight, it's about the dependency graph. My [N]Antcall is evil post goes into some detail about dependency graphs. Let's just agree here that Ant targets tend to accumulate dependencies. The point that Eric makes is that the dependency graph (otherwise known as the dependency tree) grows over time as you add targets. Being a thing that accumulates over time, the graph can get crufty. That target that was just a short term thing while Bob worked on a refactoring exercise (oh, the irony) is now central to your build. So from time to time you need to clean the dependencies out, like that burned cheese that melts off of pizzas and sticks to the bottom of the oven.

Some projects seem to run into trouble and throw dependencies out completely. Umm, good luck with that. Eric suggests a half-way house approach: a few well known targets that contain dependencies on lesser known but functional targets for the more experienced to use at their own risk. I can go along with that.

My last thought is this: your default build should never depend on a clean target. Leave it to the compiler to have a go at working out what it should do. If you use Continuous Integration, you can make a little target that depends on clean just for that case. Your fellow developers will thank you.

Links for 2008-05-10 [del.icio.us]

2008-05-11T00:00:00-05:00

I've come up with a quick test you can do to see if you should automate your deployment process.
Cautionary tales

Links for 2008-05-09 [del.icio.us]

2008-05-10T00:00:00-05:00

Phusion Passenger (a.k.a. mod_rails) - apache module for rails
When did this arrive? Being able to roll out apaches that just know how to talk to rails could be the thing. This could be huge.
Test Early » Running MSTest against .NET 2.0 on CruiseControl.NET
This does point to the fact that you either need to install the whole MS toolchain on your build server, or spend a day cheerfully sticking dependencies in the GAC or on Program Files dir. Eeek.
Get a Build Server and Keep it Clean - Jeremy D. Miller -- The Shade Tree Developer
sadly this may no longer be feasible.
Automation for the people: Choosing a Continuous Integration server
Paul Duvall article on choosing a CI server. How far things have come in 2 years....

Links for 2008-05-08 [del.icio.us]

2008-05-09T00:00:00-05:00

Open Source Tools for IT Automation
Anyone ever tried this?

Links for 2008-05-07 [del.icio.us]

2008-05-08T00:00:00-05:00

Thoughts in the Wilderness: How building a bridge is the same as building software
I was about to scream: "the civil engineering metaphor is not appropriate for software engineering"!. But that would just make me sound shrill, and the man does have a point about the deployment being where the rubber hits the road.
Git - coming to a Windows computer near you?
The skinny is that there's a Google Summer of Code project to implement (i read that this meant, not port) Git to C# to support windows and mono. sweet.

Git - coming to a Windows computer near you?

2008-05-07T09:27:00.003+01:00

Mono founder Miguel de Icaza just twittered about a Google Summer of Code project called Git# - implemented in C#, with no platform dependencies. Git is a powerful Distributed Version Control system that came from Linus Torvalds. While you can convince it to run on Windows, it has dependencies on the Unix toolchain. This project could change all that.

Links for 2008-05-06 [del.icio.us]

2008-05-07T00:00:00-05:00

Gojko Adzic » DbFit: Test-driven database development
Got a testing framework that's all row and column oriented? Got sprocs that are hard to test? You need this.
TSQLUnit testing framework
How to unit tests sprocs in SQL Server.

Ant Best Practices: Define and Reuse Paths

2008-05-05T22:15:00.006+01:00

(image taken from justpic's photostream)

Last time, we discussed dependencies. Today's installment of Ant Best Practices is purportedly about resusing filesets by reference ID, but it's really another way to avoid duplication. The title of Eric's original article is 'Define and Reuse Paths'. The same advice works for paths, filesets, and filtersets. Here's an example:

<project name="OnceAndOnlyOnce">
 
 <fileset id="output_files" dir="${build.dir}">
  <include name="duplicato-*.jar" />
 </fileset>
 
 <target name="copy_stuff">
  <copy todir="${special.shiny.dir}">
   <fileset refid="output_files" />
  </copy>
 </target>
 
</project>

You can see that any other use of the fileset will be able to refer to it by ID, rather than explicitly declare it again. One of the scariest builds that I ever saw was 4,000 lines of XML in a single file. One of my erstwhile colleagues spent a morning going through it and replacing filterset declarations with references to one filterset. It's such a useful feature to have in a build tool, and a fine piece of advice to give to someone who just started out writing build scripts.

The other point from the original article is that you need to use this technique alongside some modular approach to handling paths. You just can't really get away with having a path named 'standard.jars.classpath.without.selenium' (and yes, that's almost directly lifted from the build of a large Java project). You're much better off trying to split things like classpaths into categories like build-time and runtime. That's something I touched on in my article for the ThoughtWorks Anthology. I'm thinking of expanding on it in a future post. You like that idea? Let me know.

How to find out what files aren't added in your Perforce client

2008-04-28T22:45:00.006+01:00

(image taken from jparise's photstream)

I've first used Perforce in 2002. At the time we were using another product. Perforce was about 10 times as fast and 10 times less likely to make me want to storm into the comms room, rip the source code server out the rack and drop it off the fire escape into the street. It was also atomic.

A lot has changed in the intervening years. Subversion has come of age, distributed version control has become the next new thing, and I don't have access to the comms room any more. Oh, and I'm using Perforce again. One thing I miss is Subversion's ability to tell you the status of all files in the sandbox (which is roughly equivalent to a P4 client), including unversioned ones:

graham-tackleys-mac-mini:~/Documents/workspace/playpen jsimpson$ svn st
? a.html
? pack.sh

In this instance, you can tell that 'a.html' and 'pack.sh' aren't versioned in Subversion because they have a status of '?'.

What do you do for the Perforce command line client? If you're a Windows user, you can do this:

dir /s/b/a-d | p4 -x- have > nul: 2>missing.txt

The part to the left of the pipe (the '|' symbol) gives you a list of all the file in your current working directory. The part to the right of the pipe queries perforce to see if the files are in the repository. Any files that are in the repository get binned, whereas any files that aren't get written to the missing.txt file with the dreaded Perforce error message: file(s) not on client.

Unix users can try this one:

find . -type f | p4 -x- have > /dev/null 2>missing.txt

This valuable snippet comes from Laura Wingerd's definitive work on Perforce. Is there another Perforce one-liner that you can't live without? Let us know!

How to get repeat prescriptions from the Build Doctor

2008-04-28T21:59:00.003+01:00

(image taken from Tevor's Photostream)

Thanks for all your comments so far. It's nice to know that people are reading and (in just a few cases) correcting me. I do appreciate feedback - of all kinds. Here's an outline of the ways you could track what's going on here, and get in touch.

How to keep up to date:
RSS Feed. If you already use RSS feeds, then perhaps the most sensible thing is to subscribe to the feed for the blog. We publish a full feed which gets you ever article published, as well as daily build and deploy links from del.icio.us.

Email Subscriptions. Every page on Build Doctor has a link for subscribing by email. I take privacy seriously, and will not disclose your email to anybody. If your internet access is restricted, this could be an option.

Twitter. Twitter is a great way to keep in touch with people, that seems to be quickly gaining traction. I'm going to announce new posts on Twitter.

Talk to the doctor:
You can email the surgery at 'medic (at) build (dash) doctor (dot) com'

Every Build Doctor blog post has a comment section. I approve comments before they are published, so feel free to drop me a comment. Tell me if you'd like to be emailed back, and I can delete the comment before it gets published.

Twitter is also a good way to get in touch.

Tell your friends:
Every article also has links to social bookmarking sites at the bottom. If yours isn't there, tell me and I'll try and squeeze it in.

Thanks to the hipsters at the Disco Blog for the title gag.

Ant Best Practices: Manage Dependencies Using Ant

2008-04-24T21:58:00.004+01:00

(image taken from cotes' photostream)

In the last post, we discussed the importance of a clean target. Is it your first time? Have a look at the index page.

Today, I want to talk about managing dependencies with Ant. What does that mean? In this case, it means managing the dependencies on code within your project: dependencies in code, third party libraries, etc. I want to cover dependencies between other projects in another post.

The advice is to start from the very bottom up. You have a bunch of common utility files? Compile them up in one target. And make sure that target outputs one thing and one thing only: a jar file called common.jar. Now, that file isn't going to be your enterprise application on it's own, so do the same to the code that references the classes inside common.jar. Make sure that each one of those outputs a single artifact, and that each one of them depends on the target for common.jar. Now, you probably have a web application or two that depends on a jar file of business code, which depends on a bunch of common files that nobody can decide where to put - common.jar.

What if there's a separate project for the front-end project, vs. the back-end? You still want to package up the code as jar files for consumption by the other project. But for the love of God try to resist doing that. Most projects really only have one product that they make. To split them up into different, smaller projects adds a whole world of complexity.

What we just did was stop anybody from breaking the build by making the back-end code depend on the front-end code. Which is quite easy to do when your IDE allows you to do that in a few keystrokes.

This technique will create some good karma for your project, as long as you follow the sane practice of making every developer run the Ant build before they check in. For me, this is critical. Nice one, Eric.

Cruise: the newest CI server from ThoughtWorks

2008-04-20T21:46:00.005+01:00

ThoughtWorks Studios has announced it's latest product: a Continuous Integration server called Cruise. It's derived from the open source project CruiseControl with many new features promised. Jez Humble announced it to the CruiseControl developers on April 15. Presumably this will replace CruiseControl Enterprise, which was announced about a year ago.

Call me shallow, but I do like the name. CruiseControl always gets abbreviated to 'Cruise': why not give it the name that sticks?

The list of features looks impressive. I'm particularly interested in having a look at the Build Pipeline support. The idea of the pipeline is nice, but I'm still waiting to see a good implementation; and I'd like to see how it works in practice - I've been less than impressed in the past.

The support for Java, .NET and Ruby out of the box is a good thing. The 3 different CruiseControls that we all ended up with each have some great features. With each project being run separately, innovations tend to stick to the CruiseControl fork that they originated in.

You can read up on the rest of the feature list for yourself - click the link at the top of this article.

What will be interesting is how it's going to fit into the CI marketplace. The first movers in the commercial CI tools space like have been carving out market share. Microsoft offers CI tools. There's a load more of open source contenders now than there was a couple of years ago. Having said all that, ThoughtWorks are the people who put CI on the map. I think they will generate a lot of interest.

It's a growing market. Let's see what happens.

Link

How to add Ant and NAnt support to TextMate

2008-04-08T22:29:00.006+01:00

(image taken from Dmitry Baranovskiy's photostream)

The moment my 30 day evaluation of TextMate expired, I ordered a copy. The last time I got so enthused about an application that I stumped up the cash for it was about 3 years ago, with Delicious Library. Today I used it to edit some NAnt files. Here's how.

My ancient little Mac Mini already had Subversion. I installed Mono so I could get a working copy of NAnt. You'll need both.

You need to run this in a Terminal window to enable syntax highlighting for both tools:

export LC_CTYPE=en_US.UTF-8
mkdir -p /Library/Application\ Support/TextMate/Bundles
cd /Library/Application\ Support/TextMate/Bundles
svn co http://macromates.com/svn/Bundles/trunk/Review/Bundles/C%23.tmbundle
svn co http://macromates.com/svn/Bundles/trunk/Review/Bundles/Ant.tmbundle/
osascript -e 'tell app "TextMate" to reload bundles'

The TextMate bundles repository has some non-ASCII encodings, so the first line just ensures that you have a UTF-8 encoding selected. The middle 4 lines make a directory, and put the Ant and NAnt bundles in that directory. The last line reloads the bundles for you if TextMate is already running. What a pleasant way to do .NET.

Ant Best Practices: Provide a clean target

2008-04-06T21:17:00.004+01:00

(image taken from Bob Jagendorf's photostream)

In the last post, we discussed the importance of good help. Today, the best practice we're going to review (just joined us? have a look here) is provide a clean target. It's pretty straightforward. As you change code (especially renaming source files), you'll need to delete old files. You might as well automate the process.

I have seen builds that don't use a clean target. But then generally these builds are relying on some other process, like nuking the entire checkout and starting again. It's easier to have a clean target, honest.

You'd think that would be the end of it. Make a target called 'clean'. Make it clobber some compiled code and generated artifacts. But while you go creating your new target, think about a couple of things:

- How many different files and directories do you need to delete? Ideally, it should be one directory. Life becomes very simple when you have a single tree to clean. Especially as you often end up making the opposite of a clean target to add directories back in.

- Did I say delete? If you have a single directory that the rest of the build depends on, check in into source control. You can often configure your VCS to ignore the contents (like cvsignore or svn:ignore) of the build directory. Then you can guarantee that it's always there, but not have to worry about the built artifacts showing up when you go to commit.

Here's one that I prepared earlier:

<project>
 <target name="clean">

  <delete>
   <fileset dir="${build.dir}">
    <include name="**/*" />

   </fileset>
  </delete>
 </target>
</project>

Eric. You're on the money again.

CITCON Europe 2008, Amsterdam - registration is open

2008-04-02T22:13:00.007+01:00

(Photo taken from Copabella's photostream)

Citcon is one of my favourite conferences. A free, open space conference about Continuous Integration and Testing. I like it because you get to meet other Build Managers and see where things are headed. Registration opened today. Numbers are limited.

Amsterdam, Holland October 3 and 4. Link

The Ant Best Practices Series

2008-04-02T21:32:00.010+01:00

(Photo taken from 33mhz's photostream)

In 2003, Eric M. Burke published an article at OnJava called Top 15 Ant Best Practices. Are the practices still relevant? What has changed in the last 4 years? In this series I'll be reviewing each one and trying to find out.

1 of 15: Adopt consistent style
2 of 15: Put the build file at the root of your project
3 of 15: Prefer a single buildfile
4 of 15: Provide good help
5 of 15: Provide a clean target
6 of 15: Manage dependencies with Ant
7 of 15: Define and reuse paths
8 of 15: Define Proper Target Dependencies

This page is updated every time there's a new article.

The CruiseControl Best Practices Series

2008-04-01T22:38:00.006+01:00

(Image taken from igb's photostream on flickr)

Here's the index of all the CruiseControl Best Practice blogs. I'll be updating this as I publish more.

Part 1: Publish with a publisher [Chinese translation]
Part 2: Keep your dependencies to yourself [Chinese translation]
Part 3:Configuration the CruiseControl way [Chinese translation ]
Part 4: Bootstrap with a bootstrapper [Chinese translation ] [Config Validator]
Part 5: Refactor your configuration file [Chinese translation ]
Part 6: Scaling up [Chinese translation]

CruiseControl Enterprise Best Practices, Part 6: Scaling up

2008-03-28T23:20:00.005Z

You just started out with Continuous Integration. You're building your project in CruiseControl. Great. Now, it's time you started to plan ahead. This post is about scaling up CruiseControl. The tool can scale up to many projects, but you have to know a few things.

The very first thing to do is make sure that you can simultaneously build your projects. CruiseControl uses Java threads to manage builds and projects. Each project has it's own thread. There is a least one Builder thread that will actually run a build, and there is a Build Queue, which mediates the requests from the project threads to the builder thread. You need to give CruiseControl enough builder threads to work with: by default it gives you one thread. The impact of this is that if you have 5 projects configured, you'll only ever build one at a time. This could be good if you have a tiny build server, or projects that don't play nicely together.

On the other hand, if you have plenty of capacity to use, you will get a boost from the extra builder threads. The right setting for the number of threads will change over time; don't hesitate to do some performance management and find the right value for your server.

<cruisecontrol>

 <system>
  <configuration>
   <threads count="2" />
  </configuration>

 </system>
<!-- rest of config file suppressed -->
</cruisecontrol>

Jeffrey Frederick wisely points out that it is useless to set the number of builder threads to be higher than the number of projects that you have.

There are plenty of other factors to consider when scaling up your Continuous Integration service: the speed of your Version Control System, where you store artifacts, logs, etc. One factor that I will mention today is disk bandwidth. Building software is a disk-intensive process. Even when a build is done compiling Java code and making archives like jars, CruiseControl has to write logfiles. It's very easy to overload a single disk with all this activity. Ideally you want to make sure that your machine can build several projects at once.

On one of my very first projects at ThoughtWorks, the team was running two CruiseControl servers, apparently because the first Solaris server was too slow. That didn't seem right to me, so I dug deeper. With my systems administration background I was able to see that a single disk was 100% busy running the operating system, CruiseControl and the build. I spread the workload across the four disks in the system and the machine was able to manage many more projects.

This pattern has repeated itself on many of my subsequent projects. In my experience not many CI servers are constrained by processor overhead. Unfortunately, it's often painful to rectify disk issues once a system is up and running. Make yourself some luck and order plenty of fast disk drives before you scale up and people start to complain about the slow build.

The way to implement this is to use the configuration file. The log element is used to tell CruiseControl where to write logfiles. By default it's a suubdirectory of the CruiseControl installation. But if you change the value of the dir attribute of the logfile, you can make sure that the logs are being written to a disk that isn't already running CruiseControl. If you're using Ant to build your code, you can use the antWorkingDir attribute on the ant element in the config file to make sure that your projects are built on another disk.

I can't really do a good example for this one as each CruiseControl instance is so different. Buildix is an attempt to make many installs more homogenous. If you look at the way things are laid out on the disk, you'll see that the CruiseControl install is in /usr/share/cruisecontrol, but the projects and logs are installed in /var/spool/cruisecontrol: the reason we did this was so that you could mount the /var directory on another disk if things got busy. Drop me a comment if you want to know more.

New, shiny Rake support in Team City

2008-03-27T10:13:00.002Z

You couldn't pay me to run CruiseControl.rb now. Oh wait. You probably could.

Anyway, now that Team City has a 'Rake Runner', you can build your Ruby projects with Rake. It understands Test::Unit and Rspec test output as well. The runner gives you test timings, both for that build and a graph of timings in recent builds. It also gives you Rcov support as well. You can have a look on their public demo server.

It's available as an EAP.

Link

New colourized Logger for Nant

2008-03-26T12:22:00.005Z

Update: It was of course Eric Liu who wrote the logger. Stephen Chu wrote this adjacent post about Rails. Sorry, Eric and Stephen. Thanks to the people who left comments pointing this out.

I spent several years being sysadmin at one project. Possibly the most popular thing I ever did there was enable the Ant AnsiColorLogger for all the developers, giving them highlighting in the IDE and the build.

Anyway, Stephen Chu from ThoughtWorks just wrote the same for Nant. Nice work.

Link

Ant Best Practices: Provide good help

2008-03-23T20:25:00.004Z

This is the fourth post in my series of posts examining Eric M Burke's
Top 15 Ant Best Practices from December 2003. The last one I covered was 'Prefer a single buildfile'. Today's is 'Provide good help'. Eric starts by suggesting that you should make use of public targets to document the targets that you should be able to invoke.

Now, the distinction between a public and a private target in [N]Ant is pretty subtle: If the target has a descrption, it's public and will show up when you call Ant with the -p or the -projecthelp option. If a target has no description, it's private. So, the description attribute signifies that the target is for public consumption; and also has the side effect of telling you what you can do with it. Using this mechanism is a no-brainer - but how do you protect a private, or internal target? There's no way that you can declare a target as private, but you can stop the target from being called from the command line by prefixing it with a hyphen.

For example, if you have a target called start-weblogic that you don't want called from the command line, you can call it -start-weblogic. If you try and invoke it, look what happens:

graham-tackleys-mac-mini:~ jsimpson$ ant -start-weblogic
Unknown argument: -start-weblogic
ant [options] [target [target2 [target3] ...]]
Options:
-help, -h print this message
-projecthelp, -p print project help information
-version print the version information and exit

Eric also suggests 2 other things: using XML comments to provide more detail, and making a help target that echoes out some useful help messages. I don't think I'd ever disagree with a help target. I might disagree about the XML comments. Generally I use XML comments as temporary notes to myself. It's nicer to use the description attribute on elements in build.xml files: just about every element will accept one, and you can make sure that the text always matches the element.

<project name="MyGreatProject" default="test">
<property environment="env" />
<property file="override.properties" description="this gets all the special properties"/>

So far, that's about 4 out of 4 practices that have held up. Next: clean targets.

My article on Refactoring Ant Builds is published

2008-03-23T16:39:00.003Z

The Pragmatic Programmers have just published the first volume of the ThoughtWorks Anthology! My article on Refactoring in Ant is included: I distilled my experiences fixing broken Ant builds for ThoughtWorks into a collection of refactorings that can be applied to a crufty build. You can buy PDF or paper copies via the Prags, but if you want to buy it via Amazon, there's a link on Build Doctor.

I'll write more about the process of writing an article on my personal blog.

Link (via Josh Graham)

renaming multiple files in Perforce with the help of Ruby

2008-03-19T13:21:00.002Z

Half an hour ago, I had about 60 files that had the same irritating prefix to the filename. I wanted to drop the prefix so the reader could see the significant portion of the filename first. There is no way that I would rename that many files by hand and not mess one up.

bad_path=/my_redundant_path/

def rename(file,path)
new_file = file.sub(path,'')
system("p4 integrate #{file} #{new_file}")
system("p4 delete #{file}")
end

Dir["*.build"].each do |old_file|
rename(old_file,bad_path) if old_file.match(bad_path)
end

Two minutes spent playing with ruby goes a long way.

Ant Best Practices: Prefer a single buildfile

2008-03-19T12:26:00.003Z

This is the third post in my series about Eric M Burke's Top 15 Ant Best Practices from December 2003. Today's practice is "Prefer a single buildfile". To paraphrase Eric, it's easier to understand a single file rather than a clever hierarchy of files. Having presided over several multi-file builds, I'm only too aware that you can easily hang yourself. On the other hand, sometimes it makes sense to extract something to a different file to achieve a separation of concerns.

If you are trying to map complex dependencies across many build files, or do amazing things in Ant or Nant, then perhaps you need to try and simplify things in your project. I once worked on a project where the frontend was written by my team, and the backend by other. The frontend team branched the work of the backend team, and the backend team kept on trucking with their work. I came to the belated conclusion that we had fallen prey to Conway's Law; really we had one project, and we should have written it like that. The build on that project was a nightmare: two projects pretending they were free agents, but in reality hopelessly dependent on each other.

I'm torn on this practice, but I'm going to agree with Eric. Ideally you shouldn't need multiple files. Though it does keep me in a job.

Ant Best Practices: Adopt consistent style

2008-03-17T21:36:00.004Z

In my first post reviewing Eric Burke's Ant Best Practices article from 2003, we looked at putting the build file at the root of the project tree. I had a bee in my bonnet about that one; and so I skipped Eric's first point, which is to adopt a consistent style.

The man had a point, and it's still true. Some build files would appear to be an amalgam of different styles. Maybe I've been doing this too long, but I could tell who wrote certain parts of the build at my last project without looking at the version control system to find out. Just like Java code, you need to decide on tabs or spaces to indent with, and how many tabstops/spaces to set. If you're using Eclipse, you can configure this, and check the .project file back into your project's source repository so any Eclipse user can automatically use the same settings:

What's changed in the intervening few years between the publication of Eric's post and this one is just how much better IDE support for Ant has gotten. As long as you don't do anything odd (there's several posts in that one alone) to your build, you should be able to navigate through your build using Eclipse or IDEA. Control-clicking on a target reference will take you to that target's definition, and you can get an outline view, amongst others. You can also run and debug the build from the IDE, which is awfully convenient.

For me, it's a good reason not to put enormous XML comments in the build, like this:

<target name="name" depends="depends" description="--> description">

They would be helpful if you're editing your build.xml using a monochrome terminal and a text editor like vi (which I have done). But as IDE's and editors get better, there's less call for such things. I put my energies into making things consistent and easy to read. Eric: that's 2.

Graphing your CruiseControl build times with Gruff

2008-03-16T15:35:00.001Z

There had been some concern on my last project about build times. My erstwhile colleague Ben started graphing the builds on a sheet of paper and posting it in the kitchen so everybody in the team could see it. When the sheet fell of the wall and the cleaners threw it away, I knew it was time to do something.

Cruise Publisher
How do you get the build time in the first place? I wrote a CruiseControl publisher that called some Ruby code to extract the build duration from the CruiseControl log, which is an XML file. It was a very simple script that matched duration of the Ant build in the XML file, converted the text to a number of seconds, and then made an HTTP POST to a Rails application. The build duration got passed as a POST argument.

We had a Rails application to display other project information. So for us, this was a convenient way to store the information, with the date coming from the database. You could use a text file to store this information instead. Once everything was in place it was easy to write some more Ruby to make the graph appear.

The Code

require 'chart'
require 'gruff'

data = { "count" => [ 3, 5, 2 ] }
g = Gruff::Line.new 640
g.title = "Usages of the word 'foo'"
g.labels = { 0 => 'Monday', 1 => 'Tuesday', 2 => 'Wednesday'}
g.theme_rails_keynote
data.each_pair {|k, v| g.data(k, v)}
blob = g.to_blob

out = File.new("blob.png","w+")
out < blob

This code uses Gruff to actually generate the chart, and you'll need to install this separately. Pass it some data and it will build a PNG file with your graph. By plumbing it into the radiator, the trend in build times is visible to the entire room. It really helped to keep the team focussed on the length of the build - especially for browser based tests that can easy start to slow things down.

[ant]call is evil

2008-03-13T22:48:00.000Z

When you're writing a .NET or Java build file, consider this: antcall and it's cousin call are evil. I'm going to tell you why. First I need to give you a short history lesson.

A brief introduction to make

Make is the daddy of the Ant and Nant build tools. It originally came from the Unix world but it has been ported to just about every computer system, as it's the de facto way to build c code. Make reads makefiles, which tell make how to do your bidding. Inside the makefile, you declare targets, declare their dependencies, and let make do all the work. Here's a simple example of a makefile:

jsimpson$ cat Makefile
a: b c
echo "A: I depend on b and c "

b:
echo "B: I depend on nothing"

c: d
echo "C: I depend on d"

d:
echo "D: I depend on nothing"

The format of the makefile is simple. A target begins with a dependency line containing a name, a colon and an optional list of dependencies. The dependency line is followed by one or more command lines that are indented with a tab. The command lines do the work. In the example above you can see that target A depends on targets B and C, and that target C depends on target D. When we execute the makefile, make determines the order in which it should execute the targets to do the work. Have a look:

jsimpson$ make
echo "B: I depend on nothing"
B: I depend on nothing
echo "D: I depend on nothing"
D: I depend on nothing
echo "C: I depend on d"
C: I depend on d
echo "A: I depend on b and c "
A: I depend on b and c

The make that is installed on my Mac decided that the appropriate order in which to execute the targets was B, D, C and then A. It could have chosen D, C, B and then A. It doesn't matter because you tell make the dependency rules and then make satisfies them. This approach has been working for authors of Unix software since 1977 , thanks to Stuart Feldman at Bell Labs, who wrote the original make. Make is still actively used on many projects today, but for the bulk of Java and .NET projects, you'll be using Ant and NAnt. This is okay. Make isn't exactly cross-platform, and the two tools are evolved with support for building their platforms.

How does this help me write good Nant or Ant buildfiles?

What you are doing when you write a makefile, or build.xml or project.build is declaring dependencies. What the build tool (make, Ant or Nant) does with those dependencies is create a dependency graph. We can visualize the dependency graph of our simple build:

It becomes very important to specify the dependencies of each target correctly. If you write a build target that doesn't declare it's dependencies fully or correctly (say, it actually depends on the output of another target, which is run sometimes) then you end up with a broken or unreliable build. It's an easy mistake to make, especially as complexity on your project grows. But how does all this relate to the [n]ant call?

It seems okay at first glance. Each task will execute a named target in the same buildfile as the calling element. Which is really convenient if you explicitly want to invoke a target at a specific point in your buildfile. But what if that target has dependencies? The Ant documentation hints at the pain:

When a target is invoked by antcall, all of its dependent targets will also be called within the context of any new parameters.

If your antcall or Nant call target has no dependencies, then you have done nothing wrong but violate the dependency based style of the tool. If it has dependencies, then they will be relentlessly run every time you call the target.

One pattern I have seen a lot of is using the [ant]call to specify dependencies in order, rather than declaring them as such:

<target name="chico"/>
<target name="zeppo"/>
<target name="harpo"/>

<target name="groucho">
<antcall target="chico"/>
<antcall target="zeppo"/>
<antcall target="harpo"/>
</target>

In a way I can see why the idea is attractive: You want to be sure of the order of target execution. To be really sure, you need to specify dependencies on the targets. So in this case you'd declare that harpo depended on zeppo, for example.

<target name="groucho" depends="chico,zeppo,harpo">
</target>

Both Ant and Nant will attempt to preserve the order in which you declared them. The crucial difference is that should one of those targets depend on something else, it can adjust the order of target execution to accommodate it.

To give Nant it's due, it does give you a little more room to move: you can set the cascade attribute on the call task to force it not to execute dependencies twice. Another aspect that I haven't covered yet is reuse. If you want to try and invoke a target several times with different parameters, the dependency model doesn't really cut it. Before Ant 1.6 came out, there really was no choice but to use antcall.

Anyway, my advice for anybody using antcall is to substitute it for macrodef, where you really need reuse. .NET developers will be pleased to hear that there's some macrodef support for Nant as well. I hope to write about those soon.

Thanks to T Ashitani for the online dot implementation. Link