The Code Whisperer

Rescuing Legacy Code by Extracting Pure Functions

2011-11-27

After running a handful of Legacy Code Retreats I’ve had the chance to try a number of exercises related to rescuing legacy code. I’d like to share one that has met with very positive reactions from people: extracting pure functions. I use the term pure function to describe a function that only operates on local variables and parameters, but does not touch any state outside the function. No object fields, no global data. I have very little experience in functional programming, so I hope I use the term in a standard way.

No doubt you already know about the Composed Method design pattern, which we commonly use to help understand code as we read it. Most commonly you encounter a block of code with a comment that describes what that code does. You then extract that block of code into a method whose name matches the comment. I’ve used this technique for well on a decade to raise the level of abstraction in code, help code document itself, and eliminate misleading comments. While introducing these methods helps me read the code, it sometimes hides the tangle of dependencies that makes separating responsibilities so difficult. For this reason, I recommend trying to extract pure functions instead.

To introduce a pure function, start with a block of code and extract a method for it. Now look at all the fields and global variables that the method reads and introduce each one as a parameter to the method. Now look at all the fields and global variables that the method writes to and turn these into return values. Where the old code invokes the new function, assign each new return value to the corresponding field or global variable. You know you’ve done this correctly if, of course, the overall behhavior of the program hasn’t changed and you can mark the new function as static or whatever your language calls a class-level function.

In some languages, like Java, you’ll have to introduce a new little class to allow you to return multiple values from the new function. If you don’t want to do that right away, then return a Map of return values. Once you see that you need to return similar Maps from different functions, consider replacing those Maps with a new class. Perhaps that new class will attract some code!

When I’ve used this technique, two key things have happened: either I’ve noticed duplication in the parameter lists of the new functions or introducing a parameter has changed the behavior of the system. In the first case, I introduce Parameter Objects for the duplicated parameters, which then probably attract code and become useful domain objects. In the second case, I’ve detected temporal coupling, which requires me to separate the function into two smaller ones so that some output from the first becomes input to the second. This helps me uncover cohesion problems, usually of the type of different things written too close together.

I realise that an example would help right about now, but I would rather create some screencasts than write out examples in code, but I don’t know when exactly I intend to do that. I wanted to share this idea with you without waiting for the energy to put together a suitable screencast.

I invite you to try introducing pure functions into some legacy code and practising the technique as a kata. Get used to the various maneuvers, like introducing Parameter Objects or Return Value Objects or solving temporal coupling by splitting the function in two. It sounds crazy, but I’d like to try a Legacy Code Retreat where we practise only this technique all day. I don’t know whether anyone else would find it valuable enough to try it together for an entire day, over and over and over.

Would you? Add your comments below.

Detecting Changes in Third-Party Code

2011-11-26

One of the people who watched the 2009 version of Integrated Tests Are A Scam recently asked me: I wonder how you deal with updates of third-party libraries. How do you detect subtle API or behaviour changes? At the moment, I write state-based integration tests for these cases and I wonder whether this isn’t a sensible use of integration tests.

I write Learning Tests to discover how a third-party library works. I isolate myself from the third-party library through a layer of interfaces and adapter classes that evolve from the common ways I use the third-party library. I call this the “Pattern of Usage API”, as it represents the way my application uses that third-party library. Now my application uses the third-party library through a layer of interfaces, which means that I can introduce Contract Tests on those interfaces. These Contract Tests effectively describe the subset of the third-party library’s behavior on which I depend.

Now when I upgrade the third-party library, I run the Contract Tests against my adapters to that library. Test failures usually indicate a backwards incompatible change in the third-party library. (Sometimes they indicate a trivial difference in the API which requires a trivial fix, such as an API call having been renamed or something.)

Of course, this only helps me detect behavior changes related to computing answers, and not related to responsiveness, reliability, scalability, and so on. For that, I’ll always need system tests.

The Contract Tests are almost always state-based integration tests. I simply limit these to the implementation of Pattern of Usage API and don’t let it leak farther up the call stack. At some point you have to integrated with the Outside World. I simply teach people to look to make that integration thinner.

When should I remove duplication?

2011-09-29

As I learn to become a better programmer, I continue to follow the four elements of simple design. Of these, I have observed that “remove duplication” helps me discover an appropriate structure for the thing I want to build. In my classes, we practise removing duplication a lot, in part because most people understand the rule “remove duplication” well enough to find it useful. After the first few weeks of practice, however, programmers following this rule observe varying results: sometimes removing the duplication makes the design much clearer, and sometimes it muddies the water. At this stage, she usually looks for more detailed rules to help decide when to remove duplication and when to leave it alone. I offer some simple rules and guidelines for this situation.

When I can’t decide whether to remove a certain bit of duplication I’ve found, I fall back on two rules:

Remove duplication only after you see three copies.
If you don’t know how to remove this particular kind of duplication, then write more tests. Either you need more examples to see the pattern, or more examples will show a different, better pattern.

I also remember two guidelines:

Don’t be afraid to remove duplication by introducing a method or class or interface with a stupid name. Remember that you can always improve the name later.
If you sense duplication, but don’t really see it, or can’t explain it to others, then make the surrounding names more precise; then maybe you will see the duplication.

Writing Contract Tests in Java differently

2011-07-17

I just wanted to know whether anyone has tried nesting the concrete tests inside the abstract test like this, and if you have, then how do/did you like it?

Contract Tests: An Example

2011-07-07

I found an example of contract tests in Arlo Belshee’s series of articles about mock-free testing. I must strongly, strongly point out that Arlo uses the term “mock” narrowly to refer to runtime- or bytecode-generated proxies that intercept interface method invocations and provide the ability to set method expectations, in the way that JMock and NMock do. He does not mean the generic term “mock”, where he uses the term “test double” instead. I thank him for that.

If you click here you’ll see an almost textbook example of a contract test: that is, a test class that can run the same set of tests for two different implementations of the same interface. I would change only one thing: I’d extract the tests into an abstract superclass—something I otherwise hate to do—and pull the declaration of the method MakeTestSubject() up there, leaving two subclasses, one for the real file system and one for the simulated one. “YAGNI,” you say, and I agree, but I prefer the symmetry of the abstract superclass design to the asymmetry of having one class inherit from the other. I find it easier to grok quickly.

Either way, I feel good seeing contract tests out in the wild. I’m not so crazy after all.

Adding behavior with confidence

2011-06-22

The 30-second version

Adding behavior confidently involves having fewer parts to change (low duplication), knowing which ones to change (high cohesion), ease of changing just the part you want to change (low coupling), and understanding how you’ve changed it (strong tests).
Adding behavior requires breaking an existing assumption.
In a well-factored design, we can easily find the one place we have made that assumption. (Otherwise, why bother refactoring?)
First, make room for the new code, then add it.
To make room for the new code, extract the existing code into a method whose name describes the generalisation we want to make, or the idea we want to introduce.
By making room for the new code, we make that code easier to reuse by reducing its dependence on its surrounding context.

Want to learn more?

Visit the links in this article.
Attend Agile Design: Beyond the Basics.
Pair with me.

The Details

We all want to add behavior to a system confidently, and I have observed that my confidence in adding behavior depends on two factors:

I know where to add code.
I understand the behavior of the code I am adding.

I use test-driven development as the main technique for handling the second of these two factors, but what about the first? I have uncovered a technique that I both use and teach, and I’d like to share that with you. I call this “making room for the new code”, naming it for a phrase I vaguely remember reading in one of the Grand Old XP books. (Did Kent Beck or Ron Jeffries write it? I can’t remember.) This technique helps me quickly find a reasonable first-draft place in the code base to put new code. After I have put the new code in place, and I feel confident that it does what I expect, I then use the Four Elements of Simple Design to guide me in refactoring to improve the design.

A premise

I start with the premise that adding behavior means breaking an assumption. By this I mean that whenever we add code to a system in order to extend its behavior, we have to falsify at least one assumption we’ve previously made. For example:

In a payroll system, in order to support a second cheque printer, we likely have to break the assumption that there is only one cheque format.
In a point of sale system, in order to support separate cash and card payment reports, we likely have to break the assumption that all “we made a sale” events look the same.
In a mobile phone monitoring system, in order to support billing by the second, we likely have to break the assumption that we only have to count the number of minutes a call lasted.

Some of these seem obvious, and others less so, and it bears emphasising that the specific assumption or assumptions we break depends heavily on what we’ve built so far and the way we articulate the soon-to-be-added behavior. Even so, I conjecture that for every behavior we want to add to a system, we can identify a non-empty list of assumptions that we need to break.

The technique

Identify an assumption that the new behavior needs to break.
Find the code that implements that assumption.
Extract that code into a method whose name represents the generalisation you’re about to make.
Enhance the extracted method to include the generalisation.

The less duplication you have in the system, the better this works, because duplicate code makes it difficult to find all the code that implements the assumption in question. Similarly, the more appropriate the names in your system, the better this works, because unsuitable names make it difficult to know which code implements the assumption in question, as opposed to something unrelated. You’ll notice that these points relate both to the Four Elements of Simple Design and to the core concepts of Coupling and Cohesion.

Yes, yes…

An example

We are building a point of sale system, and we’ve just decided to implement sales tax. I live in PEI, Canada, where not only do we exclude sales tax from the price, we have two sales taxes, and we charge the second one on top of the first one. For example:

A $125 item that attracts both GST (the "Goods and Services" tax) and PST (provincial sales tax) costs a total of $144.38. GST at 5% costs $6.25, then PST at 10% of $131.25 (= $125 + $6.25) costs $13.13. The total is $125 + $6.25 + $13.13 = $144.38.

Notice that this example implies that GST or PST might or might not apply to a given product, so even before we identify the old assumption to break, we need to note the new assumption we’ll make: we assume that all products attract both GST and PST. Our customers won’t like it, but the tax authorities will love it, and only they have the power to treat us guilty until proven innocent.

Our code has a class like this in it.

What do we assume now that we can’t allow ourselves to assume any longer? The sale total should increment by the (net) price of the item we sell. By net price here, I refer to the pre-tax price, because of course, until now, our system has no notion of “tax”. Fortunately, because we’ve ruthlessly removed duplication so far, computing the running total of the sale requires only this line of code. We can pretty safely apply the technique of this article right here. To do this, we extract the assumption into a new method whose name represents the generalisation we’re about to make. In this case, we don’t want to increase the sale total by the product’s price, but rather by its cost, which includes any additional charges beyond net price. So, we introduce a method for accumulating the scanned product’s cost.

This creates space for the new code. We test-drive the new code, and end up with this delightful monstrosity.

It looks ugly, but it works. In addition to looking ugly, this method has Feature Envy. Specifically, the calculation part of #accumulate_cost only talks to product, and so it can move onto the class Product, leaving only the accumulating left behind. You could also say that this method had two responsibilities, so I separated them, then notice the feature envy in one of them, then moved it. I can almost always take smaller steps.

Notice the context independence we’ve achieved with the method Product#cost. We can now refine the notion of “cost” freely without worrying about how someone will use that information. We have an easy-to-find, easy-to-extend part of the system where we can add behavior for determining which products attract which taxes, supporting multiple tax calculation policies, and even including shipping, handling and restocking fees. Now we can really add future behavior with confidence.

A model for improving names

2011-06-15

A model for improving the names of variables, fields, interfaces, classes and namespaces in a system. Practise this and more in my course, Agile Design: Beyond the Basics.

Announcing a European Tour in 2011

2011-06-08

I’m touring Europe in 2011. I have planned some public courses and some conferences, and am searching for private consulting and training work. Over the coming weeks I will reveal where I’m appearing on the map. The colors on the map provide a hint! If you’d like to invite me to your event, then please visit here. See you later this year!

The Continuum of Names: an example

2011-05-27

I found someone trying to work on improving names in their code, and gave them a little advice. It gives a reasonable example of some of the things I think about when naming.

The Continuum of Names: A quick summary

2011-05-26

Emma López (@hell03610) shared with the world her notes from my XP2011 session, “A Simple Approach to Modular Design”. In it, she summarises a model I teach about how to improve names in code. I do plan to write about this in more detail this summer.