While I tend to put search in the query camp in the when keeping the responsibility of commands and queries separate, and often recommend that those queries be done without messaging, there are certain types of search where messaging does make sense.

In this post, I’ll describe certain properties of the problem domain that make messaging a good candidate for a solution.

Searching is besides the point – Finding is what it’s all about

Remember that search is only a means to an end in the eyes of the user – they want to find something. One of the difficulties we users have is expressing what we want to find in ways that machines can understand.

In thinking about how we build systems to interact with users, we need to take this fuzziness into account. The more data that we have, the less homogeneous it is, the harder this problem becomes.

When talking about speed, while users are sensitive to the technical interactivity, the thing that matters most is the total time it takes for them to find what they want. If the result of each search screen pops up in 100ms, but the user hasn’t found what they’re looking for after clicking through 20 screens, the search function is ultimately broken.

Notice that the finding process isn’t perceived as “immediate” in the eyes of the user – the evaluation they do in their heads of the search results is as much a part of finding as the search itself.

Also, if the user needs to refine their search terms in order to find what they want, we’re now talking about a multi-request/multi-response process. There is nothing in the problem domain which indicates that finding is inherently request/response.

Relationships in the data

When bringing back data as the result of a search, what we’re saying is that there is a property which is the same across the result elements. But there may be more than one such property. For example, if we search for “blue” on Google Images, we get back pictures of the sky, birds, flowers, and more. Obvious so far – but let’s exploit the obvious a bit.

When the user sees that too many irrelevant results come back, they’ll want to refine their search. One way they can do that is to perform a new search and put in a more specific search phrase – like “blue sky”. Another way is for them to indicate this is by selecting an image and saying “not like this” or “more of these”. Then we can use the additional properties we know about those images to further refine the result group – either adding more images of one kind, or removing images of another.

Here’s something else that’s obvious:

Users often click or change their search before the entire result screen is shown.

It’s beginning to sound like users are already interacting with search in an asynchronous manner. What if we actually designed a system that played to that kind of interaction model?

Data-space partitioning

Once we accept the fact that the user is willing to have more results appear in increments, we can talk about having multiple servers processing the search in parallel. For large data spaces, it is unlikely for us to be able to store all the required meta data for search on one server anyway.

All we really need is a way to index these independent result-sets so that the user can access them. This can be done simply by allocating a GUID/UUID for the search request and storing the result-sets along with that ID.

Browser interaction

When the browser calls a server with the search request the first time, that server allocates an ID to that request, returns a URL containing that ID to the browser, and publishes an event containing the search term and the ID. Each of our processing nodes is subscribed to that event, performs the search on its part of the data-space, and writes its results (likely to a distributed cache) along with that ID.

The browser polls the above URL, which queries the cache (give me everything with this ID), and the browser sees which resources have been added since the last time it polled, and shows them to the user.

If the user clicks “more of these”, that initiates a new search request to the server, which follows the same pattern as before, just that the system is able to pull more relevant information. When implementing “not like this”, this performs a similar search but, instead of adding to the list of items shown, we’re removing items from the list shown based on the response from the server.

In this kind of user-system interaction model, having the user page through the result set doesn’t make very much sense as we’re not capturing the intent of the user, which is “you’re not showing me what I want”. By making it easy for the user to fine tune the result set, we get them closer to finding what they want. By performing work in parallel in a non-blocking manner on smaller sets of data, we greatly decrease the “time to first byte” as well as the time when the user can refine their search.

But Google doesn’t work like that

I know that this isn’t like the search UI we’ve all grown used to.

But then again, the search that you’re providing your users is more specific – not just pages on the web. If you’re a retailer allowing your users to search for a gift, this kind of “more like this, less like that” model is how users would interact with a real sales-person when shopping in a store. Why not model your system after the ways that people behave in the real world?

In closing

If we were to try to make use of messaging underneath “classical” search interaction models, it probably wouldn’t have been the greatest fit. If all we’re doing at a logical level is blocking RPC, then messaging would probably make the system slower. The real power that you get from messaging is being able to technically do things in parallel – that’s how it makes things faster. If you can find ways to see that parallelism in your problem domain, not only will messaging make sense technically – it will really be the only way to build that kind of system.

Learning how to disconnect from seeing the world through the RPC-tinted glasses of our technical past takes time. Focusing on the problem domain, seeing it from the user’s perspective without any technical constraints – that’s the key to finding elegant solutions. More often than not, you’ll see that the real world is non-blocking and parallel, and then you’ll be able to make the best use of messaging and other related patterns.

What are your thought? Post a comment and let me know.

Some interesting numbers were stated in the talk.

• Tens of thousands to hundreds of thousands of requests per second
• Over 3 thousand web servers
• Over a thousand mid-tier servers

No wonder most big web sites don’t run .NET. The Windows licenses would put them out of business.

Well, that is if you follow those same architectural practices.

I’ve written in the past of alternative architectural approaches that can scale to those levels at easily an order of magnitude less hardware (I think it’s closer to two OOMs) – here’s one of them on the topic of weather:

Building Super-Scalable Web Systems with REST.

By the way, the client quoted in that post is now well above 60 million users with only small incremental increases in hardware. Oh, and their running everything on Windows and .NET. The question is not “can it scale”, but rather “how much will it cost to scale”.

Architecture pays itself back faster than ever in the Web 2.0 world.

My latest article has been published in issue 21 of the Microsoft Architecture Journal:

EDA: SOA Through The Looking Glass

Please leave questions and comments here.

I’ve gotten some questions about my DDD presentation there based on Aaron Jensen’s pictures:

Yes – I talk with my hands. All the time.

That slide is quite an important one – I talked about it for at least 2 hours.

Here it is again, this time in full:

You may notice that the nice clean layered abstraction that the industry has gotten so comfortable with doesn’t quite sit right when looking at it from this perspective. The reason for that is that this perspective takes into account physical distribution while layers don’t.

I’ll have some more posts on this topic as well as giving a session in TechEd Europe this November.

Oh – and please do feel free to already send your questions in.

Take a listen.

The thing is, when looking at this in light of the full software development lifecycle, there are signs that the waters run deeper than we might have originally thought.

Let’s take things one step at a time though…

What is it?

Wikipedia tells us:

“Convention over Configuration (aka Coding by convention) is a software design paradigm which seeks to decrease the number of decisions that developers need to make, gaining simplicity, but not necessarily losing flexibility. The phrase essentially means a developer only needs to specify unconventional aspects of the application.”

What this means is that frameworks built in this way have default implementations that can be swapped out if needed. So far so good.

For example…

In NServiceBus, there is an abstraction for how subscription data is stored and multiple implementations – one in-memory, another using a durable MSMQ queue, and a third which uses a database. The convention for that part of the system is that the MSMQ implementation will be used, unless something else is specified.

Developers wishing to specify a different implementation can specify the desired implementation in the container – either one that comes out of the box, or their own implementation of ISubscriptionStorage.

Things get more interesting when we consider the full lifecycle.

Lifecycle effects

When developers are in the early phases of writing a new service, they want to focus primarily on what the service does – its logic. They don’t want to muck around with MSMQ queues for storing subscriptions and would much rather use the in-memory storage.

As the service takes shape and the developers want to run the full service on their machine, possibly testing basic fault-tolerance behaviors – kill one service, see that the others get a timeout, bring the service back up, wanting it to maintain all the previous subscriptions.

Moving on from there, our developers want to take the same system they just tested on their machine and move it into a staging environment. There, they don’t want to use the MSMQ implementation for subscription storage, but rather the database implementation – as will be used in the production environment.

While it may not sound like a big deal – changing the code which specifies which implementation to use when moving from one environment to another, consider that on top of just subscription storage, there is logging (output to console, file, db?), saga persistence (in-memory, file-based DB, relational DB), and more.

It’s actually quite likely that something will get missed as we move the system between environments. Can there be a better way?

What if…

What if there was some way for the developer to express their intent to the system, and the system could change its conventions, without the developer having to change any code or configuration files?

You might compare this (in concept) to debug builds and release builds. Same code, same config, but the runtime behaves different between the two.

As I mulled over how we could capture that intent without any code or config changes, the solution that I kept coming to seemed too trivial at first, so I dismissed it. Yet, it was the simplest one that would work for console and WinForms applications, as well as windows services – command line arguments. The only thing is that I don’t think those are available for web applications.

But since we’re still in “what if” land, and I’m more thinking out loud here than providing workable solutions for tomorrow morning, let’s “what if” command line arguments worked for web apps too.

Command-Line Intent

Going back to our original scenario, when developers are working on the logic of the service, they run it using the generic NServiceBus host process, passing it the command line parameter /lite (or whatever). The host then automatically configures all the in-memory implementations.

As the system progresses, when the developer wants to run everything on their machine, they run the processes with /integration. The host then configures the appropriate implementations (MSMQ for subscription storage, SQLite for saga persistence, etc.

When the developers want to run the system in production, they could specify /production (or maybe that could be the default?), and the database backed implementations would be configured.

Imagine…

Imagine being able to move that fluidly from one environment to another. Not needing to pore over configuration files or startup script code which configures a zillion implementation details. Not needing to worry that as you moved the system to staging something would break.

Imagine short, frictionless iterations even for large scale systems.

Imagine – lifecycle-aware frameworks making all this imagination a reality.

In Closing

We’re not there yet – but we’re not that far either. The generic host we’re providing with NServiceBus 2.0 is now being extended to support exactly these scenarios.

It’s my hope that as more of us think about this challenge, we’ll come up with better solutions and more intelligent frameworks. Just as convention came to our rescue before, breaking us out of the pain of endless XML configuration, I hope this new family of lifecycle-aware frameworks will make the friction of moving a system through dev, test, staging, and production a thing of the past.

A worthy problem for us all to solve, don’t you think?

Any ideas on how to make it a reality?
Send them in – leave a comment below.

Advanced Distributed Systems Design with SOA & DDD

• 
  Sep 7th, London UK
  
• 
  Sep 21, Brussels Belgium
  
• 
  Nov 30, Austin Texas – USA

Loosely-Coupled Messaging with NServiceBus

• 
  Sep 16th, London UK
  

***

There’s also work afoot to bring the SOA course to Australia in October – if you’re interested, let the organizers know here.

Those interested in a private course for their company can be contact me here.

And a quick, final note in closing – those of you interested in NServiceBus should now be able to find more information about it as well as how it relates to current and future Microsoft technologies on the improved NServiceBus.com site.

My article on “employing the domain model pattern” has been published in the August edition of MSDN Magazine.

Here’s a short excerpt:

“In this article, we’ll go through the reasons to (and not to) employ the domain model pattern, the benefits it brings, as well as provide some practical tips on keeping the overall solution as simple as possible.”

Continue reading…

My previous post on Domain Events left some questions about how aggregate roots should be created unanswered. It would actually be more accurate to say how aggregate roots should *not* be created. It turns out that this is one of the less intuitive parts of domain-driven design and has been the source of many arguments on the matter. Let’s start with the wrong way:

   1:  using (ISession s = sf.OpenSession())

   2:  using (ITransaction tx = s.BeginTransaction())

   3:  {

   4:      Customer c = new Customer();

   5:      c.Name = "udi dahan";

   6:   

   7:      s.Save(c);

   8:      tx.Commit();

   9:  }

I understand that the code above is representative of how much code is written when using an object-relational mapper. Many would consider this code to follow DDD principles – that Customer is an aggregate root. Unfortunately – that is not the case. The code above is missing the real aggregate root.

There’s also the inevitable question of validation – if the customer object isn’t willing to accept a name with a space in it, should we throw an exception? That would prevent an invalid entity from being saved, which is good. On the other hand, exceptions should be reserved for truly exceptional occurrences. But if we don’t use exceptions, using Domain Events instead, how do we prevent the invalid entity from being saved?

All of these issues are handled auto-magically once we have a true aggregate root.

Always Get An Entity

Let’s start with the technical guidance – always get an entity. At least one. Also, don’t add any objects to the session or unit of work explicitly – rather, have some other already persistent domain entity create the new entity and add it to a collection property.

Looking at the code above, we see that we’re not following the technical guidance.

But the question is, which entity could we possibly get from the database in this case? All we’re doing is adding a customer.

And that’s exactly where the technical guidance leads us to the business analysis that was missing in this scenario…

Business Analysis

Customers don’t just appear out of thin air.

Blindingly obvious – isn’t it.

So why would we technically model our system as if they did? My guess is that we never really thought about it – it wasn’t our job. So here’s the breaking news – if we want to successfully apply DDD we do need to think about it, it is our job.

Going back to the critical business question:

Where do customers come from?

In the real world, they stroll into the store. In our overused e-commerce example, they navigate to our website. New customers that haven’t used our site before don’t have any cookies or anything we can identify them with. They navigate around, browsing, maybe buying something in the end, maybe not.

Yet, the browsing process is interesting in its own right:

• Which products did they look at?
• Did they use the search feature?
• How long did they spend on each page?
• Did they scroll down to see the reviews?

If and when they do finally buy something, all that history is important and we’d like to maintain a connection to it.

Actually, even before they buy something, what they put in their cart is the interesting piece. The transition from cart to checkout is another interesting piece. Do they actually complete the checkout process, or do they abandon it midway through?

Add to that when we ask/force them to create a user/login in our system.

Are they actually a customer if they haven’t bought anything?

We’re beginning to get an inkling that almost every activity that results in the creation of an entity or storing of additional information can be traced to a transition from a previous business state.

In any transition, the previous state is the aggregate root.

In the beginning…

Let’s start at the very beginning then – someone came to our site. Either they navigated here from some other web page, they clicked on an email link someone sent them, or they typed in our URL. This can be designed as follows:

   1:  using (ISession s = sf.OpenSession())

   2:  using (ITransaction tx = s.BeginTransaction())

   3:  {

   4:     var referrer = s.Get<Referrer>(msg.URL);

   5:     referrer.BroughtVisitorWithIp(msg.IpAddress);

   6:   

   7:     tx.Commit();

   8:  }

   9:   

And our referrer code could look something like this:

   1:  public void BroughtVisitorWithIp(string ipAddress)

   2:  {

   3:     var visitor = new Visitor(ipAddress);

   4:     this.NewVisitors.Add(visitor);

   5:  }

   6:   

This follows the technical guidance we saw at the beginning.

It also allows us to track which referrer is bringing us which visitors, through tracking those visitors as they become shoppers (by putting stuff in their cart), finally seeing which become customers.

We can solve the situation of not having a referrer by implementing the null object pattern which is well supported by all the standard object-relational mappers these days.

How it works internally

When we call a method on a persistent entity retrieved by the object-relational mapper, and the entity modifies its state like when it adds a new entity to one of its collection properties, when the transaction commits, here’s what happens:

The mapper sees that the persistent entity is dirty, specifically, that its collection property was modified, and notices that there is an object in there that isn’t persistent. At that point, the mapper knows to persist the new entity without us ever having to explicitly tell it to do so. This is sometimes known as “persistence by reachability”.

Where validation happens

Let’s consider the relatively trivial rule that says that a user name can’t contain a space.

Also, keep in mind that a registered user is the result of a transition from a visitor.

Here’s *one* way of doing that:

   1:  public class Visitor

   2:  {

   3:     public void Register(string username, string password)

   4:     {

   5:        if (username.Contains(" "))

   6:        {

   7:           DomainEvents.Raise<UsernameCantContainSpace>();

   8:           return;

   9:        }

  10:   

  11:        var user = new User(username, password);

  12:        this.RegisteredUser = u;

  13:     }

  14:  }

  15:   

This actually isn’t representative of most of the rules that will be found in the domain model, but it illustrates a way of preventing an entity from being created without our service layer needing to know anything. All the service layer does is get the visitor object and call the Register method.

Validation of string lengths, data ranges, etc is not domain logic and is best handled elsewhere (and a topic for a different post). The same goes for uniqueness.

Summary

The most important thing to keep in mind is that if your service layer is newing up some entity and saving it – that entity isn’t an aggregate root *in that use case*. As we saw above, in the original creation of the Visitor entity by the Referrer, the visitor class wasn’t the aggregate root. Yet, in the user registration use case, the Visitor entity was the aggregate root.

Aggregate roots aren’t a structural property of the domain model.

And in any case, don’t go saving entities in your service layer – let the domain model manage its own state. The domain model doesn’t need any references to repositories, services, units of work, or anything else to manage its state.

If you do all this, you’ll also be able to harness the technique of fetching strategies to get the best performance out of your domain model by representing your use cases as interfaces on the domain model like IRegisterUsers (implemented by Visitor) and IBringVisitors (implemented by Referrer).

And spending some time on business analysis doesn’t hurt either – unless customers really do fall out of the sky in your world

Background

Before diving right into the code, I wanted to take a minute to recall how we got here.

It started by looking for how to create fully encapsulated domain models.

The main assertion being that you do *not* need to inject anything into your domain entities.

Not services. Not repositories. Nothing.

Just pure domain model goodness.

Make Roles Explicit

I’m going to take the advice I so often give. A domain event is a role, and thus should be represented explicitly:

   1:  public interface IDomainEvent {}

If this reminds you of the IMessage marker interface in nServiceBus, you’re beginning to see where this is going…

How to define domain events

A domain event is just a simple POCO that represents an interesting occurence in the domain. For example:

   1:  public class CustomerBecamePreferred : IDomainEvent 

   2:  {

   3:      public Customer Customer { get; set; }

   4:  }

For those of you concerned about the number of events you may have, and therefore are thinking about bunching up these events by namespaces or things like that, slow down. The number of domain events and their cohesion is directly related to that of the domain model.

If you feel the need to split your domain events up, there’s a good chance that you should be looking at splitting your domain model too. This is the bottom-up way of identifying bounded contexts.

How to raise domain events

In your domain entities, when a significant state change happens you’ll want to raise your domain events like this:

   1:  public class Customer

   2:  {

   3:      public void DoSomething()

   4:      {

   5:          DomainEvents.Raise(new CustomerBecamePreferred() { Customer = this });

   6:      }

   7:  }

We’ll look at the DomainEvents class in just a second, but I’m guessing that some of you are wondering “how did that entity get a reference to that?” The answer is that DomainEvents is a static class. “OMG, static?! But doesn’t that hurt testability?!” No, it doesn’t. Here, look:

Unit testing with domain events

One of the things we’d like to check when unit testing our domain entities is that the appropriate events are raised along with the corresponding state changes. Here’s an example:

   1:  public void DoSomethingShouldMakeCustomerPreferred()

   2:  {

   3:      var c = new Customer();

   4:      Customer preferred = null;

   5:   

   6:      DomainEvents.Register<CustomerBecamePreferred>(

   7:          p => preferred = p.Customer

   8:              );

   9:   

  10:      c.DoSomething();

  11:      Assert(preferred == c && c.IsPreferred);

  12:  }

As you can see, the static DomainEvents class is used in unit tests as well. Also notice that you don’t need to mock anything – pure testable bliss.

Who handles domain events

First of all, consider that when some service layer object calls the DoSomething method of the Customer class, it doesn’t necessarily know which, if any, domain events will be raised. All it wants to do is its regular schtick:

   1:  public void Handle(DoSomethingMessage msg)

   2:  {

   3:      using (ISession session = SessionFactory.OpenSession())

   4:      using (ITransaction tx = session.BeginTransaction())

   5:      {

   6:          var c = session.Get<Customer>(msg.CustomerId);

   7:          c.DoSomething();

   8:   

   9:          tx.Commit();

  10:      }

  11:  }

The above code complies with the Single Responsibility Principle, so the business requirement which states that when a customer becomes preferred, they should be sent an email belongs somewhere else.

Notice that the key word in the requirement – “when”.

Any time you see that word in relation to your domain, consider modeling it as a domain event.

So, here’s the handling code:

   1:  public class CustomerBecamePreferredHandler : Handles<CustomerBecamePreferred>

   2:  { 

   3:     public void Handle(CustomerBecamePreferred args)

   4:     {

   5:        // send email to args.Customer

   6:     }

   7:  } 

This code will run no matter which service layer object we came in through.

Here’s the interface it implements:

   1:  public interface Handles<T> where T : IDomainEvent

   2:  {

   3:      void Handle(T args); 

   4:  } 

Fairly simple.

Please be aware that the above code will be run on the same thread within the same transaction as the regular domain work so you should avoid performing any blocking activities, like using SMTP or web services. Instead, prefer using one-way messaging to communicate to something else which does those blocking activities.

Also, you can have multiple classes handling the same domain event. If you need to send email *and* call the CRM system *and* do something else, etc, you don’t need to change any code – just write a new handler. This keeps your system quite a bit more stable than if you had to mess with the original handler or, heaven forbid, service layer code.

Where domain event handlers go

These handler classes do not belong in the domain model.

Nor do they belong in the service layer.

Well, that’s not entirely accurate – you see, there’s no *the* service layer. There is the part that accepts messages from clients and calls methods on the domain model. And there is another, independent part that handles events from the domain. Both of these will probably make use of a message bus, but that implementation detail shouldn’t deter you from keeping each in their own package.

The infrastructure

I know you’ve been patient, reading through all my architectural blah-blah, so here it is:

   1:  public static class DomainEvents

   2:  { 

   3:      [ThreadStatic] //so that each thread has its own callbacks

   4:      private static List<Delegate> actions;

   5:   

   6:      public static IContainer Container { get; set; } //as before

   7:   

   8:      //Registers a callback for the given domain event

   9:      public static void Register<T>(Action<T> callback) where T : IDomainEvent

  10:      {

  11:         if (actions == null)

  12:            actions = new List<Delegate>();

  13:   

  14:         actions.Add(callback);

  15:     }

  16:   

  17:     //Clears callbacks passed to Register on the current thread

  18:     public static void ClearCallbacks ()

  19:     {

  20:         actions = null;

  21:     }

  22:   

  23:     //Raises the given domain event

  24:     public static void Raise<T>(T args) where T : IDomainEvent

  25:     {

  26:        if (Container != null)

  27:           foreach(var handler in Container.ResolveAll<Handles<T>>())

  28:              handler.Handle(args);

  29:   

  30:        if (actions != null)

  31:            foreach (var action in actions)

  32:                if (action is Action<T>)

  33:                    ((Action<T>)action)(args);

  34:     }

  35:  } 

Notice that while this class *can* use a container, the container isn’t needed for unit tests which use the Register method.

When used server side, please make sure that you add a call to ClearCallbacks in your infrastructure’s end of message processing section. In nServiceBus this is done with a message module like the one below:

   1:  public class DomainEventsCleaner : IMessageModule

   2:  { 

   3:      public void HandleBeginMessage() { }

   4:   

   5:      public void HandleEndMessage()

   6:      {

   7:          DomainEvents.ClearCallbacks();

   8:      }

   9:  }

The main reason for this cleanup is that someone just might want to use the Register API in their original service layer code rather than writing a separate domain event handler.

Summary

Like all good things in life, 3rd time’s the charm.

It took a couple of iterations, and the API did change quite a bit, but the overarching theme has remained the same – keep the domain model focused on domain concerns. While some might say that there’s only a slight technical difference between calling a service (IEmailService) and using an event to dispatch it elsewhere, I beg to differ.

These domain events are a part of the ubiquitous language and should be represented explicitly.

CustomerBecamePreferred is nothing at all like IEmailService.

In working with your domain experts or just going through a requirements document, pay less attention to the nouns and verbs that Object-Oriented Analysis & Design call attention to, and keep an eye out for the word “when”. It’s a critically important word that enables us to model important occurrences and state changes.

What do you think? Are you already using this approach? Have you already tried it and found it broken in some way? Do you have any suggestions on how to improve it?

Let me know – leave a comment below.

There’s this belief that if we just reused more code, everything would be better.

Some even go so far as saying that the whole point of object-orientation was reuse – it wasn’t, encapsulation was the big thing. After that component-orientation was the thing that was supposed to make reuse happen. Apparently that didn’t pan out so well either because here we are now pinning our reuseful hopes on service-orientation.

Entire books of patterns have been written on how to achieve reuse with the orientation of the day.
Services have been classified every which way in trying to achieve this, from entity services and activity services, through process services and orchestration services. Composing services has been touted as the key to reusing, and creating reusable services.

I might as well let you in on the dirty-little secret:

Reuse is a fallacy

Before running too far ahead, let’s go back to what the actual goal of reuse was: getting done faster.

That’s it.

It’s a fine goal to have.

And here’s how reuse fits in to the picture:

If we were to write all the code of a system, we’d write a certain amount of code.
If we could reuse some code from somewhere else that was written before, we could write less code.
The more code we can reuse, the less code we write.
The less code we write, the sooner we’ll be done!

However, the above logical progression is based on another couple of fallacies:

Fallacy: All code takes the same amount of time to write

Fallacy: Writing code is the primary activity in getting a system done

Anyone who’s actually written some code that’s gone into production knows this.

There’s the time it takes us to understand what the system should do.
Multiply that by the time it takes the users to understand what the system should do
Then there’s the integrating that code with all the other code, databases, configuration, web services, etc.
Debugging. Deploying. Debugging. Rebugging. Meetings. Etc.

Writing code is actually the least of our worries.
We actually spend less time writing code than…

Rebugging code

Also known as bug regressions.

This is where we fix one piece of code, and in the process break another piece of code.
It’s not like we do it on purpose. It’s all those dependencies between the various bits of code.
The more dependencies there are, the more likely something’s gonna break.
Especially when we have all sorts of hidden dependencies,
like when other code uses stuff we put in the database without asking us what it means,
or, heaven forbid, changing it without telling us.

These debugging/rebugging cycles can make stabilizing a system take a long time.

So, how does reuse help/hinder with that?

Here’s how:

Dependencies multiply by reuse

It’s to be expected. If you wrote the code all in one place, there are no dependencies. By reusing code, you’ve created a dependency. The more you reuse, the more dependencies you have. The more dependencies, the more rebugging.

Of course, we need to keep in mind the difference between…

Reuse & Use

Your code uses the runtime API (JDK, .NET BCL, etc).
Likewise other frameworks like (N)Hibernate, Spring, WCF, etc.

Reuse happens when you extend and override existing behaviors within other code.
This is most often done by inheritance in OO languages.

Interestingly enough, by the above generally accepted definition, most web services “reuse” is actually really use.

Let’s take a look at the characteristics of the code we’re using and reusing to see where we get the greatest value:

The value of (re)use

If we were to (re)use a piece of code in only one part of our system, it would be safe to say that we would get less value than if we could (re)use it in more places. For example, we could say that for many web applications, the web framework we use provides more value than a given encryption algorithm that we may use in only a few places.

So, what characterizes the code we use in many places?

Well, it’s very generic.

Actually, the more generic a piece of code, the less likely it is that we’ll be changing something in it when fixing a bug in the system.

That’s important.

However, when looking at the kind of code we reuse, and the reasons around it, we tend to see very non-generic code – something that deals with the domain-specific behaviors of the system. Thus, the likelihood of a bug fix needing to touch that code is higher than in the generic/use-not-reuse case, often much higher.

How it all fits together

Goal: Getting done faster
Via: Spending less time debugging/rebugging/stabilizing
Via: Having less dependencies reasonably requiring a bug fix to touch the dependent side
Via: Not reusing non-generic code

This doesn’t mean you shouldn’t use generic code / frameworks where applicable – absolutely, you should.
Just watch the number of kind of dependencies you introduce.

Back to services

So, if we follow the above advice with services, we wouldn’t want domain specific services reusing each other.
If we could get away with it, we probably wouldn’t even want them using each other either.

As use and reuse go down, we can see that service autonomy goes up. And vice-versa.
Luckily, we have service interaction mechanisms from Event-Driven Architecture that enable use without breaking autonomy.
Autonomy is actually very similar to the principle of encapsulation that drove object-orientation in the first place.
Interesting, isn’t it?

In summary

We all want to get done faster.

Way back when, someone told us reuse was the way to do that.

They were wrong.

Reuse may make sense in the most tightly coupled pieces of code you have, but not very much anywhere else.

When designing services in your SOA, stay away from reuse, and minimize use (with EDA patterns).

The next time someone pulls the “reuse excuse”, you’ll be ready.

Further Reading

• Additional logic required for service autonomy
• Self-contained events & SOA
• Autonomous Services and Enterprise Entity Aggregation [MS Architecture Journal]
• Does SOA mean the end of OO? [Podcast]

I’ll be discussing business service boundaries, publish/subscribe eventing, and business activity monitoring – with some time for questions at end.

Update: Recording is now online – here.

Hope to virtually see you there.

Here’s the details of the live meeting:

Start Time: Monday, June 01, 2009 07:00 PM GMT
End Time: Monday, June 01, 2009 08:30 PM GMT

Attendee URL: http://snipr.com/virtualaltnet (Live Meeting)

“A common complaint is that the first call on a client object takes some disproportionately large amount of time, usually ten seconds or more, while successive calls are instantaneous. There are many reasons why this might happen so there’s no generic resolution for this problem.” — Nicholas Allen

The thing is that there IS a generic solution to this problem.

It’s queued messaging.

The only thing is that you have to give up talking to your services as if they were regular objects – calling methods on them and expecting a response. In other words, designing a distributed systems isn’t like designing a regular OO system just with some WCF sprinkled on top.

Even when trying to do fire and forget messaging on top of WCF (void method calls with the OneWay attribute), the underlying channel can still block your thread, as Nick mentioned.

A queue isn’t an implementation detail.
It’s the primary architectural abstraction of a distributed system.

First of all, the conventional wisdom:

“If you don’t choose assembly to represent component, the natural artifact candidate is then namespace.”

There’s only one minor assumption here that deserves being dragged out into the light.

While Visual Studio creates an assembly from every project by default, you can take those assemblies and merge them together into a single assembly using this nice little utility from Microsoft. It is likely that each project would have its own namespace too, so we should still be aligned with the conventional wisdom.

In other words, we could choose a Visual Studio project to represent a logical component and still be in the same camp as Jeremy:

“I’m very firmly in the camp that says you should only split assemblies by deployment targets”

What everyone agrees about seems to be that coupling hurts, and should be managed.

Where does coupling come from? Well, from references between two pieces of code. If we were to represent our logical components as Visual Studio projects, we could easily see those references without the help of any 3rd party tools. The compiler would even yell at us if we were to (accidentally) create an evil circular reference.

While some might complain about the long compile time when we have many projects in a single solution, good componentization often doesn’t require us to put all projects in a single solution. In fact, each component could theoretically have its own solution – since it’s reasonable to assume we’d really only be working on one component at a time. In which case, compile time per developer task would be a non-issue.

Going through the whole code base is usually only needed when doing a full-system debug when trying to track down a problem. This wouldn’t need to be done against a solution with all projects. We’d do this using PDBs of the merged projects (as that’s what actually got delivered, and where the bug was found). After spelunking through those PDBs, we’d eventually find the problematic component (or 2, or 3, or …), and open up developer tasks for each component.

Regardless of if we’re putting out a patch for an existing customer or rolling these changes into a release with other tasks, all the logical components would be built into a physical system (merged as necessary) and the system would be put through QA.

In short, it looks like just a bit of unconventional wisdom gets us a nice balance.

The common e-commerce example

We accept orders, bill the customer, and then ship them the product.

Fairly straight-forward.

Since each part of that process can be quite complex, let’s have each step be handled by a service:

Sales, Billing, and Shipping. Each of these services will publish an event when it’s done its part. Sales will publish OrderAccepted containing all the order information – order Id, customer Id, products, quantities, etc. Billing will publish CustomerBilledForOrder containing the customer Id, order Id, etc. And Shipping will publish OrderShippedToCustomer with its data.

So far, so good. EDA and SOA seem to be providing us some value.

Where’s the saga?

Well, let’s consider the behavior of the Shipping service. It shouldn’t ship the order to the customer until it has received the CustomerBilledForOrder event as well as the OrderAccepted event. In other words, Shipping needs to hold on to the state that came in the first event until the second event comes in. And this is exactly what sagas are for.

Let’s take a look at the saga code that implements this. In order to simplify the sample a bit, I’ll be omitting the product quantities.

   1:      public class ShippingSaga : Saga<ShippingSagaData>,

   2:          ISagaStartedBy<OrderAccepted>,

   3:          ISagaStartedBy<CustomerBilledForOrder>

   4:      {

   5:          public void Handle(OrderAccepted message)

   6:          {

   7:              this.Data.ProductIdsInOrder = message.ProductIdsInOrder;

   8:          }

   9:   

  10:          public void Handle(CustomerBilledForOrder message)

  11:          {

  12:               this.Bus.Send<ShipOrderToCustomer>(

  13:                  (m =>

  14:                  {

  15:                      m.CustomerId = message.CustomerId;

  16:                      m.OrderId = message.OrderId;

  17:                      m.ProductIdsInOrder = this.Data.ProductIdsInOrder;

  18:                  }

  19:                  ));

  20:   

  21:              this.MarkAsComplete();

  22:          }

  23:   

  24:          public override void Timeout(object state)

  25:          {

  26:              

  27:          }

  28:      }

First of all, this looks fairly simple and straightforward, which is good.

It’s also wrong, which is not so good.

One problem we have here is that events may arrive out of order – first CustomerBilledForOrder, and only then OrderAccepted. What would happen in the above saga in that case? Well, we wouldn’t end up shipping the products to the customer, and customers tend not to like that (for some reason).

There’s also another problem here. See if you can spot it as I go through the explanation of ISagaStartedBy<T>.

Saga start up and correlation

The “ISagaStartedBy<T>” that is implemented for both messages indicates to the infrastructure (NServiceBus) that when a message of that type arrives, if an existing saga instance cannot be found, that a new instance should be started up. Makes sense, doesn’t it? For a given order, when the OrderAccepted event arrives first, Shipping doesn’t currently have any sagas handling it, so it starts up a new one. After that, when the CustomerBilledForOrder event arrives for that same order, the event should be handled by the saga instance that handled the first event – not by a new one.

I’ll repeat the important part: “the event should be handled by the saga instance that handled the first event”.

Since the only information we stored in the saga was the list of products, how would we be able to look up that saga instance when the next event came in containing an order Id, but no saga Id?

OK, so we need to store the order Id from the first event so that when the second event comes along we’ll be able to find the saga based on that order Id. Not too complicated, but something to keep in mind.

Let’s look at the updated code:

   1:      public class ShippingSaga : Saga<ShippingSagaData>,

   2:          ISagaStartedBy<OrderAccepted>,

   3:          ISagaStartedBy<CustomerBilledForOrder>

   4:      {

   5:          public void Handle(CustomerBilledForOrder message)

   6:          {

   7:              this.Data.CustomerHasBeenBilled = true;

   8:   

   9:              this.Data.CustomerId = message.CustomerId;

  10:              this.Data.OrderId = message.OrderId;

  11:   

  12:              this.CompleteIfPossible();

  13:          }

  14:   

  15:          public void Handle(OrderAccepted message)

  16:          {

  17:              this.Data.ProductIdsInOrder = message.ProductIdsInOrder;

  18:   

  19:              this.Data.CustomerId = message.CustomerId;

  20:              this.Data.OrderId = message.OrderId;

  21:   

  22:              this.CompleteIfPossible();

  23:          }

  24:   

  25:          private void CompleteIfPossible()

  26:          {

  27:              if (this.Data.ProductIdsInOrder != null && this.Data.CustomerHasBeenBilled)

  28:              {

  29:                  this.Bus.Send<ShipOrderToCustomer>(

  30:                     (m =>

  31:                     {

  32:                         m.CustomerId = this.Data.CustomerId;

  33:                         m.OrderId = this.Data.OrderId;

  34:                         m.ProductIdsInOrder = this.Data.ProductIdsInOrder;

  35:                     }

  36:                     ));

  37:                  this.MarkAsComplete();

  38:              }

  39:          }

  40:      }

And that brings us to…

Saga persistence

We already saw why Shipping needs to be able to look up its internal sagas using data from the events, but what that means is that simple blob-type persistence of those sagas is out. NServiceBus comes with an NHibernate-based saga persister for exactly this reason, though any persistence mechanism which allows you to query on something other than saga Id would work just as well.

Let’s take a quick look at the saga data that we’ll be storing and see how simple it is:

   1:      public class ShippingSagaData : ISagaEntity

   2:      {

   3:          public virtual Guid Id { get; set; }

   4:          public virtual string Originator { get; set; }

   5:          public virtual Guid OrderId { get; set; }

   6:          public virtual Guid CustomerId { get; set; }

   7:          public virtual List<Guid> ProductIdsInOrder { get; set; }

   8:          public virtual bool CustomerHasBeenBilled { get; set; }

   9:      }

You might have noticed the “Originator” property in there and wondered what it is for. First of all, the ISagaEntity interface requires the two properties Id and Originator. Originator is used to store the return address of the message that started the saga. Id is for what you think it’s for. In this saga, we don’t need to send any messages back to whoever started the saga, but in many others we do. In those cases, we’ll often be handling a message from some other endpoint when we want to possibly report some status back to the client that started the process. By storing that client’s address the first time, we can then “ReplyToOriginator” at any point in the process.

The manufacturing sample that comes with NServiceBus shows how this works.

Saga Lookup

Earlier, we saw the need to search for sagas based on order Id. The way to hook into the infrastructure and perform these lookups is by implementing “IFindSagas<T>.Using<M>” where T is the type of the saga data and M is the type of message. In our example, doing this using NHibernate would look like this:

   1:      public class ShippingSagaFinder : 

   2:          IFindSagas<ShippingSagaData>.Using<OrderAccepted>,

   3:          IFindSagas<ShippingSagaData>.Using<CustomerBilledForOrder>

   4:      {

   5:          public ShippingSagaData FindBy(CustomerBilledForOrder message)

   6:          {

   7:              return FindBy(message.OrderId)

   8:          }

   9:   

  10:          public ShippingSagaData FindBy(OrderAccepted message)

  11:          {

  12:              return FindBy(message.OrderId)

  13:          }

  14:   

  15:          private ShippingSagaData FindBy(Guid orderId)

  16:          {

  17:              return sessionFactory.GetCurrentSession().CreateCriteria(typeof(ShippingSagaData))

  18:                  .Add(Expression.Eq("OrderId", orderId))

  19:                  .UniqueResult<ShippingSagaData>();

  20:          }

  21:   

  22:          private ISessionFactory sessionFactory;

  23:   

  24:          public virtual ISessionFactory SessionFactory

  25:          {

  26:              get { return sessionFactory; }

  27:              set { sessionFactory = value; }

  28:          }

  29:      }

For a performance boost, we’d probably index our saga data by order Id.

On concurrency

Another important note is that for this saga, if both messages were handled in parallel on different machines, the saga could get stuck. The persistence mechanism here needs to prevent this. When using NHibernate over a database with the appropriate isolation level (Repeatable Read – the default in NServiceBus), this “just works”. If/When implementing your own saga persistence mechanism, it is important to understand the kind of concurrency your business logic can live with.

Take a look at Ayende’s example for mobile phone billing to get a feeling for what that’s like.

Summary

In almost any event-driven architecture, you’ll have services correlating multiple events in order to make decisions. The saga pattern is a great fit there, and not at all difficult to implement. You do need to take into account that events may arrive out of order and implement the saga logic accordingly, but it’s really not that big a deal. Do take the time to think through what data will need to be stored in order for the saga to be fault-tolerant, as well as a persistence mechanism that will allow you to look up that data based on event data.

If you feel like giving this approach a try, but don’t have an environment handy for this, download NServiceBus and take a look at the samples. It’s really quick and easy to get set up.

Very little has changed, so the version 1.9 story described here is still accurate.

Just last week one of my clients went live with a rollout to one of the world’s biggest names in the hospitality industry and things are looking good. Since stability is such a big deal to them (and many of my other clients), they’ve rolled out on nServiceBus 1.8 but now are ready to make the move to 1.9. Hopefully we’ll be able to get a case study out of them

For more information, go to the NServiceBus site.

I’ve been to too many clients where I’ve been brought in to help them with their problems around service versioning when the solution I propose is simply to have version N+1 of the system be backwards-compatible with version N. If two adjacent versions of a given system aren’t compatible with each other, it is practically impossible to solve versioning issues.

Here’s what happens when versions aren’t compatible:

Admins stop the system from accepting any new requests, and wait until all current requests are done processing. They take a backup/snapshot of all relevant parts of the system (like data in the DB). Then, bring down the system – all of it. Install the new version on all machines. Bring everything back up. Let the users back in.

If, heaven-forbid, problems were uncovered with the new version (since some problems only appear in production), the admins have to roll back to the previous version – once again bringing everything down.

This scenario is fairly catastrophic for any company that requires not-even high availability, but pretty continuous availability – like public facing web apps.

If adjacent versions were compatible with each other, we could upgrade the system piece-meal – machine by machine, where both the old and new versions will be running side by side, communicating with each other. While the system’s performance may be sub-optimal, it will continue to be available throughout upgrades as well as downgrades.

This isn’t trivial to do.

It impacts how you decide what is (and more importantly, what isn’t) nullable.

It may force you to spread certain changes to features across more versions (aka releases).

As such, you can expect this to affect how you do release and feature planning.

However, if you do not take these factors into account, it’s almost a certainty that your versioning problems will persist and no technology (new or old) will be able to solve them.

Coming next… Units of versioning – inside and outside a service.

My article on “optimizing a large-scale Software+Services application” has been published in the April edition of MSDN Magazine.

Here’s a short excerpt:

“We had to juggle occasional connectivity, data synchronization, and publish/subscribe all at the same time. We learned that we couldn’t solve all problems either client-side or server-side, but rather that an integrated approach was needed since any changes on one side needed corresponding changes on the other side.”

Continue reading…

Those of you following me on twitter (here) have an inkling that it has to do with the connection between user interfaces and services – but that’ll have to wait. Don’t want to steal my own thunder.

A nice meaty blog post on the topic is coming.

Greg Young asserts:

“I believe that this shows there to be a rather negligible cost associated with the use of such a model. There is however a small cost, this cost however I believe only exists when one looks at the system in isolation.”

Ayende adds his perspective:

“The cost of messaging, and a very real one, comes when you need to understand the system. In a system where message exchange is the form of communication, it can be significantly harder to understand what is going on.”

Of course, both these intelligent fellows are right. The reason for the apparent disparity in viewpoints has to do with which part of the following graph you look at. Ayende zooms in on the left side:

As systems get larger, though, the only way to understand them is by working at higher levels of abstraction. That’s where messaging really shines, as the incremental complexity remains the same by maintaining the same modularity as before:

In Ayende’s post, he follows the design I described a while back on using messaging for user management and login for a high-scale web scenario. In his comments, he agrees with the above stating:

“I certainly think that a similar solution using RPC would be much more complex and likely more brittle.”

I feel quite conservative in saying the most enterprise solutions fall on the right side of the intersection in the graph.

That being said, don’t underestimate the learning curve developers go through with messaging. While the mechanics are similar, the mindset is very different. Think about it like this:

You’ve driven a car for years in the US. It’s practically second nature. Then you fly to the UK, rent a car, and all of a sudden, your brain is in meltdown. (or vice versa for those going from the UK to the US)

Summary

If you are going down the messaging route, please be aware that there are shades of gray there as well. You don’t have to implement your user management and login the way I outlined in my post if you don’t require such high levels of scalability, but even lower levels of scalability can benefit from messaging.

Just as there isn’t a single correct design for non-messaging solutions, the same is true for those using messaging. Finding the right balance is tricky, and critical.

When the code is simple in every part of the system, and the asynchronous interactions are what provide for the necessary complexity the problem domain requires, that’s when you know you’ve got it just right.

Definitely worth checking out.

Register here

What is it?

From the NServiceBus.com site:

“NServiceBus is a powerful, yet lightweight, open source messaging framework
for designing distributed .NET enterprise systems. Entirely pluggable yet simple to use, NServiceBus gives programmers a head-start on developing robust, scalable,
and maintainable service-layers and long-running business processes.”

One of the developers who downloaded nServiceBus, Jürgen, sent me this in an email:

“I took the samples, swapped in my own code, and had a machines subscribing, publishing, messaging, in like 15 minutes.

I’ve got to tell you – from reading the architecture stuff on your blog I always thought this stuff was going to be hard. I know you always say its supposed to be simple but I never really believed it. I mean, seriously, if I had to do this stuff with web services or WCF – well, I wouldn’t know where to begin.”

(His English totally surprised me – not what you’d expect from someone called Jürgen. Born in Sweden but grew up in the US.)

BTW, Sam’s showed where to begin with WCF: basic pub/sub without durability is 480 LOC.

What’s different – Containers

People who looked at earlier versions of nServiceBus will see many incremental improvements that smooth over previously rough parts of the framework.

Chris Patterson, co-founder of MassTransit mentioned one such area in his blog post:

“Both Dru and I needed a framework for asynchronous messaging to address some work-related application requirements. While MSMQ is provided out of the box [on Windows], it doesn’t directly encourage some good distributed application practices such a loose coupling. Our goal was to abstract the messaging aspects so the services could be built to deal with plain old objects (POCOs) instead of lower level transport messages.

Originally, we both looked at NServiceBus as a way to make this happen. I’ve followed Udi’s blog for a while and have really gained a lot of knowledge from his posts and presentations. However, our lack of experience in Spring.NET, along with a general lack of understanding of all the complexity of such a framework led us down the path of building our own framework.”

Version 1.9 takes a totally different approach (than v1.6/1.7 at which I believe Chris was looking back then) to dependency injection frameworks (like Spring.NET) and decreases their footprint. Developers don’t need to know anything about Spring, Castle, or any other container to get started, but always have the ability to configure it however they want and even swap in their container of choice.

What’s different – DLL Footprint

Another difference is the number of assemblies that come with nServiceBus. Nathan Stults had this to say about version 1.8:

“NServiceBus (for valid architectural reasons) is split up into eleven thousand, two hundred and nine different DLL’s. That may be an exaggeration, but many of the dll’s have only one or two code files in them, with only a few lines of code each.”

NServiceBus has always supported swapping out various technological implementations and will continue do so. For that reason, the development environment is organized into 79 projects whose dependencies are managed very strictly. As of version 1.9, most of these assemblies including many supporting libraries like Spring and Castle have been merged into NServiceBus.dll. There’s also NServiceBus.Testing.dll which supports fluent-unit-testing entire business processes.

There is one non-optional external dependency which hasn’t been merged and that is log4net – although if you configure in a Common.Logging provider to your own logging infrastructure, you can do without it as well. With log4net, the minimum deployment footprint is 2 assemblies. The other dependency which is optional is NHibernate. The reason for leaving these out is that many teams depend on specific versions of those assemblies.

In short – you reference ONE assembly. Just one.

What else?

There is quite a lot in there. Ayende’s put out several posts describing those features and similarities to the bus he’s working on:

• Managing Timeouts
• Long-running processes
• Load Balancing

Ayende’s description of the NServiceBus load-balancing capability was:

“The distributor section of NServiceBus is a thing of beauty.”

Performance

You may be surprised by the kind of performance we’re able to wring out of “basic” MSMQ. Keep in mind, though, that you can swap in your own transport – there are already some others out there on the Contrib site including ActiveMQ and shared memory.

The most thorough performance numbers I’ve seen on nServiceBus have been written up here:

“total throughput over 1 billion messages an hour. That was about 100 million per hour durable, 900 million per hour non-durable”

This was on an 3 blade centers (48 blades), 30 pizza boxes (1U), and 20 clusters, version 1.8.

On Gojko’s blog there is a video of talk on nServiceBus where Dave de Florinier mentioned throughputs of 600,000 messages per hour (no mention of supporting hardware) and I think it was version 1.7 or 1.8 of nServiceBus.

Recently, Raymond Lewallen posted his numbers for 1.9 to the discussion group:

“We load tested our full duplex and pub/sub scenarios the other day at about 500 messages per second with no hiccups at all.” (save yourself the math, its 1.8M/hr)

The really interesting numbers are coming in from one of my clients (on 1.9) where the focus is on business process (saga) throughput where they’re seeing millions completing each day on top of IO intensive technical processes. I’m excited.

Looking forward

After working with clients using nServiceBus over 4 years now (even before it was open source or had a real name), it just keeps getting better. It’s also really great to see more open source projects coming on the scene – MassTransit (now a year old), and Rhino Service Bus (young, but in production with NH-Prof).

The mutual stealing cross-pollination is increasing the pace all around.

I intend to take up TopShelf (which came out of MassTransit) as the generic service host. That combined with the robust distributor hosting model will make “grid-friendly” nServiceBus endpoints much easier. Ayende’s posts about automatically creating queues and other “first-time-developer” features have really worked to decrease the nServiceBus learning curve – and it shows.

Documentation is a notorious problem in open-source projects and nServiceBus hasn’t escaped it. API and internal documentation is only now getting close to 100% and the samples now give developers a good start on using it. Those developers looking at swapping out certain bits of functionality have a harder time, but that’s slowly improving as well. The discussion group is the best place to get those hard-to-find answers now with a community over 200 strong.

The difficulty developers have in adopting nServiceBus is giving up “request/response” thinking. This requires an adjustment to a system’s architecture and is what most of this blog has been about. Conversely, if you have been following this blog and this thinking resonates with you, you’ll find it very simple and straight-forward. The overview page on NServiceBus.com also gives a good description of messaging basics – stuff that I’ve kind of glossed over on this blog so far.

Go on then. Take it for a spin. Write a review. Sam Gentile had this to say about v1.8:

“The bottom line is: I like what I see. Although it’s a framework, not an ESB product like Neuron, it’s a powerful framework that takes the right approach on SOA and enforces a paradigm of reliable one-way, *non-blocking* calls.”

Can’t wait to hear the response to 1.9.

Factory methods on repositories

In the podcast, Rob describes an ecommerce application with orders, customers, products – all the stuff you’ve come to expect. In his pre-DDD design, the order class had multiple constructors representing different rules. Feeling the pain in testing and maintainability, Rob looked to use DDD principles. What he did to get rid of these constructors was to make use of the repository by creating methods for the various cases, like OrderRepository.CreateOrderForGovernmentCustomer(/*data*/); .

While this is better than the multiple constructors, it still has a way to go. Analyzing what’s going on here we understand that the way the order is created is dependent upon who the user is. The rules dictating terms of payment are probably different for government customers. Not only that but we know that the order created needs to be connected to that user.

Aggregate Roots & Polymorphism

For all these reasons, it looks like user, or customer, is our aggregate root. Thus, rather than our service layer calling the above method on the repository, we first get the user object by id, then create the order like so:

IUser u = session.Get<IUser>(IdOfUserLoggedIn);
u.CreateOrder(/* data */);

This way, our service layer doesn’t need to perform all sorts of business logic (if the user is a gov’t user, do this, a corporate customer, do that, etc). All of that gets encapsulated by the domain. By leveraging some polymorphism, the session will return an instance of the correct class when we ask it for a user by id. Thus, logic relating to how gov’t users create orders is encapsulated in the GovernmentUser class.

I’ve found that this pattern of having polymorphic aggregate roots is very useful and broadly applicable.

Bounded Contexts

Further into the podcast, Rob talks about how separating his system into 2 bounded contexts simplified the code greatly. The understanding that accepting an order and fulfilling an order require different logic, different data, and thus, different domain model objects is DDD at its finest.

However, when talking about the total cost of the order, it wasn’t clear what was responsible for that. From a pure programming perspective, we might think that Total was simply a property on Order or, at most, a GetTotal() method. Yet by looking at what is involved in calculating the total of an order, a different picture emerges:

The total cost of an order obviously includes all relevant taxes. We need to take into account state and federal taxes, tax-free items at each level, etc. There’s a fair amount of logic here. Once we start to take into account promotions like “buy one, get one free”, things get even more involved. There are also cases where refunds are applied within the same order that impact the total and tax (no tax on refunds). Finally, when we include shipping charges, tax on shipping, and other rules between all of the above, it’s clear that if we put all of this in a single method, we’ve got ourselves a big bloated sack of … well, you get the picture.

Separating all of this logic into different bounded contexts makes sense.

That is, until you think about how you’re going to take these and stitch out of them a single result shown to the user. This is the advanced side of DDD and ties into SOA, so I’ll leave it for a different post.

In Closing

Working with DDD provides a great deal of value by tying our code much more closely to business concepts and encapsulating business rules in the domain model.

Starting down the DDD path is intuitive and the code that results (like u.CreateOrder) is very understandable. Yet, as more DDD principles like Bounded Contexts are put to use, developers often find themselves in unfamiliar, less-intuitive waters. This is to be expected.

Some developers (and vendors) look at DDD as nothing more than the domain model and repository patterns. The truth is that they’re just the beginning.

I hope that this post has given those of you just starting down the DDD path some feeling for how deep the rabbit hole goes, and I assure you that there are patterns in place to answer all your questions. While those beginning DDD often say that it gives names to things they’ve always been doing, or always wanted to do, I can assure you that the further down you go, that is less and less the case.

Be ready to have some basic architectural assumptions shaken