Serverless: why microfunctions > microservices

2017-10-23T21:00:00Z

Mon, Oct 23, 2017

Serverless: why microfunctions > microservices

This post follows on from a post I wrote a couple of years back called Why Service Architectures Should Focus on Workflows. In that post I attempted to describe the fragility of microservice systems that were simply translating object-oriented patterns to the new paradigm. These systems were migrating domain models and their interactions from in-memory objects to separate networked processes. They were replacing in-process function calls with cross-network rpc calls, adding latency and infrastructure complexity. The goal was scalability and flexibility but, I argued, the entity modelling approach introduced new failure modes. I suggested a solution:

Instead of carving up the domain by entity, focus on the workflows.

If I was writing that post today I would say “focus on the functions” because the future is serverless functions, not microservices. Or, more brashly:

microfunctions > microservices

The industry has moved apace in the last 3 years with a focus on solving the infrastructure challenges caused by running hundreds of intercommunicating microservices. Containers have matured and become the de-facto standard for the unit of microservice deployment with management platforms such as Kubernetes to orchestrate them and frameworks like GRPC for robust interservice communication.

The focus still tends to be on interacting entities though: when placing an order the “order service” talks to the “customer service” which reserves items by talking to the “stock service” and the “payment service” which talks to the “payment gateway” after first checking with the “fraud service”. When the order needs to be shipped the “shipping service” asks the “order service” for orders that need to be fulfilled and tells the “stock service” to remove the reservation, then to the “customer service” to locate the customer etc. All of these services are likely to be persisting state in various backend databases.

Microservices are organized as vertical slices through the domain:

The same problems still exist: if the customer service is overwhelmed by the shipping service then the order service can’t take new orders. The container manager will, of course, scale up the number of customer service instances and register them with the appropriate load balancers, discovery servers, monitoring and logging. However, it cannot easily cope with a critical failure in this service, perhaps caused by a repeated bad request that panics the service and prevents multiple dependent services from operating properly. Failures and slowdowns in response times are handled within client services through backoff strategies, circuit breakers and retries. The system as a whole increases in complexity but remains fragile.

By contrast, in a serverless architecture, the emphasis is on the functions of the system. For this reason serverless is sometimes called FaaS – Functions as a Service. Systems are decomposed into functions that encapsulate a single task in a single process. Instead of each request involving the orchestration of multiple services the request uses an instance of the appropriate function. Rather than the domain model being exploded into separate networked processes its entities are provided in code libraries compiled into the function at build time. Calls to entity methods are in-process so don’t pay the network latency or reliability taxes.

In this paradigm the “place order” function simply calls methods on customer, stock and payment objects, which may then interact with the various backend databases directly. Instead of a dozen networked RPC calls, the function relies on 2-3 database calls. Additionally, if a function is particularly hot it can be scaled directly without affecting the operation of other functions and, crucially, it can fail completely without taking down other functions. (Modulo the reliability of databases which affect both styles of architecture identically.)

Microfunctions are horizontal slices through the domain:

The advantages I wrote last time still hold up when translated to serverless terminology:

Deploying or retiring a function becomes as simple as switching it on or off which leads to greater freedom to experiment.
Scaling a function is limited to scaling a single type of process horizontally and the costs of doing this can be cleanly evaluated.
The system as a whole becomes much more robust. When a function encounters problems it is limited to a single workflow such as issuing invoices. Other functions can continue to operate independently.
Latency, bandwidth use and reliability are all improved because there are fewer network calls. The function still relies on the database and other support systems such as lock servers, but most of the data flow is controlled in-process.
The unit of testing and deployment is a single function which reduces the complexity and cost of maintenance.

One major advantage that I missed is the potential for extreme cost savings through scale, particularly the scale attainable by running on public shared infrastructure. Since all the variability of microservice deployment configurations is abstracted away into a simple request/response interface the microfunctions can be run as isolated shared-nothing processes, billed only for the resources they use in their short lifetime. Anyone who has costed for redundant microservices simply for basic resilience will appreciate the potential here.

Although there are number of cloud providers in this space (AWS Lambda, Google Cloud Functions, Azure Functions) serverless is still an emerging paradigm with the problems that come with immaturity. Adrian Coyler recently summarized an excellent paper and presentation dealing with the challenges of building serverless systems which highlights many of these, including the lack of service level agreements and loose performance guarantees. It seems almost certain though that these will improve as the space matures and overtakes the microservice paradigm.

Gorecipes: Fin

2016-03-30T18:00:00Z

Gorecipes: Fin

Wed, Mar 30, 2016

Gorecipes: Fin

How hard can it be to write a book of small, useful programming recipes? Too hard as it turns out.

About a year ago I was prompted by APress publishing to submit a proposal for a book to be called Go Recipes. Knowing nothing about book publishing this surprised me: I’d always assumed that authors pitched ideas to publishers or publishers pitched ideas to authors. It never occurred to me that a publisher would ask an author to devise an idea to pitch out of the blue. I surmised that they must have spotted my Go Cookbook repository on Github. After a couple of weeks of hesitation I put together the table of contents and blurb for the kind of book I would want to use, and one I hoped I could write. The publisher liked what I put together and asked me to write it. After another couple of weeks of hesitation, knowing it would be a massive time commitment. I should have hesitated a bit longer because I completely underestimated that investment of time.

I had plenty of ideas and within a few weeks I had delivered the first chapter and then a few weeks later I managed a second. Then my enthusiasm and energy began to wane: I was busy with other things in my life; I fretted about writing compelling and relevant recipes; I was writing about things I wasn’t that interested in; I worried about repeating myself over and over when explaining things in Go. Most of all I was anxious that some of the things I was writing might not be complete enough or not perceived as correct by other Go programmers.

If you were waiting for that book then I’m sorry to disappoint you: it’s not going to be completed. I decided that it was going to take too long and APress can’t sit around forever waiting for this awesome book to emerge. I don’t know if they’ll take that idea up with someone else. Hopefully they will because it would be a great resource for new Go programmers.

All is not doom and gloom though: I have taken the best and most complete recipes I came up with and added them to the Go Cookbook repository. These are now in the public domain, free for anyone to use for any purpose. Go wild.

Another Blog Refresh

2015-02-22T10:04:00Z

Another Blog Refresh

Sun, Feb 22, 2015

Another Blog Refresh

It’s time for another blog refresh, this time back to a static site after a few years being hosted by Wordpress.com. Once again I’m convinced by Aaron’s argument that baking is better than frying. It’s not about performance, it’s about simplicity and control.

While I liked the convenience of Wordpress.com, it never really felt like a place I could tailor to my own requirements. I thought having a nice web UI and mobile apps to edit posts would encourage me to post more. It actually made no difference whatsoever. Whatever holds me back from blogging isn’t related to the editing UI.

For this move I looked at various static site generators such as Jekyll, Hyde and Hugo but I settled on a mimimal one: gostatic. My reasoning (which I admit may not be entirely justified) is that feature-led software gets updated at a much higher rate than I post to my blog. When I come to post, invariably something important has changed in the core software or in a dependency and so I’ll need to upgrade or fix that before being able to publish. I find this particularly true of larger systems in dynamic languages like Ruby or PHP.

This time around I have a single binary (gostatic) to generate the site with no dependencies. It’s deliberately feature-poor so I don’t rely on things that may be changed or deprecated some time in the future and I have a script that does the rebuild and can sync to whatever laptop I’ll be using in the coming years. It’s documented for a future me.

A few technical notes:

Posts are written in markdown and are compatible with all the static site generators I mentioned above
This move is partly motivated by moving to a new web server. I’m going to be using nginx and serving either static files or fronting Go services. This is the first time I’ve had a server that isn’t running Apache+PHP.
Hopefully the atom feed works ok – it looks ok, but there’s almost certainly some weird software out there that will break on it.
There will be broken links, but already I have fixed hundreds of bad internal links by being able to grep over all the posts locally.
Formatting will be weird in places since the posts were exported via Wordpress’s XML export. I’ll get to tidying up the individual posts as an ongoing job.
There are no comments. I have the comments as part of the Wordpress export, and I’m planning to take a look at how to incorporate them into the blog archives. However I’m not planning on adding commenting to the blog. Thank you to all of you who have commented on my posts in the past, I have enjoyed reading them. But… it’s time to admit that commenting is a broken form of communication.

For a contrary views on baked vs fried and blog commenting, see my post on moving from Moveable Type to Wordpress back in 2004 or my post on moving from a dynamic system to Moveable Type, or even my post on moving to a hosted blog on Posterous ;)

For early blog archaeology see my post on early versions of Internet Alchemy

Why Service Architectures Should Focus on Workflows

2014-03-31T23:27:12Z

Why Service Architectures Should Focus on Workflows

Mon, Mar 31, 2014

Why Service Architectures Should Focus on Workflows

Since this week saw the 15th anniversary of this blog I thought I’d celebrate by actually writing a post, and a technical one at that. Before we get to it, I have to recognise that I’m blogging in the 21st century now, so here’s a meme:

Recently I’ve been musing over the way we tend to architect distributed systems. Almost certainly none of what follows is original, but like most of my writing its designed to help me understand and codify a set of thoughts that have been buzzing around for a while.

If we practise domain driven design then it’s important to remember that the domain model is not the system. Rather, its a tool to help us understand our system better, a map of the territory if you like. It helps us navigate and use our system correctly but its insufficient by itself. Its called a model because it makes simplifying assumptions about the world. It tells us how the world might work if we didn’t have to worry about things that are hard to control like latency, bandwidth, communication failures, resource limits, concurrency and trust.

As a whole, the domain model encapsulates the constraints and business rules that apply to our system. We represent real world entities by objects that encapsulate state and manipulate it via methods. A well normalised system will ensure that objects have a single responsibility with the minimal interface required to fulfil it. Interacting with the system quite often requires the orchestration of many objects, each handling their own unique part of the work. Workflows such as taking orders, paying staff or shipping goods are accomplished by arranging the appropriate objects and invoking methods to achieve the final desired system state according to the constraints defined by the relationships between objects.

The first place we usually encounter the limitation of our domain model is when we start adding persistence. Aside from the well-known object/relational impedance mismatch we also need to introduce new abstractions to deal with the location of the backend database: batching up reads and writes for efficiency, adding transactions for integrity and connection management for reliability.

We are compensating for the deficiencies in our domain model but importantly we are not invalidating it. The constraints and rules still apply: customers have orders which have line items etc. Now, however, we are having to interface our simple model to some of the complexities of the real world. If the database was in the same process space as the domain model then we wouldn’t be as concerned about the impact of interacting with it. However most databases are physically remote over a fallible network link with huge latency compared to main memory and have multiple clients modifying data concurrently.

Another place we encounter limitations, and the main point of this post, is when we want to scale out our system to increase capacity. These days that implies we want to distribute the system over several interacting services that can be scaled as necessary. The temptation here is to simply follow the domain model constraints and distribute the system by entity type. Following this we may end up with services for customers, orders and products each encapsulating its data and exposing actions over it. They interact with the database independently to ensure data integrity and can be scaled out with additional instances as necessary as system load grows.

Placing an order might be orchestrated by the order service that gets the customer details from the customer service, product information from the product service and uses them to create the order. Along the way it might also verify whether the customer is good for credit or needs to pay immediately, check stock levels, lookup shipping costs and trigger activities in fulfilment and invoicing services.

On the surface this seems reasonable, after all these steps have to happen somewhere, but digging deeper we can expose more complexity. The customer service will probably need get data from a database, perhaps calling into an organisation service for corporate customers to get billing and shipping addresses. The product service also looks up data in the database but we only need basic information such as description and package quantities so getting a full product entity is likely to be overkill for our needs. Creating an order requires locking, persistence of line items and the order itself, and…. you get the idea: a lot is happening and it’s happening over the network.

This scale out approach carries many of the same problems we encounter with adding persistence to our domain model:

Latency – the time taken to traverse the network for each service call is at least two orders of magnitude larger than calling an in-process method. Add in disk seeks and it’s another order of magnitude slower. The domain model simplifies the world by assuming these calls are free but in reality they can compound up to crippling degrees of slowdown.
Bandwidth – retrieving a customer’s details or a product spec can result in a lot of data being transferred unnecessarily when we only need a name or quantity. Again the domain model assumes data is referenced in-place or that bandwidth is infinite. Introducing alternate service calls for summaries and subsets of the data can mitigate this problem but wasting bandwidth limits scalability of the system as a whole.
Reliability – we have to assume that every out of process request can fail unexpectedly. The possible causes are nearly infinite: the network link may be down, the server could be out of sockets, routing may be faulty or someone switched off the destination server. The domain model simplified the world by assuming every method call will succeed. A typical workaround for this is to introduce timeouts and automatic retries which can lead to pathological situations where every timeout cascades up into multiple retries at all levels tying up resources and limiting throughput.

In addition, the approach introduces some new problems:

Fragility – because the service model mirrors the domain model, services will participate in multiple of workflows and any failure will affect them all. For example a customer service is likely to be used during ordering, shipping, invoicing and payment. If the customer service is unavailable then all those workflows break at the same time. Not only that but each of these workflows depends on multiple services being available.
Scaling – because lots of services are involved in a single workflow, scaling capacity for one workflow means every service used has to be scaled as well and since the services may be invoked multiple times during a single interaction they have to be scaled faster than the invoking service. This adds unnecessary costs to scaling. If we want to take more payments then we need to scale out a bunch of services even if they are only supplying a single value in our workflow.
Hotspots – most domain models have a few central objects such as customers or companies that are involved in large numbers of workflows. When these are encapsulated as services they become hotspots and bottlenecks for performance, degrading multiple workflows simultaneously.
Evolvability – all systems evolve with business requirements but service architectures that follow the domain model too closely become difficult to evolve. A change to workflow such as adding a new required piece of information can lead to a cascade of interface changes between services to enable them to pass it through to dependent services. Conversely, removing a workflow means removing functionality from services and subsequently retesting and redeployment of large numbers of services. Given the fragility of the system, this can cause problems in unrelated workflows simply because one service is malfunctioning.

All of these problems can be mitigated with a different approach to service design. Instead of carving up the domain by entity, focus on the workflows.

In this approach each important function becomes a separate service that can be scaled independently and has minimal dependencies. Using the example earlier we would design a “place order” service that is responsible solely for placing an order. It performs the same tasks as before but instead of calling entity-based services it uses the domain model objects directly to carry out its work, connecting to the database as necessary to select just the information it requires to create the order.

Communication between workflows is performed asynchronously by sharing state in the database or publishing an event to a message queue. For example once the order has been placed, events are fired that will kick off invoicing and fulfilment workflows.

This brings a number of immediate advantages:

Deploying or retiring a workflow becomes as simple as switching a service on or off which leads to greater freedom to experiment.
Scaling a workflow is limited to scaling a single service horizontally and the costs of doing this can be cleanly evaluated.
The system as a whole becomes much more robust. When a service encounters problems it is limited to a single workflow such as issuing invoices. Other workflows can continue to operate independently.
Latency, bandwidth use and reliability are all improved because there are fewer network calls. The service still relies on the database and other support systems such as lock servers, but most of the data flow is controlled in-process.

The unit of testing and deployment is a single service which reduces the complexity and cost of maintenance.

This approach also matches the adage of small pieces loosely joined as evidenced in the UNIX philosophy. There we don’t find applications representing files, reports and users but small tools for cutting, sorting and creating these kinds of things.

It’s important to recognise the difference between the organisation of code and its deployment. We logically separate the objects we use to implement the domain model using SOLID practices that help us maintain, evolve and reason about our code. But we don’t need to be constrained in that way when it comes to deployment as a service. The service uses the domain model objects but it doesn’t need to own them – it just composes the objects it needs into the patterns required for its workflow. Data is still encapsulated by each domain object and their interactions are still well defined.

In this respect the domain model actually becomes more important because it defines the reusable business rules and logic that will be used to create services for each workflow.

So we should keep the domain model but be aware that it’s only a map to help us navigate our system and tell us how things should work under ideal conditions. As we interface that to the real world we need to introduce abstractions and aggregations tuned to the messy realities of networks, resource constraints and concurrency. To improve reliability, throughput and scaling economics we need to focus on composing domain objects into services that distinct workflows through our system and arrange our architecture to minimise dependencies between services.

Help me crowdfund my game Amberfell

2012-11-12T09:27:51Z

Help me crowdfund my game Amberfell

Mon, Nov 12, 2012

Help me crowdfund my game Amberfell

I'm excited to announce that I've launched a Kickstarter project to complete the development of Amberfell, the game I started writing a few months ago. I'm hoping to raise £20,000 so I can hire a graphic artist to make the game look amazing, spend some on promotion and marketing and the rest to fund me taking time out of contract work to finish the programming.

For those that haven't see it before on my blog, Amberfell is a resource collecting video game set in a steampunk alternate history. It has some vague similarities with Minecraft but may play a bit more like Warcraft. It'll support singleplayer against the computer or multiplayer against friends over the Internet. Each player takes on the role of a prospector seeking a substance called amberfell which not only is incredibly valuable but has amazing properties too. To gather it they need to build tools and machines to help them and hinder their opponents. This is set against a backdrop of a wilderness world populated with dangerous creatures and natural hazards. You can read more about my ideas for it on the kickstarter page.

You might ask why I am seeking to fund this through kickstarter and why now? The simple answer is that Kickstarter have just launched in the UK, literally just a couple of weeks ago. Earlier this year I looked into signing up with them but was stymied by their restriction to US bank accounts. I looked at a couple of other similar services, but Kickstarter has the biggest audience and numbers matter for crowdfunding.

Also, running crowdfunding project for a game of like this isn't really about raising the money, it's about gauging audience demand. I could continue to develop the game in my spare time and launch it in 12-18 months time but this way I get to find out if anyone is actually interested in the game much sooner. I'm hoping to get some feedback on my ideas: does it capture the imagination; does it sound like fun; would you pay for it and if not, what would you pay for?

If i don't get over 100 backers then it's going to be a clear signal that the game doesn't light any fires and I'd better rethink it. I might still continue with it just because it's a game I want to play myself, or I may radically shift it. For me, that's the most exciting part!

I'm tweeting about the game and the funding drive on amberfell's twitter account so follow me there to keep up to date.

Here's a quick video of the game in action so far:

I'd love to have your feedback on the game, even if it's something you would never buy yourself. I want to hear your opinions and thoughts on it. Please leave a comment here or head over to the Kickstarter page and tell me there. If you are also able to donate to the development fund then that would be wonderful - thank you! You can get a copy of the game as a reward or even get a likeness of you built into the game and playable by (hopefully) tens of thousands of people around the world! :)