Many of the benefits associated with microservices depend on you being able to create autonomous components. You won’t realise much in the way of improved scale, resilience and flexibility unless you can develop, test and run your services in isolation.

An increasingly common sight with microservice architectures is the “death star” component diagram where services appear to talk to each other directly in an apparently haphazard way. This makes it difficult to understand the system and its interactions on any meaningful level - there’s just a lot of stuff that is connected.

This is the "distributed big ball of mud". Instead of autonomous services collaborating to deliver business processes you have a haphazard set of components locked together in a distributed monolith. It makes for a complex system as you are constantly stubbing out services and managing dependencies. It also makes for a fragile system as if any one service fails or starts misbehaving then the consequences can ripple out across the entire system.

To avoid the distributed big ball of mud you need to address two distinct problems. Firstly, you need to ensure that your services collaborate asynchronously, as opposed to binding them together via the request\response communication. Secondly, your service boundaries have to be drawn so that each service can fulfil its obligations independently.

The need for asynchronous collaboration

As services get smaller the burden of managing their interactions tends to grow. There are more of them for starters as the number of individual interactions can grow exponentially as you add new services. The burden also grows in a relative sense as each service also has to deal with the overheads associated with remote communication such as routing, latency, failure and retries. The smaller the service, the more time you spend dealing with communication as opposed to delivering business value.

In traditional SOA architectures, much of the complexity around service interaction was managed by an enterprise service bus. Every service would be wired up to the bus which would take care of the messy business of managing service collaboration. The problem is that over time these bus platforms tend to become a repository for stray application logic and become a hub of byzantine complexity.

In the microservice world this approach has given way to the notion of “smart endpoints and dumb pipes” where services collaborate over a simple transport that does little more than request routing. The collaboration logic sits within the services rather than being devolved to a centralised hub. In many cases this has meant adopting REST as it provides a simple means of exchanging data over a widely-supported and understood protocol.

The problem with REST is that is implements a request\response pattern that tends to couple services together in a temporal sense. REST requires that there is somebody else actively listening to a request. In most cases a service cannot continue processing until it receives a response from an external service.

This temporal coupling can create real problems that undermine many of the prospective benefits of microservices. You have to create an infrastructure of stubs to be able to test anything in isolation. Deployment has to take into account shifting and cascading dependencies. It’s easy for your architecture to become a tangled web of complex interactions rather than autonomous, collaborating services.

Events to the rescue

One solution is to drop the request\response collaboration in favour of asynchronous messaging based on events. An event describes a significant change in state, such as a new order being placed. In practical terms it is a message sent on a queue that can be picked up by any other service that might be interested.

This allows a much more loosely coupled form of collaboration based on a one-way exchange of data. A service does not need to care about whether anything else is available. It just sends events when it does something interesting and consumes events that it finds relevant.

There are downsides to this approach, the main one being a more complex transport. Event-based messaging is significantly harder to get right than REST. A reliable messaging implementation has to solve a number of problems such as deserialization, routing, transactions and retries. That’s before you’ve thought about wider concerns such as monitoring, audit and message versioning.

This complexity often leads people to adopt an integration framework such as nServiceBus or Apache Camel to handle this complexity. The scope of some of these frameworks can often creep towards the kind of heavyweight, centralised bus-style architecture that microservices are supposed to free us from. Some discipline is required here to ensure that messaging remains a lightweight collaboration fabric.

Getting the service boundaries right

Microservices are often described in terms of the single responsibility principle in that they should do “one thing really well”. If you want your services to be autonomous then this “one thing” might be quite large in practice.

If you aim to create small and focused services there is a risk that you may decompose things before you fully understand them. This can give rise to closely-related services that need to share data or have to be updated en masse in response to a change in requirements. The cost of change starts to escalate and the distributed big ball of mud is born.

In service development, autonomy tends to be more important than size. Instead of aiming for services of a particular size, you should be trying to draw clear service boundaries that are aligned to problem domains. This may give rise to services that don’t feel particularly “micro”, but at least they are loosely coupled. After all, it is much easier to break a large service apart than it is to unpick the tangled web of interactions you typically find in a distributed big ball of mud.

Filed under Microservices, REST and Distributed Applications.