An event-driven architecture allows systems to collaborate via events that are published in response to something significant happening in the business.

This makes it easier for distributed systems to collaborate as data is moved around in a format that is closely aligned to the way the business actually works. Instead of binding systems together with feeds or batch exports, they are able to respond to in near real time to any activity they are interested in.

The prize here is in looser coupling with its related benefits of scalability, robustness and resilience. Realising this does rather depend on getting the design of your events right though, as poorly designed events can actually increase coupling.

What is an event?

An event is a significant change in state that has happened in the past.

The perspective is important here. In this context an event is a high-level business activity such as placing an order or approving a new customer. The abstracted nature of events is significant as they should not express any lower-level implementation detail. A well-designed event should not expose any one system's internal implementation detail.

When events are published a consuming system can choose how they wish to respond to the event. They don't have to reply to the system that issued the event and are free to ignore it. This is a more flexible approach to modelling activities than sequential workflows or batch updates. It allows systems to respond freely and pro-actively to activities and changing conditions in something close to real time.

The importance of autonomy and encapsulation

Each event should contain just enough information to allow another system to respond without requiring any external information or context. In effect, an event represents a discrete unit of work or transaction that communicates the entire change in state so it can be processed autonomously.

Autonomy and encapsulation are very important features of event design. If an event does not encapsulate the full details of the state change then applications will have to wait for additional events or query state from other systems before they can process an event. This is not consistent with the basic aim of loose coupling.

It's common to see events confused with entities. Events are an abstraction, not an expression of a relational data model. For example, you might have an event that is sent when a new product is created and a separate event for when a new product category has been added. In this case, an application won't be able to process new products until they have been told about the new category.

Given the retry semantics available on messaging implementations it might be tempting to delay processing the product events until a related category has been created.  This undermines the notion of fully encapsulated, autonomous messages and you'll find events getting “stuck” waiting for related data that never arrives. An application should have everything it needs to process a new product without having to wait around.

Does autonomy mean large events and frequent updates?

The implication of autonomy and encapsulation is that events will tend to be large. After all, they have to contain any data associated with an event and this tends to give rise to bloat.

This doesn't necessarily have to be the case, though you do have to be pretty strict over what constitutes “just enough” data.  Bear in mind that you are communicating events here, not trying to replicate databases. We may need to know that a product has a category, but we don't need to understand whether this is implemented as a separate entity or table.

Another problem concerns events that convey numerous, small data changes. An event might be sent whenever a product is updated, though this is triggered for any tiny change to a property. This tends to place an unreasonable burden on applications processing events.

Ultimately, it's down to an event sender to be reasonably responsible in the frequency of events that it sends. If minor changes are coming thick and fast then it might be reasonable to aggregate these changes and send a regular summary event that describes the new state. Failing that, it's down to the event consumer to manage the throughput of events by implementing some form of appropriate buffering or aggregation.

Ordering shouldn't matter

Another common mistake is to confuse events with feeds. An event should be a discrete item that does not rely on previous events to describe current state.

Given that events are time-based it can be logical to assume that they should be processed in a particular order. The problem here is that networks are inherently unreliable so it is very difficult to guarantee the order in which events are delivered. It is even more difficult to guarantee the order in which they are processed.

The easiest way of solving the ordering problem is to design events so the order in which they are processed does not matter. Bear in mind that if you rely on message ordering then you are effectively coupling your applications together in a temporal, or time-based, sense.

If some kind of ordering is unavoidable then a common solution to ordering involves adding a sequence number to the event data. A system that is receives events remembers the last sequence number it processed and rejects any events with an earlier sequence number. This kind of mechanism provides a simple way of expressing “happened before” ordering without sacrificing the all-important autonomy and encapsulation required by events.

Filed under Architecture, Integration, Messaging, Distributed Applications and Event-driven systems.