Managing change and version control for service interfaces
Change is an inevitable and even desirable part of distributed software development. After all, it often happens in response to an improved understanding of the system and how it is being used. Managing the impact of that change is the difficult bit, particularly when dealing with the service interfaces that bind a platform together.
The problem with versioning
When developing a distributed platform you can’t expect to get your service abstractions right first time. They will need regular refactoring to take into account any new understanding gained through implementation experience. A design needs to be able to cope with this kind of change or development will grind to a halt.
Having more than one incompatible version of the same service can have a similar effect as forking the code base. The amount of code under management mushrooms and bug fixes have to be applied to all the supported versions.
If you keep different service versions in the same code base then adding new features can have unexpected consequences, but if you hold them separately the operational burden can become overwhelming. Whatever approach you take, the end result can be a fragile and complex set of services.
Despite this, some degree of versioning may be inevitable. In the case of public APIs you have no control over the way that consumers integrate with your API so any change in the contract is likely to have unexpected repercussions. You may appear to have more control over internal integrations, but is largely illusionary as it is difficult to get consuming services into line without an oppressive degree of governance.
A service can only dictate how consumers integrate with it through the design of its public API. Therefore, a clear versioning strategy is an essential part of any service design.
Strict versioning
A strict version strategy dictates that any change should always result in a new contract. The main advantage of this approach is that is provides control and certainty. You don’t have to worry about the impact of any changes as the burden of compliance is shifted onto participating components.
It’s this burden that makes the strict strategy difficult to enforce in practise as it undermines a system’s capacity for change. Do you really have to go through the pain of issuing a new version just to make a trivial update? A strict versioning policy can easily become a barrier to system improvement as functional changes that affect contracts are resisted.
Building in the possibility of change
If a change can augment or extend an existing method without affecting any existing integrations then it can be regarded as backwards compatible. A more flexible strategy for version control allows scope for these largely additive changes to a model, though any breaking changes will still require a new version.
This approach does provide for more flexibility but it does come with a health warning. The certainty of the strict strategy is lost as it is difficult to be sure that existing integrations will be unaffected without explicitly testing them for backwards compatibility. There may be some subtle changes to the semantics of the contract that do not come to light until they cause integration errors.
There is also the risk of contract bloat unless it is subjected to careful management. Adding a new method can be a “back door” way of changing an existing method so you may find your API definition evolves several ways of doing the same thing. Methods that develop numerous parameters and switches are further evidence of this kind of contract bloat.
Some designs take this one step further and add features intended to provide greater scope for accommodating new requirements. This kind of “forwards compatibility” includes techniques such as optional parameters and wildcard properties can be used to pass in currently unknown data without having to go through the pain of version change.
This approach may appear seductive but it can give rise to poorly-defined and coarse contracts. The vague interfaces can yield loosely typed and vague data that places a heavy validation burden onto client integrations. This approach will often descend into a complex mess of switches and magic strings that undermine usability.
Consumers have responsibilities too
The burden of managing change should be shared between a service and consumer. There is more a consuming application can do to ensure they are tolerant of change.
Postel’s law, often referred to as the Robustness principle, states that you should be liberal in what you accept and conservative in what you send. When applied to service design this implies a certain tolerance in reading services.
Many deserialization techniques are incredibly strict about the format of the response and will throw an error for any extra fields that it finds. These extra fields may just implement new functionality and may not be at all relevant to the current integration.
This kind of automatic deserialization couples services and consumers by forcing them both the use the same class structure. An alternative approach is to use a less strict approach for handling responses such as XPath queries where you just look for the information you need. Martin Fowler has suggested the notion of a “tolerant reader” as a pattern where a client will be as tolerant as possible when reading the output of a service.
Open\closed principal
At the heart of this problem is the challenge of evolving existing service contracts in response to changing requirements. A variant of the strict approach involves designing a model that never changes – any new functionality should be implemented by extending the model rather than making changes to the existing contract.
Bertrand Meyer’s Open\Closed principal suggests that software entities should be “open for extension, but closed for modification”. This means that any new functionality should be implemented by writing new code rather than adjusting existing code. When applied to contracts this implies that new functionality should be implemented through new classes and methods rather than by making changes to existing ones.
This approach offers the certainty of strict versioning while allowing more scope for change. A component will only have to update its integration if it wants to take advantage of the new functionality that has been added to the contract.
The risk is that the number of classes and methods starts to mushroom as duplicates are created to implement new features. A degree of common sense is required along with careful curating to ensure that contracts remain usable and readable. As ever, careful on-going management is vital for the long term re-usability of an API.