API management and the return of the enterprise service bus
No self-respecting integration platform is complete without an API management story these days. Instead of connecting everything directly to an enterprise service bus using messaging, are we now being encourage to connect to a similar centralised hub using REST?
Large API ecosystems often use a “gatekeeper” in the form of an API gateway that decouples public end-points from the code that executes requests. This is first and foremost a security measure that allows you to validate and authenticate requests in an environment with limited privileges.
API management conflates this narrow and well-defined role with a host of other concerns. It encourages the adoption of a single platform to control the entire API life cycle, taking in design, publishing, connectivity, data transformation, documentation and even developer on-boarding. APIs are developed and hosted within a management platform rather than through individual development projects.
Much of the early running in API management was made by start-ups such as Mashery and Apigee. Many of these have been snapped up by established vendors, with CA Technologies buying Layer 7, Tibco acquiring Mashery and RedHat picking up 3Scale. Perhaps more tellingly, enterprise integration vendors such as Mulesoft, IBM and WSO2 have built API management into their existing solutions.
Why use a platform to publish your APIs?
The rationale for API management is that this makes it easier to manage generic aspects of the API life-cycle. This may be tempting if you have data stores of various hues scattered around the organisation that you want to expose as a coherent and consistent API. A platform-based approach promises the chance to publish this data without drowning in costly API development projects.
This is less about creating carefully manicured, hypermedia-based APIs that are closely aligned to user experiences. It is more about using a pragmatic style of REST as a medium for enterprise integration.
Consistency of implementation can be real problem with large-scale distributed API development. REST does not provide a strict standard so It can be very difficult to get a diverse set of agile teams conforming to a consistent style of API. You'd be surprised how much time can be spent debating the nuances of RESTful API design.
A consistent API doesn't happen by accident and needs to be curated. More often than not, this requires that you are drawn into time-consuming governance processes or appointing an “API Czar” with the thankless task of chasing teams into compliance.
A centralised platform could make this easier. It can provide a thin application layer that exposes a diverse landscape of data sources to the outside world as a coherent and consistent API. Generic features such as pagination, filtering and sorting can be asserted in this layer rather than being hand-tooled from scratch by every development team.
Added to this is the sheer volume of ancillary services involved in publishing APIs effectively. They need to be secured, preferably using a widely-used protocol such as OAuth 2.0. They need to validate incoming requests and shape traffic to guard against certain forms of attack. Once APIs are published you need to engage with developers by providing accurate, helpful documentation and tools that allow self-service access management. On top of that you will want to track usage, possibly even feeding into some kind of metering or billing.
You can implement all of this through individual “best of breed” solutions, though you will inevitably run into problems of integration, compatibility and complexity. This is the problem that enterprise API management promises to solve.
The centralised logic dump
That's the promise anyway. The reality is that it's all too easy for an API management platform to become a store of poorly maintained business logic. In this sense, API management starts to resemble an enterprise service bus – i.e. a large monster lurking in the centre of an architecture that devours any integration logic.
These style of enterprise platform tends to be maintained by a centralised team who are the only people in the organisation who have any expertise in the technology. This is not the best place to maintain business logic that ideally should be looked after by a development team with some specific domain knowledge.
Attempting to address this by propagating platform knowledge to individual teams is onerous to the point of being unrealistic. Even if you manage to embed knowledge in more than one team the platforms themselves rarely provide facilities for multi-team development beyond isolating them in separate instances.
The promise of “automated publishing” and “point and click” transformation never quite holds up to real world scenarios. A point and click interface cannot deliver much beyond very simple data mapping scenarios. Implementing anything more complex such as decision logic or advanced formatting inevitably involves scattering scripts around different parts of the UI.
Development platforms without development tooling
The reality is that API management platforms are development environments that usually have to implement complex processing logic. The problem is that they don't often provide the kind of development infrastructure and tooling you would normally expect.
The story here can vary enormously between different vendors. Mulesoft's run-time applications can support a build, test and deployment pipeline typically based on Mavern, JUnit\MUnit and Jenkins. This allows you to establish an automated development automation for the underlying processing logic but not the policy configuration required to publish APIs.
Other platforms provide almost nothing in the way of source control, unit testing or build and deployment automation. This can force you into manual test and deployment processes where code promotion is managed by manually exporting objects between tenants. Unit testing of the logic is impossible and you are stuck trying to manage unwieldy integration test environments.
A platform for life
Like many enterprise technologies, API management involves a vendor lock-in that is almost impossible to escape from. There's no gradual refactoring strategy away from an API management platform – you're there for life.
API development is too fluid a thing to abstract away into a management platform. Your APIs should evolve over time in response to changing requirements. This will require development teams who are familiar with the domain working on building, testing and maintaining features. A centralised delivery platform with such limited tooling is inevitably going to be an impediment to this.