Establishing a new architecture practice in agile organisations

Most modern, agile-orientated organisations want to give engineering teams sufficient autonomy to make their own decisions and design their own solutions. This makes sense, as teams are normally best placed to understand the domain that they are working in.

This approach can encourage a faster flow of delivery where teams are not hamstrung by dependencies or bureaucracy. It can be less effective when it comes to building larger, more complex solutions that require technical co-ordination between multiple teams. There can also be duplicated effort and suboptimal technology choices as teams focus on immediate delivery as opposed to longer term, strategic concerns.

As an organisation grows, a "strategy gap" can open between engineering silos and the wider technology and business context. Some organisations seek to establish an architecture practice to plug this gap and find a more sensible balance between team autonomy and the need for economies of scale.

This can often give rise to a poorly articulated remit for architecture. There may be talk of the need for "guard rails" and even some "governance with a small g". Everybody will tend to be on board with the basic idea and the need to do something. Things tend to get murkier when you try and identify what these "guard rails" really mean in practice. That's when you tend to come up against resistance to change and an embedded desire for autonomy and ownership.

When you first try to establish an architecture practice in virgin territory, you are likely to find that you have a shaky mandate. People aren't going to do what you say just because you have a newly created and exotic-sounding job title. How can you be effective in organisations where people are allowed to say "no"? Ultimately, this becomes an exercise in persuasion, pragmatism, and patience.

Be relevant

No architect would ever self-identify as an "ivory tower" architect, yet this is an accusation that is often levelled at architecture teams. Sometimes it sticks. It can be easy to dismiss architects as "post technical" practitioners who have long since become divorced from the reality of technology delivery.

It can also be easy for architects to make simplistic interventions based on an imperfect understanding of the organisation's problems. Architecture should be a collaborative discipline, not one that is set apart from engineering teams. They should be enabling engineering teams rather than imposing on them. This requires that architects build a close understand the problems that engineers are trying to solve.

This more collaborative stance can make the work of architecture more relevant to technology teams, helping to establish architecture as a "force for good" in the eyes of engineers.

Identify the natural leaders – and bring them in

There will already be people scattered throughout the organisation who are doing the work of architecture. Some of them might have the word "architect" somewhere in their job title, though they could also be engineers, managers, or even product owners. These are the people you need to identify and influence.

Natural leaders tend to emerge in development organisations, regardless of their formal role. In their book "Lean Software Development", the Poppendiecks referred to this role as the master developer. This idea is based on a study led by Bill Curtis where they found that for most large systems a single or small team of exceptional designers tend to emerge that take responsibility for the system design.

The implication is that this authority emerges naturally through superior knowledge rather than being asserted by hierarchy. This is something that you should harness as it can be a key means of establishing credibility and building momentum.

Understand what autonomy really means

The idea of team autonomy is a little over-played. Engineering teams want to be able to build solutions without constraints, and this is often interpreted as autonomy. However, too much autonomy can itself be a constraint. If teams become too isolated their productivity begins to suffer. 

You can leverage this to gain engagement by helping to solve problems that are just too big for individual teams to address on their own. You can identify those shared capabilities that will make it easier for teams to build solutions. These can be provided on a self-service basis, encouraging a culture of freedom and responsibility while reconciling it with organisational goals around economies of scale. You can also forge cross-team agreements to standardise collaboration and reduce the barriers between teams.

If autonomy is defined by teams being in control of their own delivery, this is better served by removing constraints than ensuring isolation. This is where architecture can add value and help to improve autonomy by enabling delivery.

You don't need everybody to begin with

You can't please all the people all the time. It's unrealistic to expect that everybody will be immediately keen to participate in the bold new experiment of architecture. It can be exhausting to try and win people over who are just not interested.

Focus on fertile ground to begin with by working with those groups who are more immediately amenable. This will give you an opportunity to establish some momentum, demonstrate competence and grow credibility.  Over time you may find that the more reluctant groups start to become more receptive. They may even come to you once it becomes clear that you've been instrumental in solving some problems.

Know your history

Context is everything and it's important to get an understanding of what hasn't worked in the past.

Many organisations have "poisoned terms" that are forever associated with a failed past initiative. The merest suggestion of these terms can inspire much eye-rolling and protests that everybody has been there and done that already. Whatever the reasons for this failure there is likely to be institutional resistance to trying again.

It's important to acknowledge past failure and indicate how you'll be learning from these mistakes. This can be an important signal that you're willing to respect the accumulated knowledge of the organisation. Another way around this can be to invent new language to reintroduce concepts into the organisation. Fresh abstractions can help people to move on and not be bound by their past experiences.

Knowing when to spend capital

Sometimes you need to show a bit of steel. It's really a matter of knowing when it's appropriate to.

Architecture inevitably involves getting people to change their behaviours, especially if you try to drive some measure of standardisation. There will be systems or processes that need to change despite the people who have been dedicated to them for years. There will be decisions where no matter what you choose, you will have to upset somebody.

You shouldn't shy away from potential conflict but choose your moments carefully. You should think carefully about the hills that you are prepared to die on and make a show of your permissiveness in other areas. Sometimes a strategic feint can buy time while you work out how to build support. This might sound Machiavellian, but that's the reality of working in large organisations.

Don't destroy value

Although you want to drive change, you need to be aware that some arcane solutions and ways of working might be in place for very good reasons. Sometimes the status quo can represent years of accumulated knowledge and expertise.

There are occasions where "raised by wolves" syndrome has set in, and the organisation doesn't know any better. It may lack the experience to solve problems any other way. The important thing is to retain an open mind and not to disrupt for the sake of it.

Consider a blended approach to frameworks

Architectural frameworks such as TOGAF and Archimate require considerable commitment to implement in full and don't always stand up to close scrutiny. That's not to say they can't add a lot of value to an organisation with no prior experience of organised architecture.

Distilled within these frameworks are years of wisdom, hard won experience, and techniques that can add genuine value. It's worth getting acquainted with the details of architectural frameworks as you can select and adapt them to suit your environment.

For example, drawing up a set of architectural principles can help to guide future decision making by establishing what you value in software architecture. It's also a useful early collaborative exercise that's straight out of the TOGAF playbook. Archimate may be overly complex, especially to newcomers, but the core concepts can still provide a useful means of thinking about how strategies, processes, and systems interact. A TOGAF-style "architecture board" may seem like a top-down "council of elders", but you can adapt the idea to establish a community of senior technologists and drive some much-needed consensus.

TOGAF authors The Open Group have recognised the need for a more agile-orientated approach and have attempted to articulate this through their Open Agile Architecture Standard. This doesn't break any new ground, but it is a reasonable summary of some of the more progressive ideas in play, from intentional architecture to evolutionary design.

Be focused

The aim of architecture is not to establish a broad governance framework or curate an impressive set of diagrams. It should be there to solve real business problems. These will present themselves early on if you take the time to immerse yourself in the business.

The organisation may be struggling to bring anything new to market, it may be bleeding money through expensive infrastructure, the applications may be struggling to scale, or be chronically unreliable. If you can really focus on what these over-riding business-related problems are then it can really help to focus your attention on the technology issues that matter.