Balancing autonomy, growth and culture in engineering teams – part one
Engineering teams are often faced with the pressures of continuous improvement to ensure long-term software quality, delivering more, while also trying to balance efficiency, team growth and culture and best practices.
I enjoy systems, thinking about the bigger picture, understanding the problems that systems have, helping to think about how we might solve some of those problems. But how teams grow and how you deliver great outcomes while building a great culture is something I'm passionate about.
These are things that make up successful engineering teams and companies that we don't often talk about enough. We tend to focus a lot on the technology and not enough on the people and culture side.
That’s why it’s incumbent on engineering leaders to ensure that the right people are making the right decisions. But more importantly they feel empowered to make the decisions themselves, when it's appropriate, and appreciate when they need to collaborate at other times.
The engineering team sees how I behave, so making sure I’m communicating clearly and safely and that people feel heard influences how others behave. It’s about holding ourselves to a high standard. How we make decisions has a significant effect on culture.
In this article, I’ll outline my thoughts on how to achieve that, including how we instil and nurture a great culture in which people can excel in and create a psychologically safe environment for team members.
I believe engineering teams should be as autonomous as possible. Motivational research shows that autonomy and mastery of purpose drive us in the knowledge worker environment. However, as a bank, we must balance the desire for autonomy with the needs of a regulated business.
When I joined ClearBank nearly three years ago, we had a centralised function, called architecture review, where only senior folk came together to discuss architectural decisions. ClearBank was smaller then, but it wasn't an approach that would scale well. Some of the people in those reviews weren’t close enough to some of the problems that were being discussed and making decisions on to really know the consequences of those decisions.
Recognising that we needed processes that scaled as the bank did, we’ve shifted to what Andrew Harmel-Law's described as the Architecture Advice Process. That uses architecture decision records (ADRs) as the mechanism for storing decisions that are mutable. We're quite strict about how well these are written because when you think about a regulator's perspective, yes, the architecture review led the purpose, we have the minutes and we could record the decisions, but they missed a lot of the nuance and the detail that the ADRs now capture.
Another consideration is that people can feel like they cannot make decisions. In a smaller firm, this can be frustrating but doesn’t adversely affect delivery, when you grow, it can. We identified that people didn’t feel they could make decisions, continuously looking upwards to somebody else to make decisions for them.
So, we introduced something called decision scopes, which were very simple. What we ask everybody to ask themselves is whether a decision just impacts a team, a domain or is enterprise level:
- Team scope: If the decision only affects your team, it is your decision to make, discuss within your team and move forward with it. However, we also need it to be captured in an ADR so that it's recorded for audit purposes.
- Domain scope: While there may be a couple of people in a particular domain, they may be changing something to do with, for example, the way we process payments that may impact multiple teams. In this scenario it requires a broader discussion with the people that it's going to impact and, once again, is captured in an ADR.
- Enterprise scope: For the largest, cross-team and domain work delivered through a new centralised forum, the Architecture Advisory Forum. That's where we create a quorum of everybody that we need to have to approve these decisions, for example the principal engineers, security teams, data teams and so on.
The Architecture Advisory Forum serves two purposes. It’s open which means everybody can have an input into these enterprise decisions because it's going to impact them. And I truly believe that that means that these decisions have a bigger impact because people feel like they've been consulted, and they’ve contributed.
It's purposely called ‘architecture advisory’. People can bring items that they would like opinions on, should they want to. Sometimes that is an opinion from the forum, but sometimes it's just we've got that quorum of people, and you can find the right people that you need to speak to, to move what you're trying to achieve forward. So, it also creates an open culture.
The final aspect of these forums is anyone can attend. On a personal level, I was really pleased when one our directors of engineering encouraged his team’s associate to attend and present. I’m proud of our collaboration with Code First Girls and to see one of that cohort feel they had a safe and trusted environment to present an enterprise decision that they wanted to be part of just 10 months into their technology career.
This is not specific to engineering by any means, but an ongoing concern is how to balance architectural advice and overburdensome bureaucracy. As ClearBank has grown it’s something we’ve had to manage.
There are examples where there are points of contention and they can’t be solved through the forum because not everyone agrees with the proposed solutions. And because, in one specific case, for the decision to make an impact we needed full buy-in. As a group, the principal engineers take a slightly different approach, where we look to have more influence in one particular area in teams whose work we think will have the most impact.
In other cases, there are decisions that we need to make. We're a bank so sometimes it relates to regulatory changes that we just need to do. So, while we don’t want to dictate things, in some instances, there is no discussion required.
It’s still a challenge to this way of working but, for example, these types of changes are also discussed in the forum and we’re clear that we need to do this and we’re sharing with everyone for information. I think it speaks volumes about the culture we have at ClearBank that everyone appreciates it and when we’ve had these situations, people still input and have suggested improvements that benefit the task and the outcome.
As ClearBank grew, it faced the challenge of maintaining its innovative culture while integrating more structured processes to manage expanding operations and, of course, ensure regulatory compliance. Within boundaries of accountability and responsibility, teams were given space to evolve their own areas, innovate a little, experiment, and continuously improve, to remain innovative.
In my next post, I’ll examine the related topics of growing a team while driving efficiency, nurturing engineering talent and maintaining culture during periods of rapid growth.
Michael Gray is a Principal Engineer at ClearBank. He has a passion for software and systems and particularly Domain-Driven Design, engineering culture and leadership, creating the right environment for engineering teams to thrive. Prior to ClearBank, Michael led the Technical Architecture team at Vanquis, providing context and direction to the engineering teams and implementing enhanced governance to ensure it met regulatory obligations while ensuring engineering teams were empowered.