7 Lessons Learned from Scaling an Engineering Organization

 7 Lessons Learned from Scaling an Engineering Organization

On paper, and from the outside, it all seems so simple. You just hire a bunch of good people, they self-organize, and you'll be good to produce software to your heart's desire. Unfortunately, things in life are rarely that simple. Here are the 7 things I've learned over the last 3 years building a team from 2 to 20 people.

1. Perfect is the enemy of good

Very few problems have a perfect solution. There is no perfect architecture, no perfect way of organizing in teams, no perfect recruitment strategy, no perfect way of organizing your Github repos... you understand where I'm going.

Instead of striving for perfection, you should aim for better. It's the axiom of the Lean movement, modified. "Build, measure, learn" can be translated into "decide, measure, learn." As long as you focus on improving one part, while keeping other areas from deteriorating, you will at some point reach solution that is pretty darn close to optimal. But part of that is making sure that your team feels empowered to strive for better, and knows that it is okay to make mistakes and not quite hit “perfect”.

The best way to achieve that comfort level? Create an open, transparent environment. Teams need to feel like there is room to fail, learn and improve in order to create their best work, and it's equally important for them to see you practicing what you preach.

2. Perfection != Quality

A perfect system has no flaws. A system can have high quality even if it is not perfect; it is all about risk management.

Can you risk that your APIs fail on certain corner cases that happen 0.001% of the time?


Can you risk delaying going to market with a product, because you need to spend another month ironing out obscure bugs?

Probably not.

3. Plans are worthless; planning is valuable

First off, you need to understand this: Unless you're doing something you've done multiple times before (e.g., running a 5k), your estimates are going to be wrong. Given that your estimates are going to be wrong, does that mean that your estimates are worthless? Well, that depends on who you're asking, and most importantly, the purpose for which you’re using your estimates.

If you're using detailed estimates of tasks to predict a release date, you're not going to be successful.

If you're using estimates to prioritize and identify the return on investment for specific features, you're using estimates in a better way.

If you're using estimates to predict what kind of competencies you need to form your future employees, you're using them in an even better way.

Because estimates can be used in so many different situations, it’s important here to return to the point I made in Lesson #1 - ideally, you’ve created a transparent environment, where you and your team are comfortable communicating the nature of the estimates you’re delivering to outside teams and stakeholders.

4. Your assumptions will be proven wrong

At Unacast we've gone through *at least* one major redesign of our main data pipeline every year. Changes aren't necessarily a bad thing. Changing our minds means that we've learned from our mistakes, and our next iteration is going to be better, faster and more resilient. If you have similar experiences, this is a sign that your business and product is improving.

We've also been through multiple variations of organizational patterns, from "everyone-in-the-same-team" to currently full-fledged autonomous product teams. Expect your organization, technology, and product to evolve at approximately the same pace.

5. Teams are your most important priority

When people talk about prioritizing the roadmap, they often think about the grand exercise of prioritizing everything in the company. They are all wrong. Prioritizing tasks and roadmap items catches only a small fraction of the real priorities you have in a product organization.

The real priority is put into effect when you split your organization into teams. Team sizes set the bandwidth of the teams, deciding how much they can accomplish in a fixed amount of time. Priorities set on a company level always have to abide by the size of the team, but those priorities should be seen as a valuable tool to understand when teams are the correct size, and when they are not.

6. Technology is just a tool

Don't fall in love with technology. Technology is mainly a tool to accomplish a specific task. When you go through changes, the chances are that a technology that's fitting for building something for a small scale isn't the right tool for something that will process orders of magnitude more data.

In my experience, putting too much faith in a programming language, tool or paradigm does not bring any good. If you want to create good stuff, you need to choose the right tool for the job at every time and make sure you replace the things that aren't working.

7. Buy before you build

To us, this is a big one. It is one of the only concepts that has been with us from the very beginning.

We have always opted to buy instead of build. What this means in practice, three years after our first MVP, is that we do not host a single VM in the cloud. All our services are fully managed by Google Cloud, and the closest we get to bare metal is our Kubernetes clusters.

Fully managed services, of course, come at a price, but also with a significant benefit; you spend way less time on operations, and more time on innovation. If there existed clear advice I could give to my previous self, it would be to go all-in on the cloud, and all-in on managed services.

What are your best lessons from working with and scaling Engineering teams? Let me know in the comments or hit me up on Twitter!

Related Articles