On Ambitious Goals

I am a fan of setting stretch goals that are just a little bit beyond what seems realistically achievable. They are motivating and communicate a certain optimism about the capabilities of the team.

However, not all dimensions of a project lend themselves to ambitious goal-setting. We should be particularly careful about goals that affect technical design decisions as they usually come with a trade-off. Make sure that the stretch indicates the direction you want the project to take.

A valid stretch goal in my book is to clearly define a vision for a goal state and then to try to arrive there a little bit earlier than what you think is realistic. For instance, let’s say you build a B2B app and you have an interesting prospect, asking by when you could have the most crucial functionality ready for a beta test and then aiming for this goal (whilst managing expectations to the outside) sounds like a valid plan.

However, here are some examples of stretch goals that are potentially dangerous:

“Our app will support millions of users per day”

Imagine your teams had some issues getting the performance right during load peaks. To motivate the team you try to set an ambitious goal: Supporting millions of concurrent requests to the backend. Unless this scale is really in the realm of what you can expect, this is a dangerous ambition. Load estimates should be based on realistic calculations. The reason is that supporting loads orders of magnitudes higher than what you would expect will require compromises in other places: Switching out cheap pieces of infrastructure with more expensive ones. Sacrificing consistency guarantees in exchange for a smaller overhead when horizontally scaling the data layer. Optimizing pieces of code that would not be a bottleneck under normal load.

“Our analytics dashboard will have minutely refreshes”

Your data analytics jobs are currently big batches of SQL running over your trusted old warehouse and you want to motivate your team to build something more fresh and modern? Do you want the users of your analytics dashboards to have access to more up-to-date information?

You might be tempted to throw down the gauntlet: A maximal latency of less than a minute!

Again you would be better served by being realistic about the exact requirements. What’s the impact of only seeing the data from one hour ago? What about two hours? Does all of the data need to be that fresh?

A very low latency requirement could imply drastic changes to your data processing pipeline with all kinds of sacrifices on the dimensions of cost and accuracy of the results.

Coming from big batch jobs your teams will have no choice but to completely rebuild the system from scratch and opt for a different paradigm.

Being realistic about requirements, on the other hand, can open up much more pragmatic corners in the solution space: Maybe the big daily batch job can keep running as long as one particular metric is refreshed more frequently?

“We can process petabytes of data”

Your platform solution is currently limited in the amount of data it can process and you want to broaden the range of customers you can target? It may be tempting to aim high. After all, if you can process petabyte-scale data, applications a couple of orders of magnitude lower should also work. You probably see where this is going: Do your homework and be realistic. An extremely ambitious figure will automatically rule out solutions that could have been more appropriate.

Conclusion

You spotted the pattern: Every technical decision trades off a number of factors which is why it is important to have a somewhat realistic view of the range of possible use cases.

Being ambitious in this context just means distorting the problem space leading to an inferior solution. Worse, your teams will spend valuable time on low-value development activities.

It all comes back to Richard Rumelt’s definition of strategy: A good plan makes use of your relative strengths and accounts for your relative weaknesses. This extends to technical decisions. Having a strict real-time constraint can be seen as a relative weakness. Knowing that the data your process has modest size is a relative strength.

Distinguishing clearly the things that matter from those that do not is an important step in the exercise of identifying the adequate point in the solution space. Giving your team the space to find this sweet spot is probably the best way to demonstrate trust.