On Greener Grass Design Decisions

When making technology choices, most of us have to learn the hard way that almost every single decision we take comes with a trade-off. Most of the time technology choice A will have some advantage over technology choice B and vice versa. Very rarely a choice will turn out to be “Pareto-dominant”, meaning superior along all relevant dimensions. We tend to suffer from the disadvantages of our current choice, making other alternatives looking more attractive than they are. I would like to advocate for caution.

A pattern that I notice from time to time is that an engineering team questions a technology decision based on recent pains. Maybe you built your product on top of a traditional relational database and you are worried about scaling, making one of the many NoSQL solutions out there look attractive. Maybe you would prefer adopting the same language across front-end and back-end to avoid the overhead of managing multiple ecosystems. Maybe the container orchestration solution of your cloud vendor feels too limiting and you are tempted to set up your own Kubernetes cluster.

Any of these can be the right move in a given situation as they constitute an improvement along one or multiple dimensions that are currently causing you headaches.

It is, however, important to consider that real-world systems face challenges on more than one dimension. Your product does not look like the system shown in the vendor’s sales pitch. Your data analytics pipeline is likely more than a big word count.

Some of these dimensions are things you might have taken for granted in your current stack: Your database offers isolation levels that prevent inconsistent writes, your backend technology choice gave you access to several libraries you depend on and the container orchestration platform “just worked” without requiring a lot of configuration.

When considering a move from A to B, it helps to build a prototype and to make a list that includes the downsides of B beyond the concrete problem at hand. Nevertheless, even then I still tend to worry about the unknown unknowns, the little details that nobody considered, that don’t show up in your prototype, and which turn out to be devastating.

Maybe it is advisable to err on the side of conservatism and to make sure you really considered all of the grass on this side of the fence before you start looking for other pastures.