Why the cloud is so complicated—an explanation, not an excuse

It’s not news that I’ve beat the cloud complexity drum for the past several years. Complexity is rarely an issue when a company first implements a cloud architecture, so many people on staff don’t feel an urgent need to deal with it. Lately, more people are noticing that the cloud is much more complex than the original sales pitch indicated. If cloud computing is our ultimate destination, we need to deal with this complexity sooner rather than later.

Adrian Bridgwater, writing for Forbes, nailed the reason cloud computing is so complex: “Cloud is complicated because computing itself is inherently complicated, and the popularization of the cloud model approach has been constitutionally riddled with chaotic platform-level mismatches that have slammed together with incongruent tectonic force.”

Cloud computing technology is easily accessible, and it supports agile configurations. Without discipline, it’s fast and easy to make things complex. Think about this question: Can you think of an enterprise with an “as is” state without cloud that turned into a digitally enabled “to be” state with cloud where the complexity was reduced? It almost never happens. Metrics that measure complexity typically show at least a 50% increase in complexity with no or very little gain in business value. 

Why does this happen? There are two basic and obvious reasons:

First, the cloud itself is complex. With hundreds of native services built into most public clouds and thousands of third-party services in the marketplace, the sheer number of available choices creates end-state architectures and applications that are more complex.

Those charged with selecting and configuring the cloud technology ultimately drive the complexity. They will tell you they are simply picking best-of-breed solutions and complexity is just a byproduct. The reality is that they are dealing with a systemically complex public cloud platform; thus, the resulting cloud solutions are likely to be complex as well. The projected complexity needs to be factored into the best-of-breed decision-making factors.

Second, most of us don’t consider complexity a negative architectural result. In the old days when we built systems on mainframes, they were much less complex because the platform of the mainframe was much less complex. Only a handful of programming languages, operating systems, and native databases ran on the “big iron.” Architectural complexity was never brought up or taught to new architects because built-in limitations kept complexity in check.

These days we lack focus on how to leverage cloud-based platforms using uncomplicated methodologies. As an aside, complexity could be a necessary outcome, depending on the business requirements. It’s difficult for me to say something is too complex without a complete understanding of the business problems we need to solve.

That said, most of the resulting cloud complexity I see is a result of not considering less complicated alternatives. Nobody asks the right questions while the solution is being developed. The ultimate result is, “Well, it works, doesn’t it?” The fact that something works does not mean the resulting solution is optimized, which typically means it has increased operational costs.

I’ve taken a mediation approach to cloud complexity rather than attempt to get ahead of all underoptimized architectural decisions. Unfortunately, it’s easier to fix things after you can show they are broken than it is to tell people at the onset, “If you disregard complexity from the start, you will do this the wrong way and you’ll have to do it again the right way.” There is an obvious cost to the “do it twice” approach. 

The best solution is to manage complexity right away in the initial architecture and application design stages. However, out here in the real world, removing complexity will be mostly reactionary for the next three to six years. There is an appetite for speed that means someone will have to loop back and deal with the resulting mistakes. 

So, now we know why complexity exists. We know what we can do about it, proactively and reactively. What should you do as an enterprise IT leader? At the start of any cloud project ask questions about added complexity. Recognize that complexity is an issue and define a strategy to deal with it from the start.