View cloud architecture through a new optimization lens

As cloud computing architecture comes of age, the ways we define success should mature as well. In 2021, I pointed out that optimizing cloud computing is more of a binary process than an analog one.

What I said then is still true: “There’s a lot at stake. Architectures that are underoptimized and costly (cloud architectures) may indeed work, but they may cause the business to lose millions a week while most people are none the wiser. Thirty technologies are used where 12 would have worked better, and not designing for change means that business agility suffers.”

Why are most cloud architectures poorly optimized? During the planning and design stages, most cloud architects do what they were taught in cloud architecture courses, or they apply what they read in any number of how-to-cloud references, or they might even adopt the tips they learned from previous cloud architecture projects and mentors. All guide the architect to a series of generic reference models, processes, and technology stacks that should be modified to address an enterprise’s unique business needs. This approach consistently results in underoptimized architectures that cost the enterprises more (or much more) money than they should. What’s going on?

To answer that question, let’s take a step back. What does an optimized cloud architecture actually mean? I defined the process of cloud architecture optimization in October 2020 and included a high-level model to leverage. I even augmented my cloud architecture course to include this concept, which will soon be released here.

Next, we need to recognize that the major focus in the past was to get everything to work together, with little regard as to how well it worked or how complex the solution became. The measure of success was “Does it work?” not “How well does it work?”

During development, the team stayed laser focused on their approaches to cloud architecture, migration, and net-new development, both in the wide (meta cloud architecture) and in the narrow (micro cloud architectures). Now it’s less about how you design and deploy your cloud migrations and net-new cloud-native developments, or how you leverage containers, serverless, or other small or large cloud computing solutions. Instead, it’s all about how you define your objectives for that solution.

IT and enterprise management in general is getting wise to the fact that a solution that “works” or “seems innovative” does not really tell you why operations cost so much more than forecast. Today we need to audit and evaluate the end state of a cloud solution to provide a clear measure of its success. The planning and development phases of a cloud deployment are great places to plan and build in audit and evaluation procedures that will take place post-development to gauge the project’s overall ROI.

This end-to-beginning view will cause some disturbance in the world of those who build and deploy cloud and cloud-related solutions. Most believe their designs and builds are cutting edge and built with the best possible solutions available at the time. They believe their designs are as optimized as possible. In most instances, they’re wrong.

Most cloud solutions implemented during the past 10 years are grossly underoptimized. So much so that if companies did an honest audit of what was deployed versus what should have been deployed, a very different picture of a truly optimized cloud solution would take shape. Perhaps there is too much or not enough use of containers. Or failure to force cloud-native refactoring—or not considering those advantages. Or the new trend that I’m seeing, making multicloud more complex than it needs to be and failing to define common cross-cloud services such as security and operations. In some cases, a solution uses too many common services, but those situations are not as common.

Speaking in generalities, cloud architects apply what they learn from books, videos, articles, and even the ways that I and other pundits report how technology should be leveraged. The architects define what the business needs, and then they match those needs to the most optimized solutions. That’s a good approach.

However, let’s say Vendor A has the best native apps available for your financial operations, Vendor B has the best native apps for your CRM needs, and Vendor C has the best native apps for your inventory requirements. Going multicloud to get the best of breed for these three requirements, as well as for dozens of other choices (security, storage, networking, and so forth), may not be in the overall best interests of your enterprise. Each of those choices adds another layer of complexity and cost that can quickly outstrip the added benefits.

This does not mean to cheap out on the technology you use to build your cloud solutions. Just be aware that getting to the most optimized architecture is still more art than science. Sometimes you need to invest in more technology, sometimes less. What’s important is to define something that is as close to optimized as it can get.

Today, cloud optimization means we must audit and reevaluate our current cloud solutions and then look at the augmentation of the metrics moving forward. This will not be easy, but consider the potential value returned to the business. In some cases, cloud optimization may even save the business.

When there are cloud solutions in place that work, many staff on Team “It’s Good Enough” tend to become one or all of the three wise monkeys: They do not want to hear, see, or speak any evil about the cloud solution they helped deploy or currently operate. Conversely, there always seems to be someone on Team “Wait, It’s Costing What?” who realizes the cloud solution will continue to drain enterprise resources much more than it should. They will be the first to suggest or insist on an audit.

Which team will you be on?