Learning cloud cost management the hard way

The allure of transformative business benefits in cloud computing prompted many organizations to invest heavily in cloud solutions over the past decade. Yet, despite this investment, many enterprises still grapple with cost inefficiencies that result in substantial financial waste. This issue persists even with the introduction and adoption of financial operations (finops) practices designed to curb this problem.

Many enterprises want to take the right steps but either they can’t get moving fast or, in many cases, they can’t find the talent that they need. They spent the last few years deploying to the cloud, but they now feel overwhelmed by the number of problems they need to solve to reduce cloud waste. Many quickly adopted finops, only to find they needed to address behavior and process; finops wasn’t a magical tool that would solve all their problems.

A persistent challenge

According to HashiCorp’s latest State of the Cloud Strategy Survey, 91% of respondents reported waste in their cloud spending. This is a troubling statistic, particularly when juxtaposed with 66% of these organizations increasing their cloud spending in the past year alone. Despite efforts to implement finops and other cost management frameworks, many enterprises have yet to learn from the mistakes of the past decade. The primary culprits for cloud waste include lacking necessary skills, overprovisioning resources, and idle or underused resources.

The rapid adoption of cloud technologies has outpaced the development of requisite skills within many organizations, leading to inefficiencies in provisioning, managing, and optimizing cloud resources. The No. 1 excuse that I hear from those overspending on cloud computing is that they can’t find the help they need to maximize cloud resources. They are kicking years of cloud-powered technical debt down the road, hoping that someone or some tool will come along to fix everything. Not likely.

Foundational to this issue is that many organizations adopt a “better safe than sorry” approach, allocating more cloud resources than needed. Although this prevents potential performance issues and outages due to running out of resources, it significantly drives up costs. Resources that are provisioned but not fully utilized contribute to wasteful spending. These idle resources often go unnoticed and continue to incur expenses.

This is cloud spending 101, but it’s surprising how many enterprises still make this mistake. More frustrating is that most know they are doing it but really don’t have a better solution than just tossing money at the cloud. Cloud providers have been a bit complicit and promote overuse by not showing their customers how to optimize and correctly size resources. In many instances, resources are overprovisioned due to cloud provider recommendations.

Still waiting for cloud adoption maturity

The survey underscores a critical point: Cloud maturity is pivotal in how organizations manage their cloud expenditures. Highly mature organizations report lower costs, faster speed to market, and more significant benefits from skilled cloud talent. Indeed, 73% of these mature organizations increased their cloud spending compared to 62% of low-maturity organizations, demonstrating a correlation between maturity and efficient cloud spending.

Enterprises must adopt a multifaceted approach to address these inefficiencies. Bridging the skills gap is imperative. Enterprises should invest in training and upskilling their workforce, ensuring that teams have the knowledge to manage and optimize cloud resources effectively. Investing in human capital can pay off in significant cost savings over time.

Implementing and adhering to finops principles is not merely a recommendation but a necessity. This includes continuous monitoring of cloud expenditures, setting budgets, and optimizing workloads to avoid overprovisioning. Effective practices should also entail regular financial and operational audits. Use outside auditors rather than risk the old rubber stamp from self-audits.

Automation tools powered by AI can play a crucial role in ensuring that resources are only provisioned when needed and decommissioned when not in use, thus preventing idle resources from unnecessarily accruing costs.

Moreover, a robust cost governance framework is essential for cloud cost management. This framework should include policies for resource provisioning, usage monitoring, and cost optimization. Governance ensures that best practices are followed across the organization and that all stakeholders are aligned.

Will we ever learn?

It’s frustrating that we’ve yet to learn how to do this correctly. 2020 wants their cloud spending problems back. This is not the only survey I’ve seen that reveals cost inefficacies on a massive scale. I see this myself.

The past decade has highlighted significant lessons about cloud cost inefficiencies. However, the persistence of these issues suggests that many enterprises still have a long way to go in mastering cloud financial management. By investing in skills, rigorously applying finops principles, leveraging automation, adopting strong governance, and conducting regular audits, enterprises can transform their cloud spend from a cost burden into a strategic asset.