Kubernetes cost management for the real world

Microsoft’s decision to make Kubernetes a foundational service in Azure is paying off, not only for Microsoft but also for the rest of the Kubernetes ecosystem. That’s because the company is investing its engineers and money in more than its own projects, supporting and contributing to upstream tools and capabilities.

Sometimes that investment is used as a lever, with the aim of improving the ecosystem for users, but other times it’s about ensuring that we can run our cloud native applications more economically and, more importantly, with the minimum impact on the climate.

One key area of Microsoft’s investment in Kubernetes has been cost management tooling. Kubernetes is an orchestrator, scaling up and down its use of compute on demand. That makes it hard to predict how much Kubernetes will cost to run, especially in a public cloud where you’re mixing virtual server types and billing rules, with changing demands in memory, networking, and storage. Yes, your application will scale to the limits you’ve set, but how does that affect your bottom line?

Adding FinOps to Kubernetes

As engineers we don’t typically pay close attention to costs. After all, most of the time the bill for what we build and run is someone else’s problem. But the same could be said about operations and security, until DevOps and DevSecOps came along. With cloud services we can use the same techniques we use to monitor our applications’ performance to monitor costs, taking advantage of the increasingly important FinOps. This new discipline gives us new visibility into how much running our code costs, and new ways to ensure that those costs are charged back to the right departments. Using FinOps tooling we’re able to directly tie code to bills, rather than bundling it all into one IT operational expense.

This is where the open-source OpenCost tooling starts to shine. Sponsored by the Cloud Native Computing Foundation, the home of Kubernetes, OpenCost is a tool for measuring and allocating the costs of Kubernetes applications, with the option of helping you keep them under control. OpenCost’s contributors come from across the Kubernetes ecosystem, with monitoring providers like New Relic and Grafana Labs at one end, and the hyperscale cloud providers AWS, Google Cloud, and Azure at the other. Microsoft has announced support for OpenCost in Azure Kubernetes Service, Azure’s managed Kubernetes platform.

OpenCost allows you to drill down through your Kubernetes installation and operations to find out which containers, pods, deployments, etc. are costing you the most. Having real-time cost allocation like this gives you the ability to go beyond simply tuning your application for performance, instead optimizing for costs. This approach lets you find the sweet spot where your users get the best performance while costs are kept in check. It’s a balancing act, and one that can take some time to get right, but it’s another compelling tuning parameter for your applications.

Working with OpenCost in AKS

While OpenCost will have production support for AKS from May 2023, a special build was made available for Kubecon EU to help you get started. There is some configuration required once it’s been installed, setting the appropriate permissions for working with Azure.

OpenCost uses the Azure Consumption Price Sheet API to get real-time pricing data for your account. That ensures it takes account of appropriate discounts, such as using reserved instances. You can set this up by adding an Azure role to your account that gives OpenCost access to your billing details as a service principal, via your billing account ID. Once you have created this Azure role, save its key and secret for use with OpenCost. You configure OpenCost to access this data via either YAML or Helm, depending on which you used to set up your installation. If you have a custom pricing relationship with Azure, you’ll need your existing offer ID to get access to your pricing through the API.

Usefully OpenCost can deliver data to Prometheus, which gives you a time series database that can store both signals from Kubernetes and pricing data from OpenCost. This makes financial information part of your observability platform, so you can watch for conditions that equate to high cost and treat them as a failure. There’s even a kubectl plugin that can query OpenCost data about your services, so you can start to script operations based on historic costs.

Using cost data to manage Kubernetes

With real time data through the OpenCost API there’s scope for automated management models based on cost. If costs spike, why not use them as an input to KEDA, the Kubernetes autoscaler, and treat a high cost as an event that can scale down a cluster? There are even options in this for providers like Azure, using OpenCost as a way to deliver dynamic pricing to users.

Why is Microsoft embracing the introduction of tooling that helps its customers spend less, not more? The company may have little choice, given that AWS and Google Cloud also are partners in the OpenCost project. However, it’s a change that fits with CEO Satya Nadella’s recent statement that Microsoft was “helping [its customers] realize more value from their tech spend.” By ensuring that customers can align their Kubernetes spending with usage, there’s an opportunity to dynamically optimize the use of Azure infrastructure.

Microsoft could also improve customer retention, giving it the opportunity to win future business, and at the same time keep its own capital expenditure under control. Running massive cloud data centers is expensive, and building out new capacity is even more costly. It’s in the best interests of both Microsoft and its customers to align on an operating model that allows both to spend, if not less, then at least just the right amount for their needs.

Building OpenCost into Azure will give both Microsoft and customers better visibility into resource usage and allow Azure to plan future expansions more carefully. In the light of Microsoft’s promise of long-term support for Kubernetes, it’s clear that cloud native development is here to stay, and that it’s now subject to the same controls as any other enterprise platform. We’re no longer experimenting, we’re building businesses and services, and those need to operate in a predictable manner if they’re to become profitable for us and for Azure.

The future for Kubernetes on Azure is going to be boring, which shouldn’t surprise us. After all, Kubernetes is infrastructure, and boring is the price we pay for maturity and enterprise acceptance. What’s interesting, as we move into a Kubernetes-powered future, is what we’ll do with that infrastructure and what we’ll build on top of it.