AI advancements are fueling cloud infrastructure spending

The latest data from the IDC Worldwide Quarterly Enterprise Infrastructure Tracker paints a compelling picture of growth in cloud infrastructure sales on demand. The fourth quarter of 2023 saw an 18.5% year-over-year increase in spending on compute and storage infrastructure for cloud deployments. It is a significant shift in the technological landscape, where AI is now front and center in the push to find cloud infrastructure to run it.

The spending surge indicates shifting budgets; a contrasting trend is the decline in the total number of units shipped. IDC says this shows a strategic move towards high-capacity, GPU-heavy servers with higher average price tags favored by hyperscalers. The idea is that these servers can do more than those driven by traditional CPU-based servers, thus, you need fewer.

[ Download our editors’ PDF AI as a service (AIaaS) enterprise buyer’s guide today! ]

The AI boom is driving change

AI is central to this growth trajectory. IDC, and most anybody with some market understanding, knows that AI is the primary driver behind the steep rise in cloud infrastructure spending.

This expansion is led by purpose-built hardware infrastructure tailored to handle AI-centric workloads. Of course, this means expensive GPU hardware investments, as evidenced by the meteoric rise in the stock of companies building and selling GPUs. The confluence of increased AI adoption and the demand for specialized infrastructure is reshaping enterprise cloud deployments and will likely drive a spending spree that will last a few years.

The most common question I get is no longer “Who’s the best cloud?” It’s “What GPU cloud should we use?” Both are still dumb questions, given that this is largely dependent on your business needs, not on loading your shopping cart with GPUs.

The cloud deployment infrastructure landscape

The IDC report offers insights into the evolving landscape of cloud deployment infrastructure spending, explicitly focusing on AI. I’m not sure that anyone will push back on that. However, there are some other market dynamics that we should be paying attention to, namely:

• Tech leaders’ rapid deployment of AI capabilities is changing infrastructure requirements, emphasizing the need for specialized, high-performance hardware. However, this will likely translate quickly into storage and databases, which are more critical to AI than processing. Who would have thunk?
• The shift towards GPU-heavy servers at higher price points but fewer units sold reflects the evolving market dynamics influenced by the priorities of cloud providers and enterprise tech behemoths. As I pointed out, this could be a false objective that leads many, including the cloud providers, down the wrong path. I suspect GPUs and GPU analogs will commoditize quickly, and cloud providers, enterprises, and big enterprise tech will overpay for tech investments that won’t return enough value.
• The significant uptick in cloud infrastructure spending underscores a robust investment in AI-related capabilities, which has far-reaching implications for technology and business landscapes. Most of this investment won’t pay off, given that companies’ technology infrastructure may be overkill in some respects and misaligned in others. In other words, they aim for requirement A when they should be aiming for requirement B.

What does all this mean for enterprise IT?

I suspect this will negatively impact rank-and-file IT leads attempting to finish the budgetary year with systems deployments, growth, maintenance, and operations. AI is on their radar screen, but as many tell me, they’ve yet to see the budgets to fund it. They will be consumers of the more “traditional” cloud services, which may be taking a backseat to AI now.  Some of those types of consumers may find they are not getting the love they once did.

Hopefully, the cloud providers are not that stupid, but I’m seeing some pretty crazy behavior now, including “cloud” conferences that are really generative AI conferences that use the word “cloud.” We’ll land somewhere in the middle, but some cloud services will be neglected as all the AI services get the funding.

We’ll likely also see higher equipment prices. While focusing on higher-end GPU-based servers, enterprises will also need higher-end servers, and if everyone is buying them up, then the price goes up and availability goes down. We’re already seeing GPU shortages, perhaps just a short-term thing, but we could see the prices for storage, networking, and processing equipment rise after a slow fall over the past 10 years.

Of course, there are some opportunities here as well. Prices may fall for non-AI-supporting cloud services, such as transactional storage and lower-end networking services. If everyone is focused on the higher end, the more traditional services may go on a fire sale. This won’t be announced, by the way. It will just happen, so keep an eye on the prices.

As I mentioned above, the more significant concern is those enterprises that made value investments in cloud computing. They hoped that the cloud computing providers would function much like power utility companies and just be there to provide the best service that they possibly could. As one of my clients said, they want the hype to blow over. It did, but it seems to be coming back with a vengeance. This will be bad for many, but it could be good for some. I wish that were not the case. The technology market is like the weather, for sure.