Need GPUs? Take a look at microclouds

As most IT people know, GPUs are in high demand and are critical for running and training generative AI models. The alternative cloud sector, also known as microclouds, is experiencing a significant surge. Businesses such as CoreWeave, Lambda Labs, Voltage Park, and Together AI are at the forefront of this movement. CoreWeave, which started as a cryptocurrency mining venture, has become a major provider of GPU infrastructure.

This shift illustrates a broader trend in which companies are increasingly relying on cloud-hosted GPU services, mainly due to the high cost and technical requirements of installing and maintaining the necessary hardware on-site. Since public cloud providers are not discounting these computing services, microclouds provide a better path for many enterprises.

Why don’t we stick with “traditional” cloud services provided by AWS, Google Cloud, and Microsoft Azure, which also offer a range of GPU resources? The answer, as usual, is money. Microclouds are often a more cost-effective solution for AI projects that require GPUs. The cost of renting popular GPUs such as Nvidia’s A100 40GB can be significantly lower on CoreWeave or another microcloud platform, compared to Azure or Google Cloud (check current pricing; this is a very general observation).

Enterprises, be wary

Despite this sector’s vibrancy, the future remains somewhat uncertain. The viability of growth for these providers depends on ensuring a consistent supply of GPUs in large quantities and at competitive prices. Also, with major cloud incumbents investing in bespoke AI processing hardware, competitive pressures could intensify and manifest as lower prices that keep the big cloud providers competitive.

Furthermore, while GPUs are currently preferable for generative AI tasks due to their speed and ability to do many things at once, not all AI workloads require GPUs. As I’ve pointed out before, GPUs are overkill for most generative AI workloads. These days they are often more of a status symbol than a necessity.

CPUs can still be used for less time-sensitive tasks, where a slower pace is acceptable. Some promising new GPU-like processors are in development that might offer more purpose-built processing for generative AI at a lower cost than current GPUs.

The core goal of using any resource, including processors, is to do the most with the least. For most AI use cases, the costs will bring many enterprises back to reality and the hype will get tossed out the window.

A bright future for microclouds

That’s the cautionary news for enterprises that might get blinded by the hype. The immediate outlook for microclouds is optimistic. Analysts expect that the burgeoning field of generative AI will continue to drive demand, which should encourage more GPU-centric AI cloud startups.

My advice? If GPUs will be required for upcoming projects, then enterprises should consider solutions that require GPUs. The cost of switching, at least for now, should not be a determinant for most of these architectures.

These new entrants could offer stiff competition to established cloud players, especially for customers who are open to adopting newer and cheaper technologies to manage their multicloud environments. Enterprises are struggling with the high cost of the cloud, which is exceeding their budgets. A cheaper alternative will be a welcome development.

Almost two decades ago, the major players began to emerge in the cloud computing market. Between 2012 and 2016, we went from about three dozen cloud companies to just a handful as the cloud market normalized. I expect to see the same pattern here.

Any new upstart has major concerns. They could lose funding, be crushed by larger players, or experience a myriad of other unforeseen catastrophes. Most likely they will be purchased down the road and combined with a larger cloud system, or perhaps a few of these new companies will combine forces. Some of these outcomes spell trouble for enterprises, but most do not. The future of microclouds remains bright.