There’s definitely more uncertainty in going with a microcloud provider, but choosing a smaller company for your GPU services may pay off big in the end. Credit: Ken stocker / Shutterstock As most IT people know, GPUs are in high demand and are critical for running and training generative AI models. The alternative cloud sector, also known as microclouds, is experiencing a significant surge. Businesses such as CoreWeave, Lambda Labs, Voltage Park, and Together AI are at the forefront of this movement. CoreWeave, which started as a cryptocurrency mining venture, has become a major provider of GPU infrastructure. This shift illustrates a broader trend in which companies are increasingly relying on cloud-hosted GPU services, mainly due to the high cost and technical requirements of installing and maintaining the necessary hardware on-site. Since public cloud providers are not discounting these computing services, microclouds provide a better path for many enterprises. Why don’t we stick with “traditional” cloud services provided by AWS, Google Cloud, and Microsoft Azure, which also offer a range of GPU resources? The answer, as usual, is money. Microclouds are often a more cost-effective solution for AI projects that require GPUs. The cost of renting popular GPUs such as Nvidia’s A100 40GB can be significantly lower on CoreWeave or another microcloud platform, compared to Azure or Google Cloud (check current pricing; this is a very general observation). Enterprises, be wary Despite this sector’s vibrancy, the future remains somewhat uncertain. The viability of growth for these providers depends on ensuring a consistent supply of GPUs in large quantities and at competitive prices. Also, with major cloud incumbents investing in bespoke AI processing hardware, competitive pressures could intensify and manifest as lower prices that keep the big cloud providers competitive. Furthermore, while GPUs are currently preferable for generative AI tasks due to their speed and ability to do many things at once, not all AI workloads require GPUs. As I’ve pointed out before, GPUs are overkill for most generative AI workloads. These days they are often more of a status symbol than a necessity. CPUs can still be used for less time-sensitive tasks, where a slower pace is acceptable. Some promising new GPU-like processors are in development that might offer more purpose-built processing for generative AI at a lower cost than current GPUs. The core goal of using any resource, including processors, is to do the most with the least. For most AI use cases, the costs will bring many enterprises back to reality and the hype will get tossed out the window. A bright future for microclouds That’s the cautionary news for enterprises that might get blinded by the hype. The immediate outlook for microclouds is optimistic. Analysts expect that the burgeoning field of generative AI will continue to drive demand, which should encourage more GPU-centric AI cloud startups. My advice? If GPUs will be required for upcoming projects, then enterprises should consider solutions that require GPUs. The cost of switching, at least for now, should not be a determinant for most of these architectures. These new entrants could offer stiff competition to established cloud players, especially for customers who are open to adopting newer and cheaper technologies to manage their multicloud environments. Enterprises are struggling with the high cost of the cloud, which is exceeding their budgets. A cheaper alternative will be a welcome development. Almost two decades ago, the major players began to emerge in the cloud computing market. Between 2012 and 2016, we went from about three dozen cloud companies to just a handful as the cloud market normalized. I expect to see the same pattern here. Any new upstart has major concerns. They could lose funding, be crushed by larger players, or experience a myriad of other unforeseen catastrophes. Most likely they will be purchased down the road and combined with a larger cloud system, or perhaps a few of these new companies will combine forces. Some of these outcomes spell trouble for enterprises, but most do not. The future of microclouds remains bright. Related content analysis Generative AI won’t fix cloud migration You’ve probably heard how generative AI will solve all cloud migration problems. It’s not that simple. Generative AI could actually make it harder and more costly. By David Linthicum Jul 12, 2024 5 mins Generative AI Artificial Intelligence Cloud Computing analysis All the brilliance of AI on minimalist platforms Buy all the processing and storage you can or go with a minimum viable platform? AI developers and designers are dividing into two camps. By David Linthicum Jul 09, 2024 5 mins Generative AI Cloud Architecture Artificial Intelligence analysis The next 10 years for cloud computing Despite AI's explosive growth, the industry still needs to face facts that customers are unhappy about costs and vendor lock-in. By David Linthicum Jul 05, 2024 5 mins Amazon Web Services Google Cloud Platform Microsoft Azure analysis Serverless cloud technology fades away Serverless was a big deal for a hot minute, but now it seems old-fashioned, even though its basic elements, agility and scalability, are still relevant. By David Linthicum Jul 02, 2024 4 mins Serverless Computing Cloud Computing Software Development Resources Videos