Public cloud providers are missing the mark with AI

Microsoft’s recent earnings call painted a sobering picture. Despite the company’s aggressive push into artificial intelligence and substantial investments in infrastructure, growth numbers fell short of expectations. As CEO Satya Nadella attempted to explain the shortfall to investors, one reality became increasingly clear: The traditional public cloud model is struggling to deliver on the promises of generative AI.

I’ve been warning about this scenario for months—losing some of my friends at public cloud providers in the process. The challenge isn’t just about having enough GPUs or data center capacity, it’s about fundamental misalignment between how public clouds are built and what AI workloads need. The public cloud providers are attempting to fit the square pegs of their existing cloud computing offerings into the round hole of AI. It’s not working out well.

When enterprises try to scale their AI initiatives on traditional cloud infrastructure, they’re often met with unpredictable costs, performance bottlenecks, and infrastructure limitations that make sustained growth hard. The problem is a basic architectural mismatch. Public cloud providers built infrastructure to accommodate generalized computing workloads. These are the standard enterprise applications that dominated the past decade. However, AI workloads are different. They require specialized hardware configurations, massive data throughput, and complex orchestration capabilities that weren’t part of the original public cloud design philosophy.

An AI and cloud mismatch

When I point this out, I get pushback. I am told that infrastructure built for general-purpose computing needs will also accommodate the special needs of AI workloads. You don’t have to be an AI engineer to understand that such a plan will not work. I suspect the cloud providers hoped to forgo the expense and risk of building out infrastructure to better accommodate AI systems and thought nobody would notice. Well, they noticed.

The mismatch manifests in several critical ways. First, the pricing models that worked well for traditional applications became prohibitively expensive when applied to AI workloads. Companies running large language models or training sophisticated AI systems are finding their cloud bills skyrocketing, often without proportional business value. I get a call a week from clients alarmed that their cloud bills are 20 times higher than what they expected. They are in a panic now that they are on the hook to get an AI system deployed that might drain IT budgets.

Second, the infrastructure itself isn’t optimized for the intensive, sustained computational demands of AI applications. What works for running a web application or database simply doesn’t cut it for modern AI workloads.

We’re already seeing the consequences. More enterprises are exploring alternative approaches, including private AI infrastructure and hybrid solutions. They’re finding that the promise of simple, scalable AI deployment in the public cloud often comes with hidden complexities and costs that make it difficult to achieve growth.

This isn’t just about technical limitations, it’s about business model adaptation. Public cloud providers need to recognize that AI requires a different approach to infrastructure, pricing, and service delivery. The current model of charging for general compute resources and adding premium fees for AI-specific services isn’t sustainable for most enterprises, and they are moving on to non-cloud alternatives.

The stakes are high. As enterprises continue to invest heavily in AI initiatives, they’ll gravitate toward platforms that can provide predictable performance, reasonable costs, and specialized infrastructure. Currently, that is AI private clouds, traditional on-prem hardware, managed service providers, and the new AI-focused microclouds, such as CoreWeave. Public cloud providers risk losing their position as the default choice for enterprise computing if they can’t adapt quickly enough. Since they are still lashing out at me, I suspect they have yet to get a clue.

What should enterprises do?

In the rapidly evolving landscape of artificial intelligence, enterprises face a pivotal moment. Imagine a company standing at a crossroads, confronted by the limitations of public cloud providers to adequately support their AI ambitions. Recognizing this challenge, savvy leaders are developing strategies to secure their organization’s future.

One approach gaining traction is the hybrid strategy, a dual-path plan that balances the agility of public cloud resources with the control of private infrastructure. Companies leverage public clouds for bursts of experimentation while dedicating specialized AI infrastructure for resource-intensive workloads. Flexibility and efficiency in an approach that has been around for a while.

Cost management is another vital consideration. The finance team should diligently monitor cloud expenses, armed with sophisticated tools that track usage in real time. They’re analyzing the total cost of ownership, uncovering insights about reserved instances and committed-use discounts, carefully picking the most economical options for their predictable AI workloads.

As they delve deeper, these enterprises begin a thorough assessment of their infrastructure needs. They ask the crucial questions: Which workloads truly require cloud scalability? What can run efficiently on dedicated hardware? By investing in specialized AI accelerators, they find the right balance between cost and performance.

Risk mitigation is paramount as well. To prevent vendor lock-in, leaders ensure their applications remain portable, mastering the art of container orchestration. They embrace flexibility in their data architecture, prepared to pivot as needed.

The path forward may be complex, but those who navigate it wisely will position themselves for success in an AI-driven world. It’s a journey to ensure not just survival, but growth and innovation to harness the true power of this AI stuff.