AI vendors move to tackle the hidden cost of inefficient enterprise code

Enterprises don’t often admit it out loud, but a good share of their cloud bills can be traced back to something deceptively mundane: inefficient code.

A research report from software delivery platform provider Harness, which was co-authored with AWS, cited 52% of 700 engineering leaders and developers surveyed in the US and UK saying that the disconnect between finops and developers is leading to wasted spend on cloud infrastructure costs.

“The reality today is that developers often view cost optimization as someone else’s problem. This disconnect leads to over-provisioned resources, idle instances, and inefficient architectures that drain budgets,” the researchers wrote in the report.

Inefficient code is such a big part of that disconnect that it should be considered a CFO-level problem now, said HFS Research CEO Phil Fersht, because AI workloads are increasing power draw, carbon cost, and infrastructure spend.

“Compute waste is enormous. Studies from large cloud providers indicate that 20 to 40% of cloud compute is underutilized or consumed by inefficient code. Enterprises pay for that waste,” he said.

This silent tax on compute has caught the attention of AI coding assistant providers.

Code evolution rather than simple generation

Google, for one, is zeroing in on it by unleashing a new coding agent, AlphaEvolve, that shifts focus from code generation to code evolution.

The Gemini-powered coding agent is available in private preview, Google said in a blog post on Wednesday.

Users must first write a definition of the problem they want to solve, a test to evaluate proposed solutions, and a first draft of the code to solve the problem. AlphaEvolve then iteratively applies Gemini LLMs to generate “mutations” in the code and tests them to see if they are better than existing solutions, until it meets the test criteria.

The ability to evolve code by altering the underlying algorithm underneath it could be a game changer for enterprises, analysts say.

“Code evolution is powerful for enterprises that want to improve performance in areas such as routing, scheduling, forecasting, and optimization. These are the areas where algorithmic gains directly translate to commercial advantage, reduced compute cost, and better time to market,” Fersht said.

Bradley Shimmin, practice leader for data, AI, and infrastructure at The Futurum Group, said Google may be aiming to help enterprises to evolve entire codebases, rather than just help with syntax completion, code generation, and documentation.

Changing a long-standing practice

Fersht sees AlphaEvolve boosting enterprises’ efforts to change a long-standing practice that developers follow: write code first and optimize later.

“For a decade, developer culture prioritised speed and frameworks over optimisation. That worked when compute was cheap. AI flipped the equation. Models are compute hungry,” Fersht said.

“Enterprises now realise that inefficient code slows models, increases cost, and impacts real world performance,” Fersht said, adding that developers are being pushed to optimize sooner rather than later in the development lifecycle.

That pressure is not just because of LLMs’ huge processing power needs: Data centre capacity is now a strategic constraint as AI inference loads are scaling faster than infrastructure, Fersht said, adding that any tool that improves code efficiency also reduces the number of GPUs and the electrical power needed to run applications.

“That is why algorithm discovery is so important: It reduces the need for brute-force compute,” he said.

Other ways to optimize compute for coding

Algorithm discovery for code evolution isn’t the only way that vendors are looking to help enterprises optimize their expenditure on compute resources related to coding.

French LLM vendor Mistral, for one, has introduced a compact new open LLM specifically for coding, Devstral 2, which it claims is as effective as larger models. Smaller models are cheaper to run than larger models, because they require less powerful hardware and perform fewer calculations to generate an answer.

Anthropic, too, is also working to support developers, bringing Claude Code to Slack, where it can help them generate better code and reduce time spent on collaboration. Typically, Slack is where development teams hold conversations about architecture, and Claude Code’s integration there will help the coding agent get better context to generate code that’s more relevant to the team.