Canalys: Companies limit genAI use due to unclear costs

As companies move from testing out generative AI tools and models into real-world use — also known as inference— they’re having trouble predicting what that use will lead to in terms of cloud costs, according to a new report from analyst firm Canalys.

“Unlike training, which is a one-time investment, inference represents a recurring operational cost, making it a crucial constraint on the path to commercializing AI,” said Canalys senior director Rachel Brindley in a statement. “As AI moves from research to large-scale deployment, companies are increasingly focusing on cost-effectiveness in inference, comparing models, cloud platforms, and hardware architectures such as GPUs versus custom accelerators.”

According to Canalys researcher Yi Zhang, many AI services rely on usage-based pricing models that charge per token or API call; that makes it difficult to predict costs when scaling up usage.

“When inference costs are volatile or excessively high, companies are forced to limit usage, reduce model complexity, or restrict implementation to high-value scenarios. As a result, the broader potential of AI remains underutilized,” said Zhang.