Why enterprise investment in AI agents hasn’t yielded results

Enterprises have rushed to capitalize on the transformative potential of AI agents, but a stark reality is emerging. Our recent survey of more than 1,000 enterprise technology leaders revealed that more than half of organizations (68%) have budgeted over $500,000 annually for AI initiatives, yet nearly all (86%) lack the foundational infrastructure needed to deploy them. This gap between ambition and execution capability isn’t merely technical—it represents a strategic challenge that threatens to undermine AI investment returns.

And the stakes are rising. Gartner predicts that by 2028, 15% of day-to-day work decisions will be made autonomously through AI agents. However, rushing development without proper controls puts organizations at risk—by that same year, the analyst firm predicts that a quarter of enterprise breaches will trace back to AI agent abuse. This tension between speed and safety is where most initiatives break down, highlighting the need for three critical foundations:

Knowledge – ensuring agents have comprehensive access to the right organizational data;

Guardrails – establishing robust security, governance and compliance controls; and

Tools – giving agents the ability to take meaningful actions beyond simple responses, transforming them from basic chatbots into valuable business automation partners.

The hidden integration challenge

The rush to show progress masks a more fundamental problem: knowledge access, not AI technology itself. For many organizations, current technology ecosystems are fragmented, which creates significant barriers to effectively implementing AI agents across diverse business processes.

In our survey, 79% of organizations expect data challenges to impact their AI rollouts. When agents need to build knowledge by accessing and reasoning across several data sources, this fragmentation becomes a showstopper (and not in a good way). The knowledge foundation requires not just data connectivity but contextual understanding to ensure agents can interpret information correctly within a specific business domain.

Breaking the build-buy loop

These knowledge and data access challenges are forcing enterprises into a difficult choice. Through various conversations with CIOs, we’re seeing teams get caught in an endless build-versus-buy cycle, particularly when it comes to equipping agents with the right tools.

Some start with custom development, pouring resources into building authentication systems and data pipelines, only to realize they’re months away from implementing actual AI capabilities. The tools foundation—which gives agents the ability to execute transactions, modify systems, or automate workflows—becomes an afterthought rather than the core design principle it should be, resulting in agents that can analyze but not act.

Others take the opposite approach, activating AI features across their SaaS stack. But with most projects requiring multiple data sources, these point solutions multiply until IT teams spend more time managing tool integrations than driving value. Their agents remain limited in scope, unable to complete end-to-end processes without human intervention.

The most dangerous path? Trying to bridge these approaches. Teams layer custom code on top of vendor tools, creating brittle connections that snap under real-world loads. We saw the same story play out during early cloud adoption—and those wounds still haven’t healed at many enterprises.

Security can’t be an afterthought

The speed-versus-safety tension plays out most critically in implementing effective guardrails. Our survey shows security concerns top the list of barriers, with 57% of organizations citing it as their primary challenge. Security is one of the rare areas where both leadership (53%) and practitioners (62%) align on the urgency of the problem.

The challenge? Traditional security approaches don’t work for AI agents, which require comprehensive guardrails spanning authentication, authorization, data handling, and decision boundaries. With 42% of respondents requiring eight or more connections to data sources to meet their AI agent initiatives, point-to-point security creates vulnerability gaps with each new integration.

More importantly, AI agents need different types of monitoring and guardrails—it’s not enough to track access. Decisions, data flows, and execution patterns need to be watched closely. Guardrails must be both flexible enough to allow productive work and rigid enough to prevent misuse or unintended consequences. With the risk of enterprise breaches looming, getting this right becomes critical.

Learning from success

Some organizations are finding a better way forward. Take the Aprende Institute, one of our customers, for example. What they estimated as a multi-quarter project went live in days by building on the right foundation. Their success came from focusing on infrastructure first.

What was their path?

Start with one high-value process where you have clear data access and measurable metrics.

Build standardized patterns for data orchestration that work across your tech stack.

Design your testing strategy to validate data pipelines, AI logic, and integration points independently.

Most importantly, plan for scale from the start. Your first agent might handle IT tickets—which 61% of enterprises identify as a top use case for AI agents.

Unlock AI’s potential through integration

The promise of AI agents extends far beyond better chatbots. We’re seeing organizations transform customer support, financial operations, and employee experiences through properly equipped agents. The winners in enterprise AI won’t be the ones with the most agents or biggest budgets, but those who first establish the three foundations: scalable knowledge access, comprehensive guardrails, and flexible tools integration patterns.

Nearly 90% of enterprises consider integration with organizational systems essential for AI success, making it clear that solving the integration challenge is the key to unlocking AI’s transformative potential. Organizations who will see real returns will be those who don’t rush to deploy agents and instead invest in the technical foundation to deploy them safely and effectively at scale. By solving these core challenges first, they’ll be positioned to move beyond prototypes to true business transformation.

Alistair Russell is co-founder and CTO at Tray.ai.

—

Generative AI Insights provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss the challenges and opportunities of generative artificial intelligence. The selection is wide-ranging, from technology deep dives to case studies to expert opinion, but also subjective, based on our judgment of which topics and treatments will best serve InfoWorld’s technically sophisticated audience. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Contact doug_dineley@foundryco.com.