Agentic mesh: The future of enterprise agent ecosystems

Every week, a new AI agent platform is announced, each promising to revolutionize how work gets done. The vision is compelling. Simply task an AI agent with a job, and it will autonomously plan, execute, and deliver flawless results. Industry leaders are leaning into this vision. Nvidia CEO Jensen Huang predicts we’ll soon see “a couple of hundred million digital agents” inside the enterprise. Microsoft CEO Satya Nadella takes it even further: “Agents will replace all software.”

The vision is compelling, but the reality is far more complex.

To fulfill this vision, agents must evolve from their current immature state to address and embody the needs of the modern enterprise. If enterprises want to use AI agents at scale, they must architect them in an “enterprise-grade” way that controls and reduces errors, while ensuring reliability, security, and observability from the ground up. In addition, these enterprise agents must run in an “enterprise-grade” ecosystem that lets agents safely and securely find each other, collaborate, interact, and even transact.

Simply put, we need an “agentic mesh”—a network of enterprise-grade AI agents running in an enterprise-ready ecosystem. This article explores what it takes to build such agents and the infrastructure needed to support them at scale.

Why enterprises need AI agents

Enterprises need to move faster, but they are slowed by manual workflows and fragmented systems. AI agents offer a new approach: software that can determine how to execute tasks. When built for enterprise use, agents can help address core challenges including:

Reducing information overload. Employees spend a significant portion of their time searching for information. Agents can surface insights proactively, eliminating the need for manual searches.

Driving efficiency and scalability. Agents can automate multi-step processes, scaling operations without scaling head count.

Enhancing customer engagement. By combining real-time insights with historical context, agents can deliver personalized experiences across channels.

Accelerating innovation. Agents offload repetitive tasks, freeing humans for more strategic work.

AI agents address rising enterprise complexity, but they aren’t just chatbots or demos. To deliver real value, agents must be built for the enterprise, with reliability, visibility, and security from the start.

The problem: Most agents aren’t built for the enterprise

Many companies describe agents as “science experiments” that never leave the lab. Others complain about suffering the pain of “a thousand proof-of-concepts” with agents. The root cause of this pain? Most agents today aren’t designed to meet enterprise-grade standards. 

Agents often:

Start as prototypes in notebooks or large language model (LLM) sandboxes. Great for demos, not for deployment.

Deployed in a single Python “main” running in a single operating system process, which is practical for only the smallest loads.

Lack the observability, traceability, and access control, essentials for operating in real-world systems.

Operate in silos, with no standard way to interact with other agents, services, or teams.

Push too much decision-making onto the model itself, trusting a stochastic system to get it right every time. The more we ask LLMs to do, the less accurate and repeatable they become.

The result is a fragile foundation, useful in isolated scenarios, but brittle at scale. To truly harness the power of agents, enterprises must treat them like first-class components in their software architecture. That means securing them, governing them, instrumenting them, and embedding them into robust infrastructure.

The danger of agent silos

As enterprises adopt more agents, a familiar problem is emerging: silos. Different teams deploy agents in CRMs, data warehouses, or knowledge systems, but these agents operate independently, with no awareness of each other.

And when agents don’t share context, it leads to duplicated effort and missed insights. For example, a CRM agent might recommend a sales action without knowing that the data warehouse agent has identified a relevant market trend. Each agent works with partial information, and teams end up building overlapping functionality.

This isn’t just inefficient, it undermines trust in the system. If agents can’t coordinate, they can’t support high-stakes or cross-functional use cases. Agents need a way to discover each other, share information, and coordinate actions. But without a common framework, every new agent increases complexity instead of value.

What’s needed is a foundation that allows agents to operate as part of a larger system, not just as standalone tools. That’s where the agentic mesh comes in. 

Agentic mesh: An enterprise-grade agent ecosystem

An agentic mesh is a way to turn fragmented agents into a connected, reliable ecosystem. But it does more: It lets enterprise-grade agents operate in an enterprise-grade agent ecosystem. It allows agents to find each other and to safely and securely collaborate, interact, and even transact.

The agentic mesh is a unified runtime, control plane, and trust framework that makes enterprise-grade agent ecosystems possible.

Foundational components of the enterprise-grade agent ecosystem. 
Confluent

Agentic mesh fulfills two major architectural goals: It lets you build enterprise-grade agents and it gives you an enterprise-grade run-time environment to support these agents.

To support secure, scalable, and collaborative agents, an agentic mesh needs a set of foundational components. These capabilities ensure that agents don’t just run, but run in a way that meets enterprise requirements for control, trust, and performance. These agentic mesh components include the following:

Marketplace: A central place where agents can be discovered, evaluated, and deployed. Teams can find prebuilt agents or publish their own, enabling reuse and reducing duplicated effort.

Registry: A system that enables agents to register, authenticate, and discover each other. This allows agents to collaborate based on defined roles, capabilities, and permissions, without custom integrations.

Observability and Governance: Tools and standards for ensuring security, traceability, and policy enforcement. This includes logging, metrics, access controls, and certifications, critical for auditability and operational support.

Communication and Orchestration: Agents need to coordinate workflows, not just act alone. The mesh supports task planning and delegation across multiple agents, backed by specialized LLMs and deterministic execution engines to improve reliability and reduce error rates.

High-level information flow for an enterprise-grade agent ecosystem. 
Confluent

Additional components not shown include the Interaction Manager, which handles both human-agent and agent-agent communication through APIs, protocols, and chat interfaces; and the Creator Workbench, which provides the tools and scaffolding needed to design, test, and publish production-grade agents aligned with enterprise standards. Together, these capabilities turn a collection of isolated agents into a cohesive, governable system, ready for enterprise scale.

Agentic mesh: Towards enterprise-grade agents

Enterprise-grade agents must meet a high standard, one that aligns with how modern infrastructure is monitored, governed, and secured. An enterprise-grade agent is not just intelligent; it’s manageable, predictable, and safe to deploy across business-critical systems. 

Achieving all of that requires the following key attributes:

Discoverability: Agents must be easy to find, whether by users or other agents. Each one is registered with a unique identity, metadata, and clear documentation.

Security: Agents must use strong authentication and authorization, such as mTLS and OAuth2. Access is governed by zero-trust policies, where agents only interact with tools and collaborators explicitly defined in their configuration.

Observability and operability: Every agent emits metrics, alerts, and logs that can be integrated into existing enterprise monitoring and operations platforms. This enables real-time visibility and incident response.

Reliability: Enterprise agents must be designed to minimize failures. This means avoiding over-reliance on unpredictable LLM behavior and ensuring task execution is deterministic where possible.

Scalability: Enterprise agents must be able to easily scalable at run time to handle expected and peak loads. However, they must also be scalable from a development perspective, allowing developers to easily and quickly build agents. In addition, they must operationally scale by fitting into an enterprise’s operational environment.

Trust: Agents are certified before use. Certifications, automated or manual, are recorded and published for visibility and governance.

Traceability and explainability: Every action an agent takes is logged, along with the reasoning behind it. This allows teams to trace outcomes back to decisions and inputs, supporting both diagnostics and compliance.

Collaborative: Agents don’t operate in isolation. They are built to work with other agents and tools in a distributed environment, sharing context and delegating tasks when needed.

When agents meet these standards, they can be safely integrated into enterprise systems and processes. But to get there, they need infrastructure that supports these capabilities by default. That’s what the agentic mesh provides, and it’s the foundation for scaling agent adoption across the enterprise.

Technical foundations of the agentic mesh

Enterprise-grade agents need to fit into modern software infrastructure. The agentic mesh builds on well-established patterns, particularly microservices, event-driven architecture, stream processing, and zero-trust security so that agents can be deployed, observed, and managed using familiar tools and workflows.

Technical architecture for an enterprise-grade agent. 
Confluent

Agents Are Smart Microservices

Agents are microservices with an LLM brain, effectively “smart” microservices. Microservices give agents a strong operational foundation. They support enterprise-grade security standards like mTLS and OAuth2, enable reliable execution within platforms like Kubernetes, and can be easily deployed using Docker and CI/CD pipelines. Because they conform to standard observability patterns, agents can be monitored and operated using existing tools like Prometheus, OpenTelemetry, and Splunk, making them manageable within established enterprise workflows.

Agents are autonomous and tool-driven

Each agent is equipped with one or more language models and a set of tools it can invoke. Agents dynamically generate task plans based on user input and available capabilities, then execute them step by step, coordinating tools, calling APIs, and, when needed, collaborating with other agents.

Agents orchestrate conversations

Enterprise use cases often involve long-running interactions between many agents; we call these conversations. Conversations can span time frames from milliseconds, minutes, days, or longer. This means that hand-offs between agents or between people and agents must not only tolerate failures but gracefully allow additional human feedback when required by an agent.

Agents are stateful

Agents are designed to maintain and manage conversational state. This allows them to track context across multiple steps or sessions, as well as restart conversations after failures.

Agents are asynchronous

Agents are inherently asynchronous. A helpful way to understand agents is to compare them to how humans communicate. While we often engage in request-response interactions for immediate feedback, we also rely on asynchronous communication, like email or text messages, where a response might come much later. We accept that delay as part of how coordination works. Agents operate the same way. Agents may wait on tools or delegate tasks, so they’re designed to operate asynchronously.

Agents are event-driven

Event-driven architecture supports how agents operate. Instead of relying on rigid point-to-point integrations, agents must be able to discover, communicate, and subscribe to each other dynamically. Through technologies like Apache Kafka and Apache Flink, which support scalable, decoupled communication, agents tap into shared event streams, subscribe to topics, react to new data, and publish outputs in real time.

Zero-trust governance

Agents operate under strict policies that define which data, tools, and collaborators they’re allowed to interact with. Access is explicitly declared and enforced through the mesh. This prevents unauthorized actions and ensures compliance with enterprise security standards.

The future of interoperable AI

The next phase of agents in the enterprise won’t be defined by how many agents are deployed, but by how well those agents are built and managed. To deliver real business value, agents must be enterprise-grade: secure, observable, reliable, and designed to work as part of a broader system. That requires more than good prompts or clever workflows. It demands an architecture that supports governance, coordination, and control at scale.

The agentic mesh provides the foundation for that architecture, making it possible to move from experimental prototypes to production-ready systems. The future of enterprise AI lies in building agents you can trust, integrate, and scale.

Sean Falconer is AI entrepreneur in residence at Confluent. Eric Broda is an executive consultant and AI architect at IBM.

—

Generative AI Insights provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss the challenges and opportunities of generative artificial intelligence. The selection is wide-ranging, from technology deep dives to case studies to expert opinion, but also subjective, based on our judgment of which topics and treatments will best serve InfoWorld’s technically sophisticated audience. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Contact doug_dineley@foundryco.com.