Designing the agent-ready data stack

Executives increasingly believe AI will reshape their businesses, but many large organizations are still stuck at proofs of concept. McKinsey’s 2025 State of AI report shows widespread experimentation, but real business value is being seen by only a small set of “high performers.” 23% of respondents report their organizations are scaling an agentic AI system somewhere in their enterprises, but the use of agents is not yet widespread. Boston Consulting Group reports that around 70% of obstacles are people and process (not model) issues, yet poor data plumbing is still a major cause of project drag.

The database bottleneck

Many engineering teams still rely on architecture optimized for transactional apps, not for AI systems that mix structured and unstructured data and live event streams. This legacy architecture has three main characteristics that slow AI adoption: rigid schemas and silos, outdated logic, and AI implemented as a bolt-on.

Rigid schemas and silos

First, current ERP and CRM systems are built around rigid schemas and operate in silos. Data warehouses, search indexes, and vector stores live in different places with different contracts. Because they use different data models and APIs, you can’t ask one system a question that assumes the logic of another without doing translation or synchronization. Because these systems were not built to understand semantics and contextual relevance, the AI layer has to glue the pieces together before it can start making sense of the data itself.

Different data storage systems present different abstractions

SystemOptimized forData modelQuery languageData warehouseAnalytical queries (e.g. total sales by region)Structured tablesSQLSearch indexKeyword or document retrieval (e.g. find documents about supply chain risk)Inverted indexSearch query syntax (Lucene, BM25, and others)Vector storeSemantic or embedding similarity (e.g. find text similar to this paragraph)Vectors (dense embeddings)Similarity search (cosine, dot product, and others)

Outdated logic

Secondly, if systems are updated only once nightly, this creates time gaps between changes in data and systems updates. Agents may be reasoning with yesterday’s data while source data may look completely different. Index drift has long-term consequences; vector stores and search indexes no longer match the reality in operational systems and on the shop floor.

Security policies can diverge too, since permissions or access controls updated in the source may not propagate immediately to the AI’s cached or copied data. Operating on outdated or inconsistent context reduces accuracy, seriously impacting compliance and trust.

AI as a bolt-on

Thirdly, because systems are not AI-native, AI is often bolted on as a sidecar, with separate security, lineage, and observability. This complicates audit and compliance, and treats AI as a separate feature set, rather than as part of a unified access, usage, and behavior trail. Operational blind spots are the consequence here. Incidents or data leaks may go undetected. Because the AI sidecar operates outside the organization’s governance framework, security teams have zero visibility into policy violations.

The RAND research organisation’s report, The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed, bears these experiences out. It highlights that organizations underestimate the data quality, lineage, access control, and deployment scaffolding needed to make AI reliable. 

Agentic AI collapses traditional data boundaries

Traditional database stacks assume clear boundaries: OLTP here, OLAP there, search elsewhere. Agentic AI use cases collapse these boundaries. Because agents need durable read-write interactions and real-time triggers, retrieval-augmented generation (RAG) must have low-latency joins across text, vectors, and graph relationships with consistent security. Legacy patterns that see data shipped to separate indexes and stores add latency, duplication, and significant risk.

The trend is convergent: bring semantic retrieval and policy closer to operational data. That’s why cloud platforms now connect vector and hybrid search into operational stores. MongoDB’s Atlas Vector Search, Databricks’s Mosaic AI Vector Search, and OpenSearch’s Neural/Vector Search are good examples. It’s also why Postgres communities extend with pgvector. However, we know that these less native “bolt-on” approaches bring their own problems. In response, a new wave of AI-native databases has the potential to close these gaps.

So, what are the technical steps engineering teams can take to prepare their systems for AI agents? And what are the choices available today?

Preparing the data layer for AI (and agents)

Migrating from one database to another is not an easy exercise. It’s complex, risky, and very expensive. The exception is where there are clear cost efficiencies to be made, but if a large company is all-in with their database stack, they’re unlikely to shift, yet.

For greenfield, AI-native projects, however, it’s important to be intentional, and choose a database model that understands what agentic systems need. Agentic systems plan, call tools, write back state, and coordinate across services, so they require specific conditions:

Long-lived memory with persistence and retrieval, not just a chat window.

Durable transactions that enable us to trust the updates agents issue.

Event-driven reactivity that supports subscriptions and streams to keep UIs and other agents synchronized.

Strong governance including low-level security, lineage, audit, and PII (personally identifiable information) protection.

Major frameworks emphasize these data requirements. LangGraph and AutoGen add persistent, database-backed memory, while Nvidia and Microsoft reference architectures center on data connectors, observability, and security in agent factories. Meanwhile, Oracle’s latest database release ships agent tooling into the core. This isn’t a model problem—it’s a state, memory, and policy problem.

Here’s what “done right” looks like when you rebuild the data layer for AI:

First, build for adaptability. This means first-class support for mixed data (relational, document, graph, time series, vector) and flexible schemas, so AI can reason over entities, relationships, and semantics without struggling with brittle ETL.

Next, commit to openness. Standard interfaces, open formats, and open-source participation allow teams to combine best-of-breed embedding models, re-ranking tools, and governance. The added bonus is avoiding vendor lock-in.

Finally, embrace composability. Build in real-time subscriptions and streams, functions close to data, and unified security so that retrieval, reasoning, and action run against one trustworthy source of truth.

Which model for my use case?

Every organization works with a mix of (usually open-source) databases, all optimized for different workloads. For instance, MySQL and PostgreSQL handle transactional data, while MongoDB and Couchbase offer document storage suited to dynamic application data. Together, this mix of databases form a polyglot persistence layer where teams select the right tool depending on the scenario.

Companies have invested heavily in creating the right mix of databases for their needs, so how can teams bring AI to existing stacks? Is this the right approach and what are the alternatives? And, are companies facing a complete database refactor to bring AI agents to their companies?

Unified operational stores with vector and hybrid search

MongoDB Atlas, Databricks Mosaic AI, and OpenSearch put approximate-nearest-neighbor and hybrid retrieval next to data, reducing sync drift. Postgres offers pgvector for teams standardizing on SQL. Oracle Database 23ai and 26ai add native vector search and agent builders into the core RDBMS, reflecting the shift toward AI-aware data layers.

This approach is a good fit for simple AI projects that rely only on one data tool (e.g. only MongoDB, or only OpenSearch, or only Postgres). But the reality is, enterprise AI systems and agents typically rely on multiple data sources and tools, and being able to search and retrieve data across a variety of databases is difficult. Being able to store the data natively in one place that can support a mix of data models and search natively across all of those data models allows teams to harness the power within their data for building AI systems.

Purpose-built vector databases

Pinecone, Weaviate, and Milvus focus on vector scale and latency; many enterprises pair them with operational databases when they need specialized retrieval at scale. This is great when embedding and vector search is a key, large‐scale workload, requiring high performance and advanced vector features. The downside is that you need to manage and operate another, separate database system.

Multi-model databases

SurrealDB is one concrete approach to this convergence It’s an open-source, multi-model database that combines relational, document, graph, and vector data with ACID transactions, row-level permissions, and live queries for real-time subscriptions. For AI workloads, it supports vector search and hybrid search in the same engine that enforces company governance policies, and it offers event-driven features (LIVE SELECT, change feeds) to keep agents and UIs in sync without extra brokers.

For many teams, this reduces the number of moving parts between the system of record, the semantic index, and the event stream.

What integrating with AI feels like today (and how it should feel tomorrow)

Trying to use AI in traditional environments is pretty painful. Engineering teams routinely face multiple copies of data, which leads to drift and inconsistent access control lists. The job of embedding refresh cycles and index rebuilds results in latency spikes and degraded quality. And, separate policy engines lead to audit gaps across chat, retrieval, and actions.

If we have an AI-ready data layer, we can store entities, relationships, and embeddings together, and query them with one policy model. We can also use real-time subscriptions to push changes into agent memory, not nightly backfills. And we can enforce row-level security and lineage at the source, so every retrieval is compliant by default.

This isn’t hypothetical. Public case studies show tangible results when teams collapse data sprawl. For example, LiveSponsors rebuilt a loyalty engine and cut query times from 20 seconds to 7 milliseconds while unifying relational and document data. Aspire Comps scaled to 700,000 users in eight hours after consolidating back-end components. Many companies cite major gains in consolidation and AI-readiness.

Principles for AI-ready architecture

AI doesn’t stall because models are “not good enough.” It stalls because data architecture lags ambition. The fastest path from pilot to profit is to modernize the database layer so retrieval, reasoning, and action happen against one governed, real-time source of truth. There are a few essential considerations that should create the right conditions for AI agents to fulfill their potential.

Design for retrieval and relationships

Treat graph, vector, and keyword as a first-class trio. Knowledge graphs paired with RAG are becoming a standard for explainable, lineage-aware answers.

Co-locate state, policy, and compute

Keep embeddings next to the system of record and push policy (role-based access control and row-level security) into the database to minimize data hops.

Make memory durable

Agents need persisted memory and resumable workflows, supported by frameworks (e.g. LangGraph and AutoGen) and enterprise “AI factory” designs from Nvidia and Microsoft.

Prefer open building blocks

Open-source options (pgvector, Weaviate, Milvus, OpenSearch) de-risk lock-in and accelerate learning curves, especially when paired with an open operational database.

Prepare the environment

Then, there are three important practical steps to take to prepare the environment:

Start with the bottlenecks. Inventory the extra hops between your app, your vectors, and your policies, then remove them.

Adopt a unified, AI-aware data layer. Whether you evolve your incumbent platforms or adopt a unified engine (e.g. SurrealDB), collapse silos and co-locate semantics, relationships, and state.

Measure business impact in milliseconds and dollars. Look for latency, accuracy, and productivity movement. In the wild, we’re seeing that sub-10-millisecond retrieval and significant simplification of stacks translate into feature velocity and cost savings. Public case studies from SurrealDB customers such as LiveSponsors, Aspire, and a collaboration between Verizon, Samsung, and Tencent illustrate both the technical and organizational dividends when the data layer is simplified.

Databases have always been at the core of traditional software applications. With the rise of agentic AI, the role of databases must evolve to serve as the “agentic memory” at the core of reliable agentic systems. The question isn’t whether to rethink data for agents, but how quickly to equip agents with the memory they need to drive decision velocity.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.