How AI agents work

When generative AI (genAI) suddenly burst onto the tech scene with the arrival in late 2022 of OpenAI’s ChatGPT, companies quickly embraced its potential for automating tasks such answering customer inquiries, handling support tickets, and generating content.

A slew of rival chatbots followed ChatGPT. But they tended to be static tools; they didn’t learn from user interactions or application integrations. Only their foundational large language models (LLMs) could be trained.

Enter agentic AI. By leveraging technologies such as machine learning, natural language processing (NLP), and contextual understanding, AI agents can operate independently, even partnering with other agents to perform complex tasks.

“Think virtual coworkers able to complete complex workflows,” McKinsey & Co. explained in a report. “The technology promises a new wave of productivity and innovation.”

According to the 2025 Connectivity Benchmark Report from Mulesoft and Deloitte, 93% of IT leaders plan to introduce autonomous AI agents within two years — and nearly half have already implemented them.

Like chatbots, AI agents have existed since the 1960s. However, it wasn’t until advances in AI, ML, deep learning, and transformer models (such as GPT-3 and ChatGPT) that they became capable of adapting to tasks and learning from data. That dramatically expanded their use cases.

Agentic AI systems typically use a transformer-based LLM as the core, enhanced with reasoning, memory, reinforcement learning, and tool integrations. The LLM’s understanding of language allows it to interpret instructions and generate responses.

In the simplest terms, an AI agent is the combination of an LLM and a traditional software application that can act independently to complete a task. They can operate autonomously, make decisions, plan, and take actions to achieve specific goals without constant human oversight.

“This is a way to deliver business value, and I think that is where the focus should be, to think about how you’re going to disrupt the business process,” said Samta Kapoor, a principal on Ernst & Young’s tech consulting team.

For example, if an employee requests vacation time, an AI agent can automate the process of entering the dates into the HR system and ensuring all other systems are aware that employee will be away for the specified time. If the employee changes his or her mind and enters new dates, the agent can reschedule everything in the HR system autonomously. All it takes is a simple set of commands and away the AI agent goes, Kapoor said.

AI agents can also autonomously write software code and offer that base code to a developer, who can then review it for accuracy and modify it if necessary. But there are also agents that can perform the code review, as well. And, best of all, it can all be done in seconds, not hours or days.

AI-assisted code generation tools are increasingly prevalent in software engineering and, somewhat unexpectedly, have become low-hanging fruit for most organizations experimenting with generative AI (genAI) tools. Adoption rates are skyrocketing, because even if they only suggest a baseline of code for a new application, automation tools can eliminate hours that otherwise would have been devoted to manual code creation and updates.

By 2027, 70% of professional developers are expected to be using AI-powered coding tools, up from less than 10% in September 2023, according to Gartner Research. And within three years, 80% of enterprises will have integrated AI-augmented testing tools into their software engineering tool chain — a significant increase from approximately 15% early last year, Gartner said.

Beyond coding, AI agents are designed to perceive their surroundings, make decisions based on that information, take actions, and sometimes learn and adapt over time to perform tasks autonomously. Reinforcement learning is key to agentic AI’s ability to continue to grow in sophistication when performing tasks.

“If you’re playing a game, you either win or lose. If you lose, you go back and evaluate why, and then play again but do it differently,” Kapoor said. “With agentic AI, there are a very defined set of KPIs that you’re asking it to meet, so it would know whether it has met them or not. And then it goes back and it reinforces itself to do this task differently.”

For agentic AI, decision-making is structured around autonomy and goal-orientation. “There is a reward system within agentic AI and this is frequently based on reinforcement learning, where the AI learns to maximize rewards through interactions with its environment,” said Arun Gururajan, NetApp’s vice president of research and data science.

The sense-think-act process and agent types

Agentic AI, Gururajan said, follows a cyclical sense-think-act process, which has the following steps:

Perception: The system gathers input from its environment and/or the user.

Reasoning and Planning: The central brain of the agent, typically a powerful LLM, reasons through the task and generates and evaluates possible actions.

Decision-making: Reinforcement learning strategies, often supplemented by human feedback as well as the memory of past interactions, help select the optimal action.

Execution: The chosen action is carried out, possibly by calling on internal/external tools via API integrations.

Feedback loop: Outcomes are assessed and used to refine future decisions, creating a continual learning process.

There are several types of AI agents that can be employed based on the complexity of the task. They include:

Reactive agents: These only respond to their environment based on predefined rules. They don’t store history or learn from it (e.g., simple game AI). The most basic of agents, they’re used in customer service bots or smart home devices that can adjust themselves automatically.

Deliberative agents: These use an internal model and reasoning to make informed, long-term decisions. They’re used in applications such as autonomous vehicles, supply chain management, and medical decision systems.

Hybrid agents: These combine reactive and deliberative approaches for more efficient decision-making. For instance, a robot might react to immediate obstacles and plan its path to a goal simultaneously. Hybrid AI is used in automating business tasks, where reactive agents handle routine actions (e.g., responding to emails) while deliberative agents plan and optimize workflows for efficiency over time.

In short, hybrid agents integrate both immediate reaction and thoughtful planning in their decision-making.

“Traditional AI — or predictive AI — is often tuned to solve a narrow and specific problem — for example, predicting drive failure in storage systems,” Gururajan said. “Agentic AI is more dynamic; It can adapt, reason and strategize.”

Imagine, for example, agents managing a data storage system by monitoring dashboards, identifying bottlenecks, predicting failures, and proactively taking action to prevent errors, ensuring system SLAs are met.

NetApp, for instance, sets up reward models based on objectives (such as maximizing uptime or minimizing energy use) that combine human preferences, real-time data, and instructions, enabling AI to optimize behavior and improve performance over time, according to Gururajan.

Reasoning techniques like Chain-of-Thought prompting, which mirrors human thought, or ReAct prompting help break down tasks and plan actions. Memory modules store context and intermediate results for tasks requiring continuity. Reinforcement learning with human feedback fine-tunes the system’s outputs to align with human values. Additionally, tool integrations enable the AI to perform complex tasks beyond text generation, such as web search and interacting with APIs.

The growing use of API integrations

API integrations with AI agents are currently the pinnacle of use cases. In agentic AI, tools via API integrations allow agents to interact with the real world. When a task requires external information, the agent generates an API call, formats parameters, authenticates, and processes the response to complete the task or take further action.

“When an agent needs to perform a task that requires external information, such as searching a database, sending an email, executing another ML model,”  Gururajan said, “it generates an API call based on its understanding of the task and the API’s documentation.

Executing on an API involves formatting aspects with the correct parameters and authenticating with the API, which in turn, returns data (or performs actions); the agent processes the response and completes the task or takes subsequent actions if needed, Gururajan explained.

Looking ahead, there are still improvements needed for agentic AI to mature, such as addressing challenges with API discoverability and adaptation, and dealing with issues such as a lack of standardization and documentation, Gururajan said.

Change management also makes it difficult for agents to select the right APIs. And API security and authentication remain crucial, requiring robust protocols and access control to protect sensitive data. Implementing service-level credentials could provide more granular control, such as restricting agents to read-only access or specific actions.

There is emerging research involving agents, such as multi-objective optimization, which focuses on solving conflicting task goals using goal-based programming. Additionally, system-level heuristics can be created as general rules reflecting core principles, constraints, or safety measures.

Heuristics can be incorporated into the agentic framework by: (a) filtering goals (such as removing goals requiring restricted data), (b) modifying objectives (insuring safety overrides efficiency), and (c) integrating reinforcement learning to weight goals.

Looking ahead, there’s a need for agents to autonomously create their own APIs for tasks, as most agents currently rely on pre-existing ones. “This would be a positive step towards Artificial General Intelligence or AGI,” Gururajan said.