Skip to content
How AI agents work· Guide 7 of 9Beginner6 min readUpdated

AI agent vs chatbot ​

A chatbot answers. You send a message, it sends text back, and a person decides what to do with that text. An AI agentAI agent: A system in which a language model decides, step by step, which tools to call and in what order, reading each result before it chooses the next step, until it can answer. acts. It decides which tools to call, reads what they return, calls more, and keeps going in a loop until the task is done or a limit stops it, and some of those calls change things in other systems. The chat window can look identical; the difference is who carries out the next step. That one change moves the risk from "the answer was wrong" to "the action was wrong", and it changes how you test, what you pay and what you must log.

It is a spectrum, not two boxes ​

Most confusion comes from treating "chatbot" and "agent" as two products. In practice there are four steps, and each one adds something:

StepWhat it can doWho acts on the result
1. Plain chatbotAnswer from the model's training and the conversationThe person
2. Chatbot with retrievalAnswer from your documents, found by search before the model is calledThe person
3. Assistant with read toolsChoose to look things up (an order, a calendar) during the answerThe person
4. Agent with write toolsChoose to change things (refund, file a ticket, push code)The software

Steps 1 and 2 are chatbots: one model call per turn, and retrieval is a fixed step the program runs, not a choice the model makes. Step 3 is where the model starts to choose its own steps through tool callingTool calling: A model's structured request to run one named tool with JSON arguments, instead of answering in text. The application runs the tool and returns the result., which is the agent loopAgent loop: The cycle an AI agent repeats: call the model, run the tools it asks for, add the results to the conversation, then call the model again until it answers. described in what an AI agent is. Step 4 is the one that changes your risk the most: the loop can now take actions with effects outside the conversation.

Anthropic's guide to building agents describes this middle ground for support: it "combines familiar chatbot interfaces with enhanced capabilities through tool integration." The interface stays a chat; what sits behind it is what you are choosing.

One turn of a chatbot and one turn of an agentTop: a chatbot turn is one straight line from the user's message to one model call to a text answer, which the user acts on. Bottom: an agent turn sends the message to a model call that may request tools; the runtime runs them, which can read or change other systems, and calls the model again until it answers.Chatbot: one call, a person actsMessageModel callText answer→ person actsAgent: a loop, the software actsMessageModel callRun toolsOther systems changeAnswertool callresult
In a chatbot, every action passes through a person. In an agent, the loop between the model and the tools can act before any person reads the answer.

What changes when you add actions ​

Risk: wrong words vs wrong actions ​

A chatbot's worst case is a bad answer: wrong, rude, leaking something it was given. That harm is real, but a person stands between the text and any consequence. An agent removes that person from some steps. OWASP's Top 10 for LLM applications names the resulting risk excessive agencyExcessive agency: OWASP's name for an LLM application that can take damaging actions because it has more functions, permissions or autonomy than its task needs.: "the vulnerability that enables damaging actions to be performed" in response to unexpected or manipulated model output, with root causes in excessive functionality, permissions or autonomy.

Prompt injectionPrompt injection: Input that makes a language model ignore or change the instructions its application gave it, so it does something the application did not intend. shows the difference most clearly. Text hidden in a web page that tells a chatbot to "say the product is free" produces a wrong answer a person may notice. The same text read by an agent with a refund tool can produce a refund. The attack is the same; the blast radius is set by the tools. That is why controls for agents sit on the actions themselves: which tools exist, which arguments are allowed, and which calls wait for a person (least privilege for agents).

Testing: the answer vs the path ​

You test a chatbot by its answers: given this question and these documents, is the reply correct, grounded and safe? One input, one output, judged on its own.

An agent needs that, plus a check on what it did. One run may reach the right answer through a tool it should never touch, so agent tests also assert on the calls: which tools ran, in what order, with which arguments, and which tools must never appear. Because the model chooses the path, run each case more than once; a single pass proves little. Agent evals covers how to build such a test set.

Cost and latency: one call vs several ​

A chatbot turn is one model call. An agent turn is as many calls as the loop takes, and each call resends the conversation, the tool definitions and earlier tool results. Workflows vs agents covers the same cost for fixed steps against model-chosen ones. Agents need a stop: an iteration limit per run, and budgets per user or team (LLM cost control).

Operations: what you must record ​

For a chatbot, keeping the conversation is usually enough to explain what happened. For an agent, you also need every tool call with its arguments and result, who or what the run acted for, and who approved what. When a customer asks why their order was cancelled, "the model decided" is not an answer; the trace and the audit log are.

The comparison ​

QuestionChatbotAgent
Model calls per turnOneOne to many, up to a limit
Who picks the next stepThe program, or the personThe model, inside the loop
Effects outside the chatNone; a person actsTool calls can read and change other systems
Worst caseA wrong or harmful answerA wrong or harmful action
Main controlsPrompt, retrieval quality, output reviewTool lists, argument rules, approvals, limits
What a test checksThe answerThe answer and the tool calls
Cost per turnPredictableVaries with the path the model takes
What to logThe conversationThe conversation, every tool call, approvals

When a chatbot is the right choice ​

A chatbot is enough when a person will act on the answer anyway: drafting an email, explaining a policy, answering from documentation. Adding retrieval (step 2) often closes the gap people expect an agent to close: the bot can now answer from your content, with sources, at the cost of one search and one model call.

Move to an agent when the task needs steps that depend on what the run finds, such as looking up an account and then deciding what to check next, and when having the software carry out those steps is worth the extra controls. Start with read tools only. Add a write tool when you can say who approves its calls, or why they need no approval.

In L2M ​

L2M builds both shapes on one canvas.

  • A chatbot is a short workflow: a Chat Trigger or webhook, optionally RAG Retrieve for documents, a Prompt Template and one LLM Call. There is no loop and no tool call; the RAG knowledge base recipe is this shape, with a fixed question as its input.
  • An agent is an Agent Orchestrator node with a chat model on its chat_model port and MCP Tool or Workflow Tool nodes on its tool port. maxIterations is required and new nodes start at 6 (agent loop).
  • Controls on actions: tool policy and the write-tool switch are optional and off until you set them (built-in content rules do check an agent's tool arguments by default). MCP tool policy rules allow, deny or require approval per tool and per argument, and a workflow's Ask before write tools switch pauses an agent before each write toolWrite tool: A tool that can change something outside the agent, such as a file, a record or a message, as opposed to a read-only tool that only fetches data. call for an approvalApproval gate: A step that pauses a run until a person approves or rejects what happens next, such as a payment, an email or a write tool call. (human approval for agent tool calls).
  • Tests on the path. Golden sets can score an agent's tool calls with the tool_sequence scorer, including tools that must never be called, not only its final answer.

Try it in L2M ​

The zero-key MCP agent recipe wires an agent to two MCP tools and runs with the built-in echo provider, which proves the wiring but never asks for a tool; attach a real chat model to see the loop call tools. To watch a recorded agent run that does call tools, open the support agent in the demo.

Further reading ​

Last updated:

This website (not the L2M product) uses Google Analytics, only after you accept cookies. Cookie settings