AI Agents vs Chatbots: The Real Difference and When Each Wins

The difference between an AI agent and a chatbot is who controls the steps. A chatbot answers turn by turn: you ask, it replies, and you decide what happens next. An AI agent gets a goal, and the model decides its own next steps and tool calls, in a loop, until the task is done or it hands off.

Write access is a separate line, and it drives risk. An agent can be read-only, and fixed automation can write records. The AI agents vs chatbots choice gets serious when one system has both.

What is the difference between an AI agent and a chatbot?

A chatbot handles one turn at a time. It may follow a script, generate its reply with an LLM, or run a lookup the developer wired in. Then it waits. It does not plan its own sequence of tool calls.

An agent runs the sequence itself. OpenAI’s practical guide to building agents defines agents as “systems that independently accomplish tasks on your behalf.” Apps that do not use the LLM “to control workflow execution,” such as “simple chatbots, single-turn LLMs, or sentiment classifiers,” are “not agents.” Anthropic’s Building effective agents says agents are “systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.” It contrasts them with workflows, “orchestrated through predefined code paths.” NIST’s AI standards center, in a January 2026 request for information, calls agent systems “capable of planning and taking autonomous actions that impact real-world systems or environments.”

So the test is one question: who picks the next step and the next tool call? A person or a script: chatbot or workflow. The model, in a loop toward a goal: agent.

Write access is the second question, and it is about risk, not the label. OpenAI’s own starter example is a weather agent whose only tool is get_weather: read-only. RPA bots write records all day without being agents. The two questions give four boxes:

Degrees of autonomy are covered in the copilot vs agent autonomy ladder.

AI agents vs chatbots, side by side

The loop drives most rows: more calls, more cost, more steps that can fail. Write tools decide how bad a failure gets. A hijacked read-only agent misreports. A hijacked write-capable agent acts.

Decision rule: look at a run trace. If every run calls the same tools in the same order, it is a chatbot or a workflow, whatever the label says. If the model chose the calls, it is an agent. Then ask which of those tools can write.

Why the “agent” label on a vendor page proves nothing

In a June 2025 release, Gartner named the problem “agent washing”: rebranding assistants, RPA, and chatbots “without substantial agentic capabilities.” It estimated only about 130 of the thousands of agentic AI vendors are real, an analyst estimate rather than an audit.

The better line is the advice. Gartner analyst Anushree Verma said companies can start “by using AI agents when decisions are needed, automation for routine workflows and assistants for simple retrieval.” Anthropic agrees: add complexity only when needed, which “might mean not building agentic systems at all.”

My recommendation: skip the label and ask the vendor for three things. First, a trace of a real run. If every run calls the same tools in the same order, it is what Anthropic calls a workflow, which may be fine and cheaper. Second, the tool list with read or write scope on each, which tells you the risk. Third, what stops it: limits, approvals, a halt switch. The failure mode is paying agent prices for chatbot capability, or write access with no controls.

Risk and security: what write access changes

A chatbot’s risk is real but indirect. In Moffatt v. Air Canada (February 14, 2024), Air Canada’s website chatbot suggested a grieving customer could apply for bereavement fares retroactively, which the airline’s policy did not allow. Air Canada argued, in effect, that the chatbot was responsible for itself. The tribunal called that “a remarkable submission” and wrote that “it makes no difference whether the information comes from a static page or a chatbot.” The harm still passed through a person.

A write-capable agent removes that person. OWASP’s LLM06:2025 Excessive Agency (LLM03 in the 2026 edition) describes “damaging actions to be performed in response to unexpected, ambiguous or manipulated outputs from an LLM,” caused by excessive functionality, permissions, or autonomy. With write tools, model output becomes a command.

Tool access also raises what prompt injection buys an attacker. NIST’s agent hijacking evaluation (January 2025) tested agents built on the October 2024 Claude 3.5 Sonnet. New red-team attacks raised attack success from 11% for the strongest baseline attack to 81%. Models have improved since. Nobody claims the problem is solved.

Simon Willison’s lethal trifecta is a quick check: private data, untrusted content, and a way to communicate externally. Add an email tool to a retrieval bot and it has all three.

Decision rule: never ship all three legs without a hard block, in code, on at least one. See the prompt injection guide and least-privilege design.

How you evaluate an agent vs a chatbot

You grade a chatbot on its answer. You grade an agent on the outcome, which Anthropic’s January 2026 guide to agent evals defines as “the final state in the environment at the end of the trial.” Its example: a flight-booking agent says it booked, but the test is whether the reservation exists.

Consistency matters too. The τ-bench paper (June 2024) introduced pass^k, the chance an agent succeeds on all k tries of the same task. Top function-calling agents such as GPT-4o then succeeded on under 50% of tasks, and pass^8 in retail was under 25%. Current models score higher; the metric is the lesson. Anthropic’s guide gives the arithmetic: 75% per try means about 42% for three in a row (0.75 cubed).

Steps compound the same way. Hypothetically, with independent errors, six steps at 98% each succeed 88.6% of the time (0.98 to the sixth); at 95%, 73.5%.

Decision rule: never sign off from a demo. Check task success from the end state over repeated runs. The agent evaluation guide shows how to write those tests.

How much more an agent costs to run

Anthropic reports its agents “typically use about 4× more tokens than chat interactions,” and multi-agent systems “about 15× more.” The mechanism: each loop step resends the instructions, tool definitions, and every earlier result.

A hypothetical worked example, using Sonnet 5.5 at $2 per million input tokens and $10 per million output, per Anthropic’s pricing page on October 15, 2026:

• Chatbot answer: one call, 4,000 input and 300 output tokens. $0.008 + $0.003 = $0.011.
• Agent task: six calls. The first sends 5,000 tokens, and each later call adds 1,200 (a 1,000-token tool result plus the prior 200-token output). That is 48,000 input and 1,200 output tokens. $0.096 + $0.012 = $0.108, about 10 times the chatbot.

That is 49,200 tokens against 4,300, or 11.4 times the tokens for 9.8 times the cost. These ratios reflect this example’s assumptions and prices, not Anthropic’s token figure. The ratio grows faster than the step count because each call rereads the earlier ones: three calls cost about $0.043 (3.9 times), ten about $0.228 (21 times). Prompt caching cuts the repeated part.

A runaway loop is the bigger risk. In McKinsey’s 2026 State of AI survey (1,719 respondents, May 4 to June 8, 2026), about one in five said operating costs, including tokens, constrained their AI use. Set a step cap and a per-task token budget. Alert at twice the median.

What changes in governance

EU AI Act Article 50(1) requires AI systems that interact directly with people to tell them they are dealing with AI, unless that is obvious. Per Ropes & Gray’s August 2026 analysis, the Digital Omnibus left 50(1) “untouched,” so it applies from August 2, 2026. Customer-facing agents are covered like chatbots. I am not a lawyer.

Write access adds a second layer. OpenAI’s guide says actions that are “sensitive, irreversible, or have high stakes should trigger human oversight until confidence in the agent’s reliability grows,” such as canceling orders, large refunds, and payments. So a write-capable agent needs its own identity with scoped permissions, an approval rule per risky write, an action log, a stop switch, and a named owner.

Decision rule: a chatbot needs a content owner who keeps answers correct. A write-capable agent needs a process owner who answers for what it did. No owner, no write access. The escalation design guide covers the handoff that owner runs.

Same request, chatbot vs agent: three hypothetical examples

1. “My order hasn’t shipped. Can you send it to my office instead?” The chatbot sees the order is in the warehouse and links the address-change form. The agent verifies the customer, confirms the order is not picked, updates the address, and confirms. If the box has left, it escalates. New requirement: write access to orders and a rule for which changes need review.

2. “I’m locked out of the VPN.” The chatbot walks through self-service reset steps. The agent confirms identity with a multifactor push, reads the lock reason, unlocks the account if the cause was failed passwords, and logs a ticket. A security-tool lock goes to a person. New requirement: an agent that can unlock accounts is a social-engineering target, so the identity check lives in code, not the prompt.

3. “Why was invoice 4471 short-paid?” The chatbot, with retrieval over the payables policy, lists common reasons. The agent pulls the invoice, purchase order, and goods receipt, finds 40 units received against 50 billed, drafts a supplier note, and prepares a debit memo for approval. Up to the finding, this agent is read-only. The memo is where write access starts, so it needs draft rights and an approval before anything posts.

Each time, the chatbot answers one turn and stops. The agent chooses its own lookups, and only some steps write.

Do you need an agent? A yes/no decision test

Answer for one workflow. Questions 1 to 4 test need, 5 to 8 readiness. The thresholds are mine.

How to read it. A “no” on 1 ends the test. A “no” on 2 points to traditional automation. Yes on 1 to 4 but a “no” on 5 to 8: ship a chatbot with a handoff and close the gaps. All yes: pilot the agent, read-only first where you can, with approvals on every write.

When a chatbot is the better choice

A chatbot wins when the value is in the answer: policy and product questions, how-to help, and intake that ends with a human decision. It wins when the target system has only coarse permissions, so an agent would need more access than the task deserves. It wins when errors are cheap to catch by reading but expensive as actions: a wrong refund policy can be questioned, a wrong refund has already gone out. And it wins at low volume, because evals, approvals, and ownership are fixed costs.

McKinsey’s 2026 survey found chatbots are the most widely scaled AI tool: 47% of respondents say their organizations are scaling them across the enterprise, against about two in ten for AI agents. The customer service guide covers where support agents fit.

Watch for quiet drift. A chatbot gets “create ticket,” then “update order,” then “issue credit,” and nobody repeats the security review. Count write tools on every bot each quarter. Above zero, it needs write controls, whatever you call it.

Bottom Line

A chatbot answers turn by turn. An agent runs its own steps toward a goal. Write access is what makes either one dangerous.

Pick the chatbot unless the steps after the answer are the cost to remove. If you build an agent, test outcomes, cap the loop, block one leg of the trifecta, and name a process owner for every write.

Run the eight-question test on one workflow this week.

FAQ

Is a RAG chatbot an AI agent?

Usually not. A RAG chatbot runs one retrieval per turn on a path the developer fixed. Anthropic calls an LLM with retrieval, tools, and memory “the augmented LLM,” the building block of agents. When the model decides what to search, whether to search again, and when it is done (“agentic RAG”), it is an agent, even with no write access.

Who is liable when a chatbot gives a wrong answer?

In the Air Canada case, the company. The tribunal ordered it to pay C$650.88 in damages, C$812.02 with interest and fees. Small sum, clear rule.

Can a chatbot be upgraded into an agent later?

Yes. Add a step cap, redo the security review per write tool, switch evals to outcome checks, rebuild the cost model, and assign a process owner.

Does the EU chatbot disclosure rule apply to AI agents?

Yes, for agents that interact directly with people. A back-office agent that never talks to a person generally falls outside Article 50(1). The Omnibus’s only Article 50 extension covers content marking in 50(2), for generative systems already on the market, until December 2, 2026.

Three Ways to Work With TMA

Need an agent built? We deploy production AI agents in your infrastructure. Working pilot. Real data. Measurable ROI. → Schedule Demo

Want to co-build a product? We’re not a dev agency. We’re co-builders. Shared cost. Shared upside. → Partner with Us

Want to join the Guild? Ship pilots, earn bounties, share profit. Community + equity + path to exit. → Become an AI Architect