Multi-Agent Systems: When to Use Multiple AI Agents (and When Not To)

Use a multi-agent system when the work splits into independent pieces that can run in parallel, each piece produces a lot of context the others do not need, and the task is valuable enough to pay several times the tokens. For sequential work where every step depends on the last, one agent with good tools usually wins.

Below: the evidence on both sides, a decision table with thresholds, worked cost math, the orchestration patterns that hold up, and A2A protocol status as of October 2026.

When should you use multiple AI agents? What the evidence says

The strongest case for multi-agent systems comes from Anthropic’s write-up of its Research feature (June 2025). A lead agent on Claude Opus 4 that handed work to Claude Sonnet 4 subagents beat a single Opus 4 agent by 90.2% on Anthropic’s internal research eval. Running 3 to 5 subagents in parallel, each calling tools in parallel, cut research time by up to 90% for complex queries.

The mechanism is capacity. On the BrowseComp browsing benchmark, Anthropic found token usage alone explained 80% of the variance in performance. Each subagent gets its own context window, so the system can spend more tokens on the problem than one window holds. The same post names the limits: domains that “require all agents to share the same context or involve many dependencies between agents are not a good fit,” and “most coding tasks involve fewer truly parallelizable tasks than research.”

The largest controlled study so far points the same way. Google Research’s Towards a Science of Scaling Agent Systems (version 3, April 2026) tested 260 configurations across six benchmarks, five architectures, and three model families, with tools, prompts, and compute held equal. Against a single agent, results ranged from +80.8% on decomposable financial reasoning to -70.0% on sequential planning. All multi-agent variants lost 39% to 70% on the planning benchmark. The authors also report a fitted threshold of about 45%: once a single agent scores above it, adding agents tends to bring negative returns, and that tool-heavy tasks paid a coordination penalty.

My read: multi-agent is a bet on task shape. Breadth-first work that splits cleanly wins. Step-by-step work loses, sometimes badly.

Why do multi-agent systems fail?

The best-known argument against them is Cognition’s Don’t Build Multi-Agents (June 2025). Walden Yan gave two principles: share full agent traces, not just messages, and remember that “actions carry implicit decisions, and conflicting decisions carry bad results.” His example: split “build a Flappy Bird clone” across two subagents and one builds a Super Mario style background while the other builds a bird that does not match. His 2025 conclusion was that collaborating agents produce fragile systems, so use a single-threaded agent.

The research backs the concern. Why Do Multi-Agent LLM Systems Fail? (latest version October 2025) built the MAST taxonomy from 150 traces with expert annotators (inter-annotator agreement κ = 0.88), then labeled 1,642 traces from seven open-source frameworks. It found 14 failure modes in three groups: system design issues, inter-agent misalignment, and task verification. Failure rates across seven leading open-source systems ran from 41% to 86.7%. The authors note that gains over single agents or simple best-of-N sampling “often remain minimal.”

Two fixes in that paper show where the gains come from. The paper reports +9.4% task success from a role and workflow change in ChatDev. Adding a high-level check of the task objective raised it 15.6% on their ProgramDev set. Both are design changes, not model changes. The authors conjecture that better base models alone will not remove these failures.

Anthropic hit the same class of problems. Its early research agents spawned 50 subagents for simple queries, searched endlessly for sources that did not exist, distracted each other with too many updates, and ran duplicate searches.

Detect it early

These are my alert rules, mapped to the failure modes above. Track identical tool calls with identical arguments inside one run; more than two repeats is step repetition. Count handoffs per conversation; more than three is a routing loop. Log the overlap between subagent queries; overlap above about 20% means the orchestrator wrote vague task boundaries. Record why each run stopped, because runs that hit the step cap mean nobody defined termination. Wire these into your agent observability tooling before launch, not after the first bad week.

Single-agent vs multi-agent decision table

Score your workflow against each row. If most answers land in the left column, build one agent and stop. Thresholds come from the cited sources where noted; the rest is my judgment, and all of them will move as models improve.

One more rule from Anthropic’s January 2026 guide: splitting agents by role (planner, implementer, tester, reviewer) is “often counterproductive.” Each handoff loses context, and in one experiment the subagents spent more tokens coordinating than working. Split by context boundary instead: the agent that builds a feature also writes its tests. The same guide says “teams invest months building elaborate multi-agent architectures only to discover that improved prompting on a single agent achieved equivalent results.”

Microsoft’s guide frames it as a ladder: a direct model call, then a single agent with tools (“often the right default for enterprise use cases”), then multi-agent orchestration. Climb only when the rung below fails your eval. The agent architecture guide covers that single agent’s components.

What does a multi-agent system cost? A worked comparison

Anthropic reported agents use about 4 times the tokens of chat and multi-agent systems about 15 times. Its 2026 guide puts multi-agent at “3-10x more tokens than single-agent approaches for equivalent tasks.” Here is what that looks like in dollars, using current Claude API prices checked October 15, 2026.

This example is hypothetical. A team produces vendor due-diligence briefs covering four independent areas: financials, security incidents, litigation, and sanctions. That is breadth-first work, the shape where multi-agent should win.

Option A: one Sonnet 5.5 agent. Across the run it writes 120,000 new tokens to the 5-minute cache ($2.50 per million), reads 480,000 cached tokens ($0.10 per million), and produces 15,000 output tokens ($10 per million). Cost: $0.30 + $0.048 + $0.15 = $0.498 per brief.

Option B: an Opus 5.5 lead with four Sonnet 5.5 subagents. Each subagent writes 80,000 tokens to cache, reads 320,000, and outputs 10,000: $0.20 + $0.032 + $0.10 = $0.332, or $1.328 for four. The lead writes 60,000 ($5 per million), reads 140,000 ($0.20 per million), and outputs 12,000 ($20 per million): $0.30 + $0.028 + $0.24 = $0.568. Cost: $1.896 per brief.

Option B uses 1,852,000 tokens against 615,000, about 3.0 times as many, and costs 3.8 times as much. That sits inside Anthropic’s 3 to 10 times range.

Now add failures. Say an analyst needs 45 minutes at a $60 hourly loaded rate to fix a bad brief, so $45 per failure. If Option A passes 80% of briefs and Option B passes 90%:

Option B wins. The break-even is the extra token cost divided by the fix cost: $1.398 / $45 = 3.1 points of pass rate. If Option B lifts the pass rate by only 2 points, it loses ($9.996 vs $9.498). If a failure costs $5 to fix (an automatic retry, say), Option B needs a 28-point lift, more than the 20 points left above Option A’s 80%, so it cannot win. At 2,000 briefs a month, the extra tokens cost $2,796.

Sensitivity check: putting the lead on Sonnet 5.5 too drops its cost to $0.284 and Option B to $1.612, so the break-even falls to 2.5 points at $45 per fix. The rule holds either way. Multi-agent pays only when the pass-rate lift times the cost of a failure exceeds the extra token spend. Measure both on your own cases before you build. Ways to cut the token side are in reducing agent token costs.

Orchestration patterns: orchestrator-worker, handoffs, and the rest

If the table says multi-agent, pick the simplest pattern that fits. Anthropic’s Building effective agents defines orchestrator-workers as “a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results,” and says it fits “complex tasks where you can’t predict the subtasks needed.” That is the research system above. If you can predict the subtasks, you do not need an LLM to plan them. Fan them out in code.

The OpenAI Agents SDK docs draw the line people most often blur. With agents as tools, “a manager agent keeps control of the conversation” and calls specialists for bounded subtasks. With a handoff, the specialist “becomes the active agent for the rest of the turn.” Use agents as tools when one agent must own the answer. Use handoffs when a specialist should take over, as in support triage. The same docs note that orchestrating in code “makes tasks more deterministic and predictable, in terms of speed, cost and performance.”

Microsoft’s guide covers the rest, with the failure to watch for each:

Cognition’s April 2026 follow-up, Multi-Agents: What’s Actually Working, narrows the field to patterns where “agents contribute intelligence while writes stay single-threaded.” That covers a reviewer agent with a clean context, a “smart friend” call to a stronger model, read-only search subagents, and a manager that splits work and synthesizes results. Yan says his original observations “still hold today for parallel-writer swarms.” My recommendation is to start there: one writer, any number of readers and reviewers. Framework choices for each pattern are in choosing an agent framework and the LangChain, LlamaIndex, and CrewAI comparison.

Where does the A2A protocol stand in October 2026?

A2A (Agent2Agent) is an open protocol for agents to discover and call each other over a network, across vendors. Google introduced it in April 2025. The Linux Foundation took it on on June 23, 2025, citing support from more than 100 companies. The v1.0.0 specification shipped on March 12, 2026, and v1.0.1 on May 28, 2026, the latest release as of October 15. On August 27, 2026, the project announced it was accepted as a Growth Stage project at the Agentic AI Foundation, a Linux Foundation body, with backing from over 150 organizations. Its technical steering committee includes AWS, Cisco, Google, IBM Research, Microsoft, Salesforce, SAP, and ServiceNow.

Version 1.0 broke compatibility with 0.3. Signature handling for Agent Cards (the discovery document an agent publishes) was formalized with JWS and RFC 8785 canonicalization, the three bindings (JSON-RPC, gRPC, HTTP+JSON) were formally specified, and enum values, operation names, and pagination all changed. Pre-1.0 integrations need migration work.

A2A does not replace MCP. The A2A site calls them “highly complementary”: MCP connects an agent to tools and data, and A2A connects agents to agents. See what MCP is for the tool side.

Decision rule: if all your agents live in one codebase, you do not need A2A. A function call or your framework’s handoff is simpler and easier to trace. Consider A2A when agents are owned by different teams or vendors, run as separate services, and must be found and called at runtime. Even then, the coordination failures in MAST do not go away because the wire format is standard.

Bottom Line

Default to one agent with good tools and a real eval. Go multi-agent only for breadth-first work that splits cleanly, and keep writes with one agent. Price it as pass-rate lift times failure cost versus 3 to 10 times the tokens. Instrument repeats, handoff loops, and overlap from day one. Use A2A only when agents cross team or vendor boundaries.

FAQ

How many agents should a multi-agent system have?

Fewer than you think. Anthropic’s research system tells its lead agent to use 1 agent with 3 to 10 tool calls for simple fact-finding, 2 to 4 subagents with 10 to 15 calls each for direct comparisons, and more than 10 subagents only for complex research. Microsoft suggests capping group chat orchestration at three or fewer agents.

Do multi-agent systems work for coding?

Mostly as reviewers and researchers, not parallel writers. Cognition reports that on pull requests written by Devin, its Devin Review agent still catches an average of 2 bugs per PR, roughly 58% of them severe. It works best when the coding and review agents share no context beforehand. Cognition says parallel agents taking write actions still make conflicting implicit choices.

What is the difference between MCP and A2A?

MCP connects one agent to tools and data sources. A2A lets independent agents discover each other through Agent Cards and exchange tasks, messages, and artifacts. A2A 1.0 also removed the OAuth implicit and password flows and added the device code flow, so check your auth setup before upgrading.

How do you test a multi-agent system?

Start small. Anthropic began with about 20 real queries and found a single LLM-judge call scoring 0.0 to 1.0 with a pass-fail grade worked best, while manual testing stayed essential. Microsoft recommends rubric or judge scoring instead of exact-match assertions, because agent outputs vary run to run. The agent evaluation guide covers building the case set.

Three Ways to Work With TMA

Need an agent built? We deploy production AI agents in your infrastructure. Working pilot. Real data. Measurable ROI. → Schedule Demo

Want to co-build a product? We’re not a dev agency. We’re co-builders. Shared cost. Shared upside. → Partner with Us

Want to join the Guild? Ship pilots, earn bounties, share profit. Community + equity + path to exit. → Become an AI Architect