Interview knowledge
Article

šŸ§± Your Multi-Agent Architecture Is a 2016 Microservices Diagram With the Boxes Renamed

by Byte-Sized Design12 Sept 2026

Multi-agent systems are microservices where every call runs an LLM. Which patterns transfer, which break, and one heuristic for deciding when to split agents.

Read the original ā†’

Highlights

Passages saved from the original.

  • Router. A cheap classifier reads the request and forwards it to exactly one specialist, one hop with no synthesis step.
    • Use for: support triage, intent routing in a product assistant, any case where the categories are known and the work per category is self-contained
    • Router model: your cheapest one, a fine-tuned classifier, or a regex if the categories are clean enough
    • Common mistake: letting the router ā€œhelpā€ by adding context. It should classify and get out of the way
  • Orchestrator-worker. One capable model decomposes the task, dispatches subtasks to workers, and synthesizes the results. Workers are stateless, parallel, and can run on a different model than the orchestrator.
  • This is where the cost story lives:
    • Orchestrator-tier and worker-tier models from the same vendor sit 5x to 15x apart on per-token price
    • In a decomposed task, most tokens are worker tokens
    • The number circulating in vendor write-ups this year is a 40 to 60 percent cost reduction versus running everything on the frontier model. Those write-ups are selling something, but the mechanism is sound
    • Anthropicā€™s public data point: an Opus lead with Sonnet subagents beat single-agent Opus by 90.2 percent on their research eval. The same post reports multi-agent runs burn ~15x the tokens of a chat interaction, which is the other half of that ledger
  • Two rules for making it pay:
    • Workers get a schema, an example, and a budget. Nothing else
    • The orchestrator sees results, never raw worker transcripts, or your context bill grows with every worker you add
  • Supervisor. A loop rather than a fan-out. One model holds the plan, calls an agent, reads the result, revises the plan, calls the next one.
    • Sequential and stateful. The supervisorā€™s context accumulates every result
    • Use for: long-horizon tasks where step N depends on step N-1 (coding agents, migration tooling, multi-stage pipelines)
    • Wrong for: anything parallelizable. It turns a 4-second fan-out into a 40-second chain

Summary AI-generated

Summarised by an AI assistant when this source was saved. Double-check against the original.

In summary, while multi-agent architectures mirror microservices in structure, they break the traditional playbook because the \"physics\" of a call has shifted from millisecond-latency deterministic requests to multi-second non-deterministic inference passes.

  • Treat agents as microservices by using typed contracts (like MCP) and narrow prompts to avoid the \"monolith problem\" of degrading performance.
  • Implement containment via tool-call budgets, model-tier fallbacks, and selective retries to manage non-deterministic failures.
  • Choose orchestration based on task dependency: use Routers for triage, Orchestrator-Workers for parallel tasks to save costs, and Supervisors for sequential chains.
  • Maintain full logs of prompts and outputs keyed by trace IDs, as standard distributed tracing is currently insufficient for debugging agent reasoning.

Notes this inspired