When multi-agent is the wrong answer
A team proposes decomposing an existing single-agent workflow into seven specialized agents. Latency is currently 4 seconds and accuracy 88%. What do you ask before approving, and under what conditions do you reject it?
The constraint. Every agent boundary is a serialization point and a context copy. Seven agents mean at minimum seven model calls in sequence, seven prompt preambles, and six lossy handoffs. If the current call is 4 seconds, the decomposed version is plausibly 15-25 seconds and 3-5x the token cost. That has to buy something.
The three questions that settle it.
- 01Is the decomposition parallel or serial? Parallel fan-out over independent subtasks is the only topology that improves latency. A serial chain of seven specialists is a slower, more expensive prompt chain wearing a costume.
- 02Does each agent need a different tool scope or a different model? This is the legitimate case. An agent that may only read, and one that may write, are genuinely different security principals. An expensive reasoner and a cheap extractor are genuinely different cost profiles. Splitting for "separation of concerns" alone is not a reason — that is a function, not an agent.
- 03What is lost at each handoff? Agents communicate in natural language, which is lossy and unvalidated. Seven agents means six opportunities to silently drop a constraint. If the handoff can be a typed schema, the boundary is probably a function call, not an agent.
When it is right. Tool scope isolation for security. Context isolation, where a sub-agent burns 50k tokens searching and returns a 500-token summary, keeping the parent window clean — this is the strongest genuine case. Cost tiering. Independent parallel work with a deterministic merge. Different failure and retry semantics per stage.
When it is wrong. Latency-sensitive paths. Anything where the subtasks share mutable state. Anything where the "agents" are really steps in a fixed sequence — that is a workflow, and it should be a graph with deterministic edges and model calls at the nodes.
The reframe. The useful axis is not "one agent or many" but "how much of this is deterministic." Most production systems are a deterministic workflow with model calls at a few nodes. Push work out of the model, not into more models.
Stack. LangGraph if the topology is a state machine with checkpoints. Temporal if durability and replay matter more than agent ergonomics — and at seven stages they do. Reject CrewAI for this: LLM-driven routing makes cost and latency non-deterministic and failures non-reproducible, which is the opposite of what a seven-stage system needs.
Where answers fail. Accepting the premise. The interviewer is testing whether you will design the thing you were asked for or the thing that should exist.