Handoffs vs. Supervisor: Choosing Your Multi-Agent Control Flow

The Framework Debate Is a Distraction

Every week, another LangGraph vs. CrewAI vs. AutoGen benchmark drops, and practitioners dutifully re-architect their stacks. The argument is misplaced. Your framework choice—whether it’s LangGraph’s state machine, OpenAI’s agent SDK, or a hand-rolled loop—matters far less than the control flow you encode within it. Control flow is the invisible skeleton that determines token burn, latency, failure modes, and whether your system collapses under real traffic. This post is a decision-oriented deep dive into the two dominant patterns—supervisor and handoffs—plus the overhyped swarm pattern, with production economics drawn from Anthropic’s field data [1].

Pattern 1: Supervisor (Orchestrator-Worker)

The supervisor pattern centralizes routing logic in a single orchestrator node. The supervisor receives the user’s task, decomposes it, dispatches to specialized worker agents, and aggregates results. In LangGraph, this is a cyclic graph where the supervisor node holds a shared state object, and workers are sub-nodes that read/write to that state via typed channels [2].

Mechanics in LangGraph:

  • The supervisor uses an LLM to decide which worker to call next, based on the current state (e.g., messages, pending_tasks).
  • Workers are stateless functions that receive a slice of state and return updates.
  • The graph loops until the supervisor emits a FINISH signal or hits a recursion limit.

When it shines:

  • Parallelizable workloads: The supervisor can fan out to multiple workers simultaneously (e.g., analyzing 10 documents independently).
  • Bounded scope: Each worker has a narrow, well-defined task, making it easy to test and validate.
  • Deterministic control: The supervisor dictates the exact sequence, which aids debugging and audit trails.

LangChain’s pattern table maps this to their “Subagents” pattern, which is the recommended approach for parallelizable work with bounded scope [4]. If your task is a classic map-reduce—scrape, summarize, compare—this is your pattern.

The catch: The supervisor becomes a single point of failure and a latency bottleneck. Every turn of the conversation round-trips through the supervisor’s LLM call, which adds cost and delay. Anthropic’s production data shows that multi-agent systems burn ~15x more tokens than a single chat session—and a supervisor pattern amplifies this because the orchestrator’s context window grows with every worker result it must aggregate [1]. In their BrowseComp experiments, token usage explained ~80% of performance variance—meaning token budget is the dominant cost driver, and supervisors that aggregate large contexts burn through that budget fast [1].

Pattern 2: Handoffs

Handoffs invert the control flow. Instead of a central router, each agent is autonomous and decides, mid-conversation, to transfer control to another agent. This is the native pattern in OpenAI’s Agents SDK, where handoffs are implemented as tools [3].

Mechanics in OpenAI SDK:

from agents import Agent, handoff

refund_agent = Agent(
    name="Refund Agent",
    instructions="Handle refund requests and escalate to human if needed.",
    tools=[get_refund_status]
)

billing_agent = Agent(
    name="Billing Agent",
    instructions="Handle billing inquiries.",
    handoffs=[handoff(refund_agent)]
)

# Run the conversation
result = Runner.run_sync(billing_agent, "I need a refund for my last invoice")

The key primitive is handoff():

  • on_handoff callback: Fires side effects when the transfer occurs—e.g., logging, state persistence, or notifying the next agent of prior context.
  • input_filter: Prevents context bloat by stripping irrelevant conversation history before passing to the receiving agent. This is critical because the receiving agent starts with a fresh context window, and dumping the entire prior thread defeats the purpose [3].

Why it works for user-facing flows:

  • Multi-hop conversations: A user might start with billing, pivot to refunds, then to account deletion. Handoffs let each agent handle its domain cleanly, then pass the baton without a supervisor re-analyzing the whole thread.
  • Lower latency: The transfer is a single tool call, not a round-trip through an orchestrator.
  • Natural isolation: Each agent’s context window is scoped to its own domain, reducing token waste.

LangChain’s guidance aligns: handoffs are the best fit for multi-hop user-facing flows where the path is unpredictable [4]. This is the pattern used by customer support bots that need to route between billing, technical support, and sales without a central brain.

The catch: Handoffs require careful termination design. Without an explicit loop bound, two agents can ping-pong indefinitely if their routing logic is ambiguous. Anthropic’s production lesson is relevant here: subagents should write outputs to a filesystem or artifact store and pass lightweight references back to the coordinator—not embed full payloads in the handoff message. This keeps the transfer cheap and avoids blowing the next agent’s context [2].

Pattern 3: Swarm/Network — Why It’s Rarely the Right Default

The swarm pattern—agents freely messaging each other in a peer-to-peer topology—is seductive in theory and difficult in practice. Without structured control flow, state management, debugging, and reasoning about system behavior become exponentially harder [5].

Why it fails:

  • No termination guarantee: Without a central coordinator, there’s no natural stop condition. Loops are common and expensive.
  • Context fragmentation: Each agent holds a partial view of the conversation, and there’s no shared state to reconcile conflicting interpretations.
  • Debugging nightmare: You cannot replay a deterministic trace because the execution order is emergent.

The only scenario where swarm-style networking makes sense is when you have a genuinely decentralized problem—e.g., multiple agents simulating a market or a social network—where emergent behavior is the goal, not a bug. For production task automation, stick to supervisor or handoffs.

Decision Table: Choosing Your Control Flow

Criterion Supervisor Handoffs
Parallelizable subtasks? Yes — optimal No — sequential transfer
User-facing multi-hop? Poor — high latency Yes — natural fit
Well-defined team boundaries? Yes — stable roles Yes — but roles can overlap
Dynamic routing based on user input? Weak — supervisor must re-plan Strong — agent decides locally
Token efficiency Poor — supervisor context grows Good — input filters + scoped contexts
Failure mode Supervisor bottleneck Handoff loops

Guidance:

  1. If your task is map-reduce (parallelizable, bounded scope), use a supervisor.
  2. If your task is conversational (user can pivot topics), use handoffs.
  3. If you need both, use a hybrid: a supervisor for the top-level task decomposition, and handoffs within each worker for sub-conversations. Anthropic’s orchestrator-worker pattern for their multi-agent research system follows this approach [2].

Production Reality: Token Economics, State, and Failure Modes

Token economics: Anthropic’s engineering team measured that multi-agent systems consume ~15x more tokens than a single chat session, and ~4x more than a single agent with tools [1]. This is not a rounding error—it’s a budget line item. In their BrowseComp runs, token usage explained ~80% of performance variance, making token efficiency the primary optimization target. The practical implication: design for token poverty. Use handoffs with input_filter to trim context [3], and have subagents write intermediate results to an artifact store (filesystem, vector DB) rather than passing them in-memory [2].

State management: The supervisor pattern centralizes state in the orchestrator’s graph, which is great for debugging but terrible for context limits. The handoff pattern distributes state across agents, which requires explicit handoff callbacks to persist and resume context. Anthropic’s recommendation is to treat the filesystem as the shared memory: subagents write outputs to files, and the coordinator reads only lightweight references. This decouples state size from the LLM context window [2]. FrankX’s taxonomy adds three cross-cutting state patterns: shared-state/blackboard (simple, conflict-prone), message-passing/actor (isolated, harder to debug), and event-sourcing/append-only log (auditable, replayable) [5].

Failure modes:

  • Supervisor bottleneck: The orchestrator’s LLM call is on the critical path for every turn. If it fails or times out, the entire system stalls. Mitigate with a fallback supervisor or a timeout with a deterministic router.
  • Handoff loops: Two agents with overlapping triggers can enter an infinite transfer cycle. Mitigate with a global hop counter (e.g., max 5 handoffs) and a “human handoff” escape hatch.
  • Drifting routing prompts: In handoff patterns, each agent has its own routing instructions. Over time, these prompts can drift out of sync—Agent A thinks it handles refunds, Agent B also thinks so, and neither handles billing. Mitigate with a centralized prompt registry and versioned routing rules [5].

TL;DR

Choose control flow before frameworks. Supervisor for parallelizable, bounded tasks; handoffs for multi-hop user conversations; avoid swarm unless you’re simulating emergence. Optimize for token scarcity—use artifact stores and input filters. Bound your loops, and version your routing prompts.

Further reading:

← Back to all posts