The architecture of agentic workflows

The previous post ended with a claim: for complex workflows, break the work into specialized agents that communicate through structured handoffs. This post is the deep dive into what that actually looks like in production.

We've been running multi-agent systems in production for six months. Some of them work beautifully. Others failed in ways that were genuinely surprising. What follows is everything we learned: the architecture, the failure modes, and the patterns that survived contact with real users.

Why single-agent systems hit a ceiling

A single agent with 30 tools, a long system prompt, and a prayer is the default architecture most teams start with. It works for demos. It works for simple tasks. It stops working the moment complexity crosses a threshold.

The failure mode isn't dramatic. The agent doesn't crash or hallucinate wildly. It just gets mediocre. Tool selection accuracy drops. Plans become vague. The model spends its reasoning capacity managing its own complexity instead of solving the user's problem.

This is the same reason companies have departments. An engineer could theoretically also handle sales calls, write legal contracts, and manage payroll. In practice, specialization wins because depth beats breadth when quality matters.

The multi-agent architecture

The core idea is simple: instead of one agent doing everything, you have a coordinator that delegates to specialists. Each specialist has a narrow focus, a small tool set, and clear boundaries.

The orchestrator never touches tools directly. It only does three things: decompose tasks, delegate to specialists, and merge results. This separation is critical. The orchestrator reasons about what needs to happen, while specialists reason about how to do it.

interface TaskEnvelope {

id: string;

type: "research" | "draft" | "review";

payload: Record<string, unknown>;

constraints: {

  maxSteps: number;

  deadline?: Date;

  qualityThreshold?: number;

};

}

interface ResultEnvelope {

taskId: string;

status: "complete" | "partial" | "failed";

confidence: number;

data: unknown;

warnings: string[];

}

async function orchestrate(goal: string): Promise<Result> {

// Decompose into typed subtasks

const tasks = await planner.decompose(goal);

// Execute with dependency resolution

const results = new Map<string, ResultEnvelope>();

for (const task of topologicalSort(tasks)) {

  const agent = getSpecialist(task.type);

  const result = await agent.execute(task);

  if (result.status === "failed" && !task.optional) {

    return handleFailure(task, result, results);

  }

  results.set(task.id, result);

}

// Merge results into final output

return merger.combine(results);

}

The handoff protocol

The most important design decision in a multi-agent system isn't which model to use or how many agents to deploy. It's how agents talk to each other.

If agents communicate through natural language, say "hey, can you look up the pricing for these three competitors?", you get the same ambiguity problems that plague human organizations. Instead, we use typed message envelopes with strict schemas.

{ task: "find_pricing", constraints: { max_age: "30d" }, deadline: "step_3" }

{ status: "complete", confidence: 0.92, data: [...], warnings: ["2 sources >30d"] }

{ task: "draft_comparison", source: ref("research_001"), format: "table" }

{ status: "complete", artifact: ref("draft_001"), word_count: 340 }

Every handoff is a typed envelope. The TaskEnvelope has required fields: task type, constraints, deadline. The ResultEnvelope has status, confidence, and structured data. No prose. No ambiguity.

Failure recovery at scale

Single-agent systems have a simple failure model: the agent either succeeds or fails. Multi-agent systems have a much richer failure space. An agent can partially succeed. A downstream agent can fail because an upstream agent returned low-quality data. The orchestrator itself can make a bad delegation decision.

We handle this with a layered recovery strategy:

1. Tool-level retry: transient failures (network timeouts, rate limits) are retried automatically by the tool layer. The agent never sees them.
2. Agent-level fallback: if the primary strategy fails, the agent tries an alternative. Cache instead of live API. Heuristic instead of model call.
3. Orchestrator-level rerouting: if an agent fails completely, the orchestrator can reassign the task or adjust the plan.
4. Human escalation: when no automated recovery is possible, escalate with full context and a suggested action.

“

The goal isn't zero failures. It's zero surprises. Every failure should be caught, classified, and handled by the right layer.

State management across agents

Here's where most multi-agent tutorials lie to you: they show agents passing messages back and forth as if that's all the state you need. In production, you need three distinct state layers:

// Layer 1: Conversation state (per-agent)

// Each agent has its own context window.

// This is ephemeral. It dies when the task is done.

interface AgentState {

messages: Message[];

currentStep: number;

toolResults: Map<string, ToolResult>;

}

// Layer 2: Workflow state (shared, structured)

// The orchestrator manages this. All agents can read it.

// Persisted to a database, survives crashes.

interface WorkflowState {

id: string;

tasks: TaskEnvelope[];

results: Map<string, ResultEnvelope>;

status: "running" | "paused" | "complete" | "failed";

checkpoints: Checkpoint[];

}

// Layer 3: Knowledge state (long-lived)

// Facts, preferences, and learned patterns.

// Survives across workflows and sessions.

interface KnowledgeState {

facts: VectorStore;

preferences: UserProfile;

patterns: LearnedPatterns;

}

The critical distinction: conversation state is private to each agent. Agent A cannot read Agent B's message history. They communicate only through the structured handoff protocol. This isolation is a feature, not a limitation. It prevents context pollution and keeps each agent focused.

Evaluation and observability

You cannot run a multi-agent system in production without observability. Every handoff, every tool call, every decision point needs to be logged with enough context to reconstruct the full execution path after the fact.

We instrument at three levels:

• Trace-level: a unique trace ID follows a request through every agent it touches, similar to distributed tracing for microservices.
• Decision-level: every time an agent chooses an action over alternatives, we log the choice and the reasoning. This is where you find systematic biases.
• Outcome-level: did the final result actually solve the user's problem? This requires ground truth, which is hard to get at scale, but even partial outcome data is invaluable.

// Every agent action is wrapped in a span

const span = tracer.startSpan("researcher.query_api", {

traceId: workflow.traceId,

agentId: "researcher-001",

taskId: task.id,

toolName: "pricing_api",

input: sanitize(task.payload),

});

try {

const result = await pricingApi.query(params);

span.setStatus("ok");

span.setAttribute("result.confidence", result.confidence);

span.setAttribute("result.item_count", result.data.length);

return result;

} catch (err) {

span.setStatus("error");

span.recordException(err);

throw err;

} finally {

span.end();

}

When not to use multi-agent

Not every problem needs this architecture. The overhead is significant: typed protocols, orchestration logic, and observability infrastructure all add up. Use multi-agent when:

• The task naturally decomposes into distinct phases (research → create → review)
• Different phases need different tool sets or model capabilities
• You need different failure/recovery strategies at different stages
• The quality bar requires specialized validation (compliance, factual accuracy)

For everything else, a well-designed single agent with constrained tools is simpler, faster, and easier to debug.

“

Architecture is the art of drawing lines. Multi-agent is powerful because the lines are explicit. Just make sure you only draw them where the complexity demands it.

Multi-agent systems aren't a silver bullet. They're a way of managing complexity by making it visible. Typed handoffs make communication explicit. Layered recovery makes failure handling predictable. Observability makes debugging possible. The agents themselves are the easy part. It's the space between them that determines whether the system works.