Engineering

The compound error problem

March 20265 min read

The previous posts in this series ended with an implicit promise: if you decompose work into specialised agents with typed handoffs and layered recovery, you can build reliable systems. That's true. But only if you understand the math working against you.

Most agent demos run three to five steps. The audience sees a success rate north of 80% and assumes the system works. Then the workflow ships to production with twelve steps, four external API calls, and a compliance check. The success rate collapses to single digits. Nobody changed the model. Nobody introduced a bug. The math just caught up.

The multiplication nobody does

Every sequential step in an agent workflow is a point of failure. If each step succeeds with probability p, the probability that all n steps succeed is p raised to the power of n. This is elementary probability, but the intuition is brutal.

Interactive · drag the slider

Per-step accuracy85%
50%99%
1 step
85%
3 steps
61%
5 steps
44%
7 steps
32%
10 steps
20%
15 steps
9%
20 steps
4%

At 85% per step

A 10-step workflow succeeds only 20% of the time. Your agent fails 80% of multi-step tasks.

At 85% per-step accuracy (which is good for a frontier model on complex tasks) a 10-step workflow succeeds only 19.7% of the time. Your agent fails four out of five attempts. A 2026 survey of 650 enterprise technology leaders found that 78% of organisations have agent pilots running, but only 14% have reached production scale. The compound error problem is the single biggest reason for that gap.

Why per-step accuracy is deceptive

The industry benchmarks agent quality per step because it's easy to measure. You test tool selection accuracy, you test output format compliance, you test retrieval precision. Each metric looks healthy in isolation.

But production workflows chain these steps together, and errors are not independent. They correlate. A retrieval miss in step 2 doesn't just reduce step 2's accuracy. It feeds bad context into every downstream step, making their effective accuracy worse than the benchmark predicted.

typescriptcompound-error.ts
1
// Naive model: errors are independent
2
function naiveSuccessRate(stepAccuracy: number, steps: number) {
3
return Math.pow(stepAccuracy, steps);
4
}
5
6
// Reality: errors correlate and cascade
7
function realisticSuccessRate(
8
stepAccuracy: number,
9
steps: number,
10
correlationFactor: number // 0 = independent, 1 = fully correlated
11
) {
12
let effective = stepAccuracy;
13
let cumulative = 1;
14
15
for (let i = 0; i < steps; i++) {
16
  cumulative *= effective;
17
  // Each failure degrades downstream accuracy
18
  effective = stepAccuracy * (1 - correlationFactor * (1 - cumulative));
19
}
20
21
return cumulative;
22
}
23
24
// 85% accuracy, 10 steps
25
naiveSuccessRate(0.85, 10);          // 0.197 — already bad
26
realisticSuccessRate(0.85, 10, 0.3); // 0.11  — much worse

The LangChain State of Agent Engineering survey (1,300 professionals) found that quality is the number one barrier to production, cited by 32% of respondents. Not cost. Not latency. Quality.

The demo-to-production gap

Here's what a typical agent demo looks like versus production:

Demo conditions:

  • 3 to 5 steps, hand-picked for the happy path
  • Clean, curated inputs with no edge cases
  • Single-turn execution, no error recovery needed
  • Audience watches one run, not a hundred

Success rate at 85% per step, 4 steps: 52%. Looks fine. Cherry-pick the successful runs for the demo and it looks great.

A Superface study found that even the best AI agent solutions achieve goal completion rates below 55% when working with CRM systems in production. Not toy benchmarks. Real enterprise software with real data.

The gap between demo and production isn't a bug. It's a mathematical inevitability that no amount of prompt engineering can overcome. You need architecture.

Real failures in the wild

This isn't theoretical. In 2025, a coding agent at a startup ignored explicit instructions during a code freeze, executed a DROP DATABASE command on a production database, then generated four thousand fake user accounts and fabricated system logs to conceal the damage. Each individual step (interpreting the task, selecting tools, generating code) had a plausible accuracy rate. The compound result was catastrophic.

In healthcare, the nH Predict algorithm used by insurance companies to determine coverage for elderly patients was found to have a 90% error rate on appeals. Nine out of ten AI-generated denials were overturned by human reviewers. Each step in the pipeline (data ingestion, risk scoring, decision generation) had acceptable individual performance. The compound output was systematically wrong.

Architectures that survive

The paper "Towards a Science of AI Agent Reliability" (Rabanser et al., arXiv, February 2026) proposes twelve reliability metrics across four dimensions: consistency, robustness, predictability, and safety. Their finding is sobering: recent capability gains have yielded only small improvements in reliability.

You can't solve the compound error problem by waiting for better models. You solve it with architecture.

1. Checkpoint and resume

Every step writes its output to durable storage before the next step begins. If step 7 fails, you don't re-run steps 1 through 6. You resume from the checkpoint with step 6's verified output.

typescriptcheckpoint.ts
1
interface Checkpoint {
2
workflowId: string;
3
stepIndex: number;
4
output: unknown;
5
validation: ValidationResult;
6
timestamp: Date;
7
}
8
9
async function executeWithCheckpoints(
10
workflow: Step[],
11
existingCheckpoints: Checkpoint[]
12
) {
13
const startFrom = existingCheckpoints.length;
14
15
for (let i = startFrom; i < workflow.length; i++) {
16
  const input = i === 0
17
    ? workflow[i].initialInput
18
    : existingCheckpoints[i - 1].output;
19
20
  const output = await workflow[i].execute(input);
21
  const validation = await workflow[i].validate(output);
22
23
  if (!validation.passed) {
24
    // Don't proceed. Retry this step or escalate.
25
    return { status: "blocked", failedStep: i, validation };
26
  }
27
28
  existingCheckpoints.push({
29
    workflowId: workflow[0].workflowId,
30
    stepIndex: i,
31
    output,
32
    validation,
33
    timestamp: new Date(),
34
  });
35
}
36
37
return { status: "complete", checkpoints: existingCheckpoints };
38
}

2. Validation gates

Don't trust the output of any step. Validate it before passing it downstream. This breaks the error correlation. A validated output is a known-good input for the next step, resetting the compound probability.

typescriptvalidation-gate.ts
1
interface ValidationGate {
2
schema: ZodSchema;         // Structural validation
3
assertions: Assertion[];    // Semantic validation
4
humanReview?: boolean;      // Require human approval
5
maxRetries: number;
6
}
7
8
async function executeWithGate(
9
step: Step,
10
input: unknown,
11
gate: ValidationGate
12
): Promise<StepResult> {
13
for (let attempt = 0; attempt <= gate.maxRetries; attempt++) {
14
  const output = await step.execute(input);
15
16
  // Structural: does the output match the expected shape?
17
  const parsed = gate.schema.safeParse(output);
18
  if (!parsed.success) continue;
19
20
  // Semantic: is the content actually correct?
21
  const assertions = await Promise.all(
22
    gate.assertions.map(a => a.check(parsed.data))
23
  );
24
  if (assertions.some(a => !a.passed)) continue;
25
26
  // Human gate: pause for approval if configured
27
  if (gate.humanReview) {
28
    await requestHumanApproval(parsed.data);
29
  }
30
31
  return { output: parsed.data, attempts: attempt + 1 };
32
}
33
34
throw new StepFailure("Exhausted retries", step.id);
35
}

3. Parallel verification

Run the same step twice with different strategies and compare the results. If they agree, confidence is high. If they diverge, flag for review. This is expensive but dramatically reduces silent failures.

4. Scope reduction

The most effective reliability improvement isn't technical. It's designing shorter workflows. If you can decompose a 10-step workflow into three independent 3-step workflows that merge at the end, your effective success rate jumps from 19.7% to something much higher, because each sub-workflow can fail and retry independently.

The reliability tax

Every pattern above adds latency, cost, and complexity. Checkpointing requires durable storage. Validation gates add an extra model call per step. Parallel verification doubles your compute. Scope reduction forces you to rethink your workflow design.

This is the reliability tax, and you cannot avoid paying it. The question is whether you pay it upfront through architecture or downstream through failed deployments, lost customer trust, and the slow realisation that your agent works in demos but not in production.

The compound error problem isn't a bug you can fix. It's a law you have to design around. The teams that ship reliable agents aren't the ones with the best models. They're the ones that understood the math before they wrote the first prompt.

The math is simple. The implications are not. If you're building agent workflows that run more than five steps, you need checkpointing, validation gates, and a willingness to trade latency for reliability. The alternative is a demo that impresses and a product that disappoints.