Reliability & failure modes

Non-determinism, compounding error, and how agent failure differs from software failure.

15documents on this topic
12organizations represented
2issues named
9sourced citations
0sourced statistics

The state of it

One of 6 topics within Agent orchestration.

15 documents from 12 organizations address how agents fail. The material is thinner than the deployment enthusiasm would suggest, and what exists agrees on one uncomfortable point: agent failure is not software failure.

A deterministic system fails the same way twice. An agent given the same input can take a different path, so a fix cannot be verified by re-running the case that broke. Anthropic notes that multiple agents introduce new challenges specifically in coordination, evaluation and reliability. IBM notes that inherent complexity produces unpredictable behaviour that complicates assurance. Neither is a reason not to build; both are reasons the QA process you already have does not transfer.

The issues, by agreement

How many independent organizations name each issue as a problem. An issue is only as real as the number of separate publishers that identify it, so the count is the ranking. Bars are organizations, not documents. Where the count reads ours, no publisher here states the issue and the analysis is our own.

Where they disagree

No contradictions recorded on this topic yet.

The issues in full

Each issue carries the organizations that name it, the numbers behind it, and the remedies proposed - with the concrete steps under each. Every citation points at a section of a named document, so any count here can be checked.

Issue 012 organizations name it2026 evidence

Non-determinism breaks the testing process the organization already has

The same input can produce a different path on the next run, so a passing test proves less than it used to and a fix cannot be confirmed by reproducing the original failure.

How to fix it — 2 approaches, 5 steps

Test the distribution, not the case

Run each scenario many times and assert on the rate, not on a single pass. A single green run tells you almost nothing.

Done when The eval set includes known-hard real cases, each runs repeatedly, the assertion is a success rate against a defined floor rather than a single pass, and release is blocked on regression.

  1. Build an eval set of real cases including known-hard ones, and run each repeatedly.0-30 daysCTO
  2. Assert on a success rate with a defined floor, not on individual outcomes.30-90 daysCTO
  3. Re-run the full set on every model or prompt change, and block release on regression.ongoingCTO

Grade with a separate call, not the acting model

Asking the model that produced the output to also judge it collapses the check into the thing being checked.

Done when Guardrail and quality evaluation run in a call separate from the one doing the work, and the grader output is retained as the evidence trail for the decision.

  1. Move guardrail and quality evaluation into a separate call from the one doing the work.0-30 daysCTO
  2. Retain the grader output as the evidence trail for the decision.30-90 daysRisk
The evidence — 5 documents
OrganizationDocumentPosition
AnthropicFrontier labBuilding multi-agent researchOur reading States that multiple agents introduce new challenges in coordination, evaluation and reliability, and that open-ended work is chosen precisely because a fixed path cannot be hardcoded.Coordination, evaluation and reliability challengesnames it
CISAOther · April 2026Careful adoption of agentic AI servicesOur reading Names precisely why the existing test process does not transfer: identical prompts can produce different actions, because the model is stochastic and the context window and the environment it reads both move underneath it. Reproducibility is therefore not a property the system has by default, which is the assumption every regression suite is built on.The same prompt is not the same testnames it
AnthropicFrontier lab · December 2024Building effective agentsOur reading Proposes separating guardrail evaluation into its own model call rather than having one call handle both the guardrail and the response, and using separate calls to evaluate different aspects of performance.Parallelization: sectioning; automating evalsproposes a fix
OpenAIFrontier lab · August 2026A scorecard for the AI ageOur reading Proposes classifying every result as ready to use, needing correction, or needing escalation, which measures dependability in a way a pass/fail test cannot.Three outcomes: ready, needs correction, needs escalationproposes a fix
UberEnterpriseUber - Journey to Generative AIOur reading Describes codifying safe deployment, retraining and monitoring into the platform itself because per-project quality practice does not survive scale.Codifying practice in the platformproposes a fix

Issue 021 organization name it2026 evidence

There is no accepted way to evaluate an agent, so every team invents one

Model benchmarks do not measure an agent, and no standard has replaced them. Each team builds its own harness, which makes results incomparable across the portfolio and unauditable from outside it.

This is the gap that makes every other reliability control weaker than it looks. Anthropic names evaluation explicitly as one of the new challenges multiple agents introduce. OpenAI proposes an outcome classification - ready to use, needs correction, needs escalation - which is the closest thing to a portable standard in this research, because it measures what a business cares about rather than model accuracy. Until something like it is adopted consistently, a board comparing two agent programmes is comparing two different measurements.

How to fix it — 1 approach, 3 steps

Adopt one outcome scale across every agent

Pick a single classification - ready, needs correction, needs escalation - and apply it to every agent in the portfolio so results are comparable.

Done when Every production agent reports monthly against one outcome classification, the portfolio is compared on it, and something has actually been retired from the persistent bottom.

  1. Adopt one outcome classification and publish it as the standard for all agent reporting.0-30 daysCIO
  2. Require every production agent to report against it monthly.30-90 daysCIO
  3. Compare across the portfolio and retire the persistent bottom.ongoingCOO
The evidence — 4 documents
OrganizationDocumentPosition
AnthropicFrontier labBuilding multi-agent researchOur reading Names evaluation as one of the specific new challenges introduced by multiple agents, alongside coordination and reliability.Coordination, evaluation and reliabilitynames it
GoogleHyperscaler · September 2024Agents (Google whitepaper)Our reading Sets out the agent as a designed architecture with distinguishable components, which is what makes component-level evaluation possible at all.Cognitive architectures and implementationproposes a fix
OpenAIFrontier lab · August 2026A scorecard for the AI ageOur reading Proposes classifying results as ready to use, needing correction, or needing escalation, and dividing full cost by the number of tasks meeting the bar, on the grounds that this shows whether work is genuinely being reduced in a way an accuracy score does not.Three outcomes; cost per successful taskproposes a fix
UberEnterpriseUber - Journey to Generative AIOur reading Describes investing in tooling that measures model quality across all stages and codifying it into the platform, so evaluation is a platform property rather than a per-team invention.Codifying practice in the platformproposes a fix

Who is represented

This dossier is drawn from 16 organizations working on the subject, 5 of which are cited directly in the issues above.

Consultancy — 3

Deloitte 2 Genpact 1 QuantumBlack, AI by McKinsey 1

Institution — 2

Cloud Security Alliance 1 NIST 1

Academic — 2

arXiv (research) 2 Carnegie Mellon SEI 1

Hyperscaler — 3

Google 1 Google Cloud 2 AWS 1

Frontier lab — 2

Anthropic 2 OpenAI 1

Enterprise — 2

Uber 1 LinkedIn 1

Vendor — 1

Cursor 1

Other — 1

CISA 1