Non-determinism, compounding error, and how agent failure differs from software failure.
15documents on this topic
12organizations represented
2issues named
9sourced citations
0sourced statistics
The state of it
One of 6 topics within Agent orchestration.
15 documents from 12 organizations address how agents fail. The material is thinner than the deployment enthusiasm would suggest, and what exists agrees on one uncomfortable point: agent failure is not software failure.
A deterministic system fails the same way twice. An agent given the same input can take a different path, so a fix cannot be verified by re-running the case that broke. Anthropic notes that multiple agents introduce new challenges specifically in coordination, evaluation and reliability. IBM notes that inherent complexity produces unpredictable behaviour that complicates assurance. Neither is a reason not to build; both are reasons the QA process you already have does not transfer.
The issues, by agreement
How many independent organizations name each issue as a problem. An issue is only as real as the number of separate publishers that identify it, so the count is the ranking. Bars are organizations, not documents. Where the count reads ours, no publisher here states the issue and the analysis is our own.
Each issue carries the organizations that name it, the numbers behind it, and the remedies proposed - with the concrete steps under each. Every citation points at a section of a named document, so any count here can be checked.
Issue 012 organizations name it2026 evidence
Non-determinism breaks the testing process the organization already has
The same input can produce a different path on the next run, so a passing test proves less than it used to and a fix cannot be confirmed by reproducing the original failure.
Run each scenario many times and assert on the rate, not on a single pass. A single green run tells you almost nothing.
Done when The eval set includes known-hard real cases, each runs repeatedly, the assertion is a success rate against a defined floor rather than a single pass, and release is blocked on regression.
Build an eval set of real cases including known-hard ones, and run each repeatedly.0-30 daysCTO
Assert on a success rate with a defined floor, not on individual outcomes.30-90 daysCTO
Re-run the full set on every model or prompt change, and block release on regression.ongoingCTO
Grade with a separate call, not the acting model
Asking the model that produced the output to also judge it collapses the check into the thing being checked.
Done when Guardrail and quality evaluation run in a call separate from the one doing the work, and the grader output is retained as the evidence trail for the decision.
Move guardrail and quality evaluation into a separate call from the one doing the work.0-30 daysCTO
Retain the grader output as the evidence trail for the decision.30-90 daysRisk
The evidence — 5 documents
Organization
Document
Position
AnthropicFrontier lab
Building multi-agent researchOur reading States that multiple agents introduce new challenges in coordination, evaluation and reliability, and that open-ended work is chosen precisely because a fixed path cannot be hardcoded.Coordination, evaluation and reliability challenges
names it
CISAOther · April 2026
Careful adoption of agentic AI servicesOur reading Names precisely why the existing test process does not transfer: identical prompts can produce different actions, because the model is stochastic and the context window and the environment it reads both move underneath it. Reproducibility is therefore not a property the system has by default, which is the assumption every regression suite is built on.The same prompt is not the same test
names it
AnthropicFrontier lab · December 2024
Building effective agentsOur reading Proposes separating guardrail evaluation into its own model call rather than having one call handle both the guardrail and the response, and using separate calls to evaluate different aspects of performance.Parallelization: sectioning; automating evals
proposes a fix
OpenAIFrontier lab · August 2026
A scorecard for the AI ageOur reading Proposes classifying every result as ready to use, needing correction, or needing escalation, which measures dependability in a way a pass/fail test cannot.Three outcomes: ready, needs correction, needs escalation
proposes a fix
UberEnterprise
Uber - Journey to Generative AIOur reading Describes codifying safe deployment, retraining and monitoring into the platform itself because per-project quality practice does not survive scale.Codifying practice in the platform
proposes a fix
Issue 021 organization name it2026 evidence
There is no accepted way to evaluate an agent, so every team invents one
Model benchmarks do not measure an agent, and no standard has replaced them. Each team builds its own harness, which makes results incomparable across the portfolio and unauditable from outside it.
This is the gap that makes every other reliability control weaker than it looks. Anthropic names evaluation explicitly as one of the new challenges multiple agents introduce. OpenAI proposes an outcome classification - ready to use, needs correction, needs escalation - which is the closest thing to a portable standard in this research, because it measures what a business cares about rather than model accuracy. Until something like it is adopted consistently, a board comparing two agent programmes is comparing two different measurements.
Pick a single classification - ready, needs correction, needs escalation - and apply it to every agent in the portfolio so results are comparable.
Done when Every production agent reports monthly against one outcome classification, the portfolio is compared on it, and something has actually been retired from the persistent bottom.
Adopt one outcome classification and publish it as the standard for all agent reporting.0-30 daysCIO
Require every production agent to report against it monthly.30-90 daysCIO
Compare across the portfolio and retire the persistent bottom.ongoingCOO
The evidence — 4 documents
Organization
Document
Position
AnthropicFrontier lab
Building multi-agent researchOur reading Names evaluation as one of the specific new challenges introduced by multiple agents, alongside coordination and reliability.Coordination, evaluation and reliability
names it
GoogleHyperscaler · September 2024
Agents (Google whitepaper)Our reading Sets out the agent as a designed architecture with distinguishable components, which is what makes component-level evaluation possible at all.Cognitive architectures and implementation
proposes a fix
OpenAIFrontier lab · August 2026
A scorecard for the AI ageOur reading Proposes classifying results as ready to use, needing correction, or needing escalation, and dividing full cost by the number of tasks meeting the bar, on the grounds that this shows whether work is genuinely being reduced in a way an accuracy score does not.Three outcomes; cost per successful task
proposes a fix
UberEnterprise
Uber - Journey to Generative AIOur reading Describes investing in tooling that measures model quality across all stages and codifying it into the platform, so evaluation is a platform property rather than a per-team invention.Codifying practice in the platform
proposes a fix
Who is represented
This dossier is drawn from 16 organizations working on the subject, 5 of which are cited directly in the issues above.