Treating a model as a controlled asset with an owner, a lifecycle, and limits.
28documents on this topic
21organizations represented
4issues named
10sourced citations
1sourced statistics
The state of it
One of 7 topics within Governance & oversight.
28 documents from 21 organizations treat the model as a controlled asset. The research splits cleanly, and revealingly, by who wrote it.
The consulting and institutional material describes model risk in governance terms - inventory, validation, lines of defence. The practitioner material describes the same problem operationally, and in more useful detail: Uber writes about codifying safe model deployment, automatic retraining and feature monitoring into the platform itself, because doing it per-model does not survive scale.
The gap between those two descriptions is the finding. Governance frameworks treat validation as an event, and production treats it as a continuous cost. McKinsey's framing is the bridge worth borrowing: agents should be measured, monitored, and either retrained or retired - the same lifecycle applied to an employee, with the retirement step actually used.
The issues, by agreement
How many independent organizations name each issue as a problem. An issue is only as real as the number of separate publishers that identify it, so the count is the ranking. Bars are organizations, not documents. Where the count reads ours, no publisher here states the issue and the analysis is our own.
The chart above counts positions; this shows whose they are. Read down a column for what one organization holds across the whole topic, and across a row for who lines up on one issue. Where a cell carries more than one position, the strongest is shown and the rest are in the tooltip.
Ddisputes itQqualifies itNnames it as a problemPproposes a fix
Model risk management: 4 issues against the 6 organizations cited on them. The number under each name is how many of these issues it is cited on.
A dot means this organization is not cited on that issue. It does not mean they are silent on it: an organization is cited where its document takes a position we could locate, and the absence of a citation is the absence of a finding, not a finding of absence. Who is represented lists everyone working on this topic, including those not cited above.
Where they disagree
No contradictions recorded on this topic yet.
The issues in full
Each issue carries the organizations that name it, the numbers behind it, and the remedies proposed - with the concrete steps under each. Every citation points at a section of a named document, so any count here can be checked.
Issue 012 organizations name itnewest evidence Jun 2025
Pretrained and open-source models enter production without validation
A downloaded model artefact arrives with no validation history, no provenance, and the same trust level as internal code - which is to say, more than it has earned.
Treat an imported model as an untrusted dependency: scanned, recorded, versioned, and owned.
Done when Every imported model artefact is scanned, its provenance captured and its hash recorded, an independent evaluation against your own data precedes production use, and it sits in the register with an owner like any other asset.
Require scanning, provenance capture and hash recording for every imported model artefact.0-30 daysCISO
Run an independent evaluation against your own data before production use.30-90 daysCIO
Record the artefact in the register with an owner, like any other production asset.30-90 daysRisk
The evidence — 2 documents
Organization
Document
Position
AccentureConsultancy · June 2025
State of cybersecurityOur reading Argues AI security must be embedded into governance structures with robust third-party risk management rather than remaining fragmented.Third-party risk and AI security
names it
Booz Allen HamiltonConsultancy · December 2024
Countering enterprise AI security threatsOur reading Notes these models conceal sizeable risks, and that a model engineering team downloading a pretrained open-source product may overlook malware and other threats.Third-party, pretrained and open-source models
names it
Issue 021 organization name it1 qualifies it2026 evidence
Models degrade quietly, and the first signal is usually a customer
Performance decays as the world moves away from the training distribution. Without continuous measurement the decline is invisible until it surfaces as a complaint, a loss, or a regulator's question.
Monitoring that produces a dashboard nobody watches is not a control. Bind the measurement to an automatic consequence.
Done when Each production model states its metric, baseline and the level at which it must not operate, that level is wired to an action rather than an alert, and breaches reach the risk committee with the action taken.
For each production model, define the metric, the baseline, and the level at which it must not operate.0-30 daysCIO
Wire the threshold to an action - retrain, route to human, or disable - not to an alert.30-90 daysCTO
Report threshold breaches to the risk committee with the action taken.ongoingRisk
Use the retirement step
Every lifecycle in this research includes retirement and almost no organization exercises it. A model nobody will ever switch off is not managed.
Done when At least one production model has actually been retired, with a record of what the retirement took, so the path exists before it is urgent.
Identify production models whose measured value no longer justifies their cost or risk.30-90 daysCIO
Retire at least one and record what it took, so the path exists when it is urgent.90-180 daysCIO
The evidence — 3 documents
Organization
Document
Position
UberEnterprise
Uber - Journey to Generative AIOur reading Describes investing in production capabilities including safe model deployment and automatic model retraining, and tooling that measures model quality across all stages, precisely because manual maintenance does not scale.Platform investments: safe deployment and retraining
names it
AWSHyperscaler · April 2026
Governance, risk and compliance for responsible AI in financial servicesOur reading Qualifies the cause by way of the management standard it cites: a system that keeps learning changes how it behaves while it is in use, which is named as one of the properties separating this from conventional software. The thing signed off at launch is therefore not the thing running later, and nothing has to go wrong for that to be true.Why the drift is built in, not a defect
qualifies it
OpenAIFrontier lab · August 2026
A scorecard for the AI ageOur reading Proposes tracking every result as ready to use, needing correction, or needing escalation, on the grounds that these show whether the system is actually reducing work in a way raw accuracy does not.Three outcomes: ready, needs correction, needs escalation
proposes a fix
Issue 031 organization name it2026 evidence
Validation is treated as a launch gate, not a continuing cost
Models are validated once, at approval, against a snapshot. The operating budget contains no line for revalidating them, so it does not happen.
If retraining and revalidation are not funded as a run cost, they compete with new features and lose every time.
Done when Every production system carries a run-cost line for monitoring, retraining and revalidation, deployment approvals without one are refused, and actual is reviewed against budgeted lifecycle spend annually.
Add a run-cost line for monitoring, retraining and revalidation to each production system.0-30 daysCFO
Refuse deployment approvals that carry no run-cost line.30-90 daysCFO
Review actual against budgeted lifecycle spend annually.ongoingCFO
The evidence — 3 documents
Organization
Document
Position
UberEnterprise
Uber - Journey to Generative AIOur reading Recommends codifying safe model deployment, model retraining and feature monitoring into the platform, treating them as standing capability rather than per-project work.Codify best practices in the platform
names it
AWSHyperscaler · April 2026
Governance, risk and compliance for responsible AI in financial servicesOur reading Reports that the firms it considers mature hold standing processes for revalidating systems already in production, and set how hard and how often from the materiality of the use case rather than applying one cadence to everything. That makes validation a recurring commitment sized to consequence, which is the shape a launch gate cannot take.Revalidation frequency set by what the use case carries
proposes a fix
OpenAIFrontier lab · July 2026
How to control AI usage and spendOur reading Proposes defining the quality bar before testing, running evaluations that reflect real tasks including edge cases, and measuring the full cost of reaching that standard - attempts, completion rate, latency and human review - which makes validation a recurring measurement rather than a launch gate.Evaluate model efficiency by outcome
proposes a fix
Issue 04Our analysis2026 evidence
The second line cannot independently verify a model performance claim
The three-lines model assumes the second line can check the first. For AI it often cannot, because the skills to reproduce an evaluation sit entirely in the team being checked.
96%Respondents expecting risk and compliance roles to expand as AI embedsMoody’s · Sep 2025
How to fix it — 1 approach, 3 steps
Test whether the second line can reproduce one number
Pick a model performance figure that has been reported upward and ask the second line to reproduce it independently. The result is your assurance capability, measured.
Done when The second line has attempted to reproduce one reported model performance claim from the last quarter without help from the owning team, and the capability gap that revealed is funded rather than absorbed by self-reporting.
Select one reported model performance claim from the last quarter.0-30 daysAudit Committee
Have the second line attempt to reproduce it without help from the owning team.30-90 daysRisk
Fund the capability gap the exercise reveals rather than accepting self-reporting.90-180 daysCFO
The evidence — 2 documents
Organization
Document
Position
AWSHyperscaler · April 2026
Governance, risk and compliance for responsible AI in financial servicesOur reading Puts something ahead of the audit itself: the business owner, the risk and compliance function and internal audit have to agree on context, on definitions, and on what is coming toward them. The second line cannot independently check a claim while it is still working from a different definition of what counts as an AI system, and that disagreement is invisible until someone tries to verify a number.Align the three lines on the words first
proposes a fix
DeloitteConsultancy
Strategic governance of AIOur reading Positions independent assurance over AI as an emerging oversight responsibility for audit committees.Roadmap: risk and performance
proposes a fix
Who is represented
This dossier is drawn from 23 organizations working on the subject, 6 of which are cited directly in the issues above.
Consultancy — 4
Deloitte 2Accenture 1Booz Allen Hamilton 1McKinsey & Company 3
Institution — 4
Cloud Security Alliance 2NIST 2Association of Corporate Counsel 1World Economic Forum 1