What demonstrates a control actually operates, to a board or a reviewer.
45documents on this topic
28organizations represented
3issues named
16sourced citations
0sourced statistics
The state of it
One of 6 topics within Security & threat model.
A control that operates and cannot be demonstrated is worth very little to a board, an insurer, an acquirer or a regulator. Assembling that demonstration retrospectively is where the cost lands, and it is almost always higher than building it in.
A deliberate limit is worth stating. Nothing here tells you whether you comply with any particular regime - that is a legal question about your circumstances, and anyone selling a document that answers it is selling you a risk. What is generalisable is narrower and still useful: the evidence a reviewer typically asks for, which is consistent across regimes because it follows from what a control is.
The documented incident shows the mature version. The organization involved brought in external advisors to validate its own account of what happened, commissioned an independent third-party assessment of the behaviour, and committed to publishing a technical report. That is not compliance activity. It is the difference between asserting a control worked and being able to show it.
The issues, by agreement
How many independent organizations name each issue as a problem. An issue is only as real as the number of separate publishers that identify it, so the count is the ranking. Bars are organizations, not documents. Where the count reads ours, no publisher here states the issue and the analysis is our own.
The chart above counts positions; this shows whose they are. Read down a column for what one organization holds across the whole topic, and across a row for who lines up on one issue. Where a cell carries more than one position, the strongest is shown and the rest are in the tooltip.
Ddisputes itQqualifies itNnames it as a problemPproposes a fix
Evidence & assurance: 3 issues against the 7 organizations cited on them. The number under each name is how many of these issues it is cited on.
A dot means this organization is not cited on that issue. It does not mean they are silent on it: an organization is cited where its document takes a position we could locate, and the absence of a citation is the absence of a finding, not a finding of absence. Who is represented lists everyone working on this topic, including those not cited above.
Where they disagree
No contradictions recorded on this topic yet.
The issues in full
Each issue carries the organizations that name it, the numbers behind it, and the remedies proposed - with the concrete steps under each. Every citation points at a section of a named document, so any count here can be checked.
Issue 012 organizations name itnewest evidence Apr 2026
Nobody can list where the organization is exposed, so assurance has nothing to cover
AI has spread through business units, employee tools and back-office integrations faster than any register of it. Without a list of what exists, assurance covers the systems somebody remembered.
Every other control in this theme presumes you know what you are protecting, and that presumption usually fails first. The reported gaps are self-assessed and wide - insufficient safeguards for AI data management, telemetry and observability short of what is needed, and a large majority reporting a rise in unsanctioned use by employees. An inventory is unglamorous and it is the precondition for saying anything defensible to a board or a reviewer, because an assurance statement about an unknown denominator is not an assurance statement.
Build the inventory before promising assurance over anything
Produce and maintain a list of every AI system in use, including the ones nobody approved, and state coverage against it.
Done when An inventory of every AI system in use exists, including employee tools nobody approved, each entry either names an owner or is marked as having none, and coverage is stated as a fraction of it.
List every AI system in production, including employee tools nobody approved.0-30 daysCISO
Name an owner for each, and mark which have no owner - those are the finding.30-90 daysHead of risk
State assurance coverage as a fraction of the inventory, never as a bare count.90-180 daysCISO
The evidence — 7 documents
Organization
Document
Position
AWSHyperscaler · April 2025
Navigating the security landscape of generative AIOur reading Names the mechanism that makes the inventory wrong rather than merely incomplete: where an organization bans this outright, or simply moves slowly, people reach for consumer applications instead, and that usage lands nowhere any register can see it. The stricter the stated position, the less the list describes what is actually running.Restriction is what empties the register
names it
IBMHyperscaler
Securing enterprise AI at scaleOur reading Reports that organizations often fail to see where they are exposed, with 38% conceding insufficient safeguards for AI data management, 42% for telemetry and observability, and 76% reporting a rise in unsanctioned AI use by employees.Exposure that organizations cannot see
names it
AWSHyperscaler · April 2025
Navigating the security landscape of generative AIOur reading Pairs the two halves that only work together: put approved tooling in front of people so the ordinary path is a visible one, and instrument for what still goes around it - tracking which models are live, what is being sent to them and returned, whether the guardrails engaged, and endpoint signals for the tools nobody declared.Sanction something, then instrument for the rest
proposes a fix
Cloud Security AllianceInstitution
AI Model Risk Management FrameworkOur reading Proposes a maintained per-model record and is candid that it decays into a snapshot without management sponsorship.A maintained record per model
proposes a fix
FS-ISACInstitution · April 2026
Preparing the enterprise for AI-enabled vulnerability discoveryOur reading Sets the bar for the register at same-day decisioning as risks emerge, which rules out a spreadsheet refreshed quarterly. Asks for dependencies and connections to be held alongside the assets, and for the internet-facing exposure to be known including the part of it that belongs to third parties - the half of the estate an internal inventory routinely stops at.An inventory that can answer the same day
proposes a fix
IBMHyperscaler
Securing enterprise AI at scaleOur reading Proposes continuously inventorying the AI attack surface across the enterprise as the basis for enforcing identity-based controls, rather than a point-in-time audit.Continuous inventory of the attack surface
proposes a fix
NISTInstitution · January 2023
AI Risk Management Framework 1.0Our reading Places mapping the context and the systems ahead of measuring or managing them, which is the same ordering.Map before manage
proposes a fix
Issue 021 organization name itnewest evidence Apr 2026
The control operates and there is no evidence it operated
A control is implemented and believed effective. What exists to demonstrate it ran, over a stated period, to somebody who was not there, is assembled retrospectively or not at all.
The asymmetry is severe: designing the evidence alongside the control costs very little, and reconstructing it afterwards from logs that were never intended for the purpose costs a great deal and often fails. What a reviewer asks is consistent regardless of regime - what was the control, over what period did it operate, what shows that, and who would have known if it stopped. Those four answers are worth designing for once.
For each control, decide at build time what artifact demonstrates it operated, where that artifact lives, and how long it is kept.
Done when For each control on one live system the artifact that demonstrates it ran is named, with where it lives and how long it is kept, and a dry-run evidence pack has been produced and timed.
For each control on one live system, name the artifact that shows it ran.0-30 daysHead of risk
Set retention long enough to cover the period a reviewer would ask about.30-90 daysHead of data
Produce the evidence pack for one system as a dry run, and time it.90-180 daysCISO
The evidence — 6 documents
Organization
Document
Position
CISAOther · April 2026
Careful adoption of agentic AI servicesOur reading Names why logging an agent does not produce what an assurance question needs. Long reasoning chains and large context produce enormous logs that are frequently repetitive, loosely structured, or simply beside the point, so the work is extracting signal rather than capturing data. Underneath that sit two harder problems: agents spawn sub-agents and follow delegation chains operators cannot see, and identical prompts can yield different actions, so reproducing what happened is not guaranteed even with the record in hand.The log is voluminous and still not evidence
names it
AWSHyperscaler · April 2025
Navigating the security landscape of generative AIOur reading Points at how hospitals handle the same tension, where clinicians must reach sensitive records to do the work and blocking access carries its own harm: run the access log through automated anomaly detection rather than restricting the access, an approach it reports has surfaced staff opening records they had no business in.A precedent from somewhere the problem is older
proposes a fix
AWSHyperscaler · April 2025
Navigating the security landscape of generative AIOur reading Treats how much to log as a decision with a cost on each side, and says plainly that where it lands determines whether anything can be monitored, audited or answered afterwards. Log too little and there is nothing to reconstruct; log carelessly and the record becomes another copy of the sensitive material, which is why it asks for the content to be stripped or masked as it is written.The logging decision cuts both ways
proposes a fix
Cloud Security AllianceInstitution
AI Model Risk Management FrameworkOur reading Proposes a maintained record of each model, and notes candidly that it is a snapshot which decays without sponsorship and enforcement from management.Model cards as a maintained record
proposes a fix
NISTInstitution · January 2023
AI Risk Management Framework 1.0Our reading Places measurement and documentation inside the framework as a function in its own right rather than as an output of implementation.Measure as a named function
proposes a fix
NISTInstitution · July 2024
Generative AI Profile (NIST AI 600-1)Our reading Sets out documentation as a continuing lifecycle obligation, which is what makes evidence contemporaneous rather than reconstructed.Documentation across the lifecycle
proposes a fix
Issue 03Our analysis2026 evidence
Who checks the account of what happened
After an incident or a control failure the organization investigates itself and reports its own conclusion. Whether anything independent tests that account is a question the sources here do not settle.
This is not usually dishonesty; it is the ordinary limit of investigating a system you built with the people who built it. The mature pattern in the material is explicit about the remedy: external advisors brought in to validate the reconstruction, an independent assessment of the behaviour commissioned from parties with no stake in the answer, and a commitment to publish. For most organizations the proportionate version is smaller - one external review of the most consequential system - but the principle holds, and it is what turns an assertion into evidence. No organization in this index names the absence of independent verification. It is set out here as our own reading of how these accounts are produced, and the consensus count is zero for that reason.
Commission an external review of the most consequential AI system annually, with a scope you do not control and a finding you publish internally.
Done when An external review of the most consequential AI system, commissioned from a party with no stake in the result, is complete within the last twelve months and its findings have been to the board.
Choose the system whose failure would cost most, and scope an external review.0-30 daysCISO
Commission it from a party with no stake in the result, and do not set the conclusion.30-90 daysHead of risk
Take the findings to the board with what was changed as a result.90-180 daysCISO
The evidence — 3 documents
Organization
Document
Position
Cloud Security AllianceInstitution
AI Model Risk Management FrameworkOur reading Places independent challenge inside the model risk process rather than leaving verification to the team that built the system.Independent review in the risk process
proposes a fix
NISTInstitution · January 2023
AI Risk Management Framework 1.0Our reading Proposes proportionate independent assessment, scaled to consequence rather than applied uniformly.Independent assessment where stakes justify it
proposes a fix
OpenAIFrontier lab · July 2026
Security incident during model evaluation, with Hugging FaceOur reading Records external advisors engaged to validate the reconstruction of what the models did, an independent third-party assessment of the observed behaviour commissioned separately, and a commitment to publish a technical report.External validation and third-party assessment
proposes a fix
Who is represented
This dossier is drawn from 30 organizations working on the subject, 7 of which are cited directly in the issues above.
Consultancy — 9
Deloitte 3Boston Consulting Group 2EY 2McKinsey & Company 2Capgemini 1Heidrick & Struggles 1KPMG 1PwC Malaysia 1UST 1
Institution — 7
Cloud Security Alliance 3NIST 3FS-ISAC 1World Economic Forum 3Association of Corporate Counsel 1Institute of Directors 1Moody’s 1
Academic — 2
Carnegie Mellon SEI 1National Bureau of Economic Research 1