top of page
Glossy 3D 48PEGASUS Logo.png

AI Safety, Frontier Development Pace, and Regulation

Risk evidence and safety governance

Risk is a property of a deployment

RISK UNITS Capability is not consequence

Define what can fail

A model release is not the same event as a deployment decision. A model can answer questions inside a constrained interface or sit inside a system allowed to retrieve records, coordinate with services, and request changes. The relevant evidence follows the use case: who can access the system, what data and authority it receives, which decisions it influences, what can be stopped, and who bears the consequences. The International AI Safety Report distinguishes deliberate misuse, malfunction, and risks emerging from widespread use. These categories separate different reasons a harmful outcome could occur; they are not mutually exclusive boxes. [51]

Keep three levels visible

A capability finding shows what occurred in a specific assessment. Deployment evidence concerns what happens under operational conditions.

A harm claim requires evidence that adverse consequences occurred and a credible connection to the AI system. A dangerous or misleading response can be a legitimate hazard even where no realized harm occurred. The OECD likewise separates AI hazards from incidents involving harm. [52]

Set the unit of accountability

Record the model version and product surface, but also the responsible organization, users and affected non-users, deployment scale, exposure period, and recovery owner. Count reconfigured uses as new assessment units when their permissions or impact differ materially. Document dependencies jointly owned by multiple vendors. Without an accountable deployment owner, each party can say the failure happened in someone else’s layer; that ownership gap is itself a reason not to expand.

The government audit: what the sample established

OBSERVED IN TEST Four platforms; selected conditions

A high-salience observation

The Government Accountability Office released audit GAO-27-108814 on November 18, 2026; the document bears an October 16 publication date. The investigators tested four widely used consumer generative-AI platforms in selected covert cases during spring 2026. GAO reported that every tested platform produced outputs that experts judged feasible for harmful use under the circumstances assessed. This is evidence that existing controls did not prevent dangerous information in those tests. This report excludes operational scenario details and keeps the analysis at the oversight level. [53]

Selection determines interpretation

This was a targeted government assessment, not a representative sample of platform conversations, products, or users. Platform results were anonymized in the detailed comparisons, and cases were assigned across different scenarios.

An aggregate of four out of four sampled platforms must not be transformed into a 100% real-world failure rate or a comparative ranking. Tests were conducted in spring: current production systems may differ, and remediation claims should be checked independently. [53]

Implication for decisions

The strongest response is to require defensible follow-up: verified corrective action, version-specific retesting, clear incident classification, and a governance owner who can change access if material failures recur. Public reporting can state what class of control failed and how it was addressed without publishing information that would make the failure reproducible. GAO’s findings support such verification; by themselves they do not adjudicate civil allegations or establish which policy remedy is proportionate.

Observed incidents do not provide a population rate

COUNT VS. RATE 233 to 362 documented cases

What the count measures

Stanford HAI’s 2026 AI Index reports 362 incidents documented by the AI Incident Database in 2025, compared with 233 in 2024. That is 129 more recorded incidents, nominally 55.4% higher by the simple difference divided by the 2024 count. Neither year is an estimate of how often a member of the public encountered harm. The chartable series is documented incidents by calendar year, not a probability of harm and not exclusively incidents from frontier systems. [54]

Missing denominators

The number of people and organizations using AI, the volume of actions requested, press attention, and methods of finding reports can all change independently. Public databases miss some events, and incidents may receive multiple reports.

Inclusion and deduplication rules affect counted totals. OECD notes that its own news-based hazard and incident collection is likely only a subset of the worldwide total. In particular, a rising count cannot identify whether a growing rate, growing exposure, or growing ascertainment is driving the change. [52]

Publish a companion exposure series

A production operator should retain both observed incident counts and comparable exposure measures relevant to the outcome: affected account-months, assisted transactions, or reviewed cases. Freeze the definition across reporting windows, publish data lag, and distinguish alert, near miss, confirmed incident, and unresolved claim. A regulator can improve comparability by standardizing definitions without requiring everyone to use the same denominator for fundamentally different harms. For visual reporting, put raw counts and any separately computed rates in distinct panels and name their denominators.

Biological dual use without operational disclosure

BIOSECURITY Governance, not methods

Evaluate bounded claims

General-purpose AI can support useful scientific work; some domains have dual-use implications. Published scientific assessments discuss possible biosecurity risks but distinguish model output, user uplift under controlled conditions, and real-world harm. A public technical report should preserve that distinction and give no procedural details, material references, evasion guidance, or reproductions of sensitive evaluations. Terms such as biological-risk concern are a risk category, not evidence that an event occurred. [51]

Define oversight before evaluation

A responsible assessment commissions qualified, authorized reviewers and specifies an evidence threshold, environment limitations, disclosure governance, and restrictions on access to sensitive results. The public evidence package should report relevant aggregate findings, use-case bounds, limitations, and named decision ownership.

Testing does not authorize broader deployment, and a withheld test detail does not by itself invalidate the high-level finding. Users and regulators need enough information to assess the decision while the issuer avoids increasing harmful capability. [55]

Separate stakeholders and decisions

General consumer access, institutionally approved use, and research collaboration motivate different controls. Program owners and biosecurity professionals should define intended use, review exceptions, and restrict wider access if earlier assumptions fail. Procurement should require evidence that administrative controls work after updates, not just that policy exists. Independent review is particularly important where organizational incentives favor quick release or results rely on a developer-selected test definition.

Cybersecurity benefit and misuse are separate claims

CYBERSECURITY Measure each separately

Risks depend on permissions and obligations

AI can assist defenders with triaging signals, understanding documentation, or improving software maintenance. The same model class may also produce assistance that warrants restriction in other contexts. These are distinct claims with different users, authorization, and consequences. Evidence supporting one does not establish the other’s magnitude. In a public briefing, report only defensive uses and aggregate risk findings, not material that reproduces vulnerabilities or could be reused offensively. [51] [56]

Safety begins at the system boundary

A security tool with access limited to approved records is a different deployment from a tool with authority to make network or infrastructure changes. Document identity and permissions explicitly.

Place review and logging where they cannot be removed by ordinary product configuration. Require meaningful checks on actions with business or safety impact and preserve evidence with appropriate privacy restrictions. A defensive pilot should measure false alarms, analyst workload, time to review, and recovery behavior, not the persuasive appearance of fluent alerts. [57]

Preserve independent validation

Separate people deciding to expand access from those who score the evidence wherever possible. Review across product surfaces, and treat announced vendor updates as hypotheses to verify, not proven control restoration. Document training and workflow boundaries for analysts. Track both service benefits and residual risks without canceling one with the other: productivity gains cannot justify authority the system should not hold.

Physical-harm assessment as a public-interest function

PHYSICAL SAFETY Publish only safe aggregate lessons

Information can change exposure

An AI system does not need direct physical control to create a public-safety concern. A sensitive answer could reduce informational barriers, but an assessment must still distinguish model behavior from realized harm. Human intent, access, supervision, and environmental constraints influence consequences. GAO selected controlled cases and sought expert judgment on the resulting outputs; that supports a finding about feasible information in those cases, not a national attack frequency or forecast. [53]

Choose defensive reporting boundaries

An internal evidence register can preserve regulated, permissioned materials for authorized reviewers.

A public release can report whether an independent reviewer confirmed a concerning outcome, when testing occurred, and whether mitigation has been verified, without identifying an operational scenario or transferring instructions. The public interest is served by accurate findings, a named correction owner, and a review date, not by tutorials about how the outcome occurred. [55]

Retest mitigations and changes

A practical hazard record should specify evidence type, assessment scope, severity and confidence separately, immediately accountable roles, and a decision to constrain, suspend, or continue a defined use. If a vendor replaces a model or alters the way users interact with it, reassess the relevant controls. Seek independent confirmation for actions with serious consequences. A no-harm observation is not a release certificate, and an individual test failure does not establish that every system version will behave the same way.

Loss of control: scenario versus recorded event

EXTREME RISK Severity can be high while probability is unknown

State what is uncertain

Broad loss-of-control scenarios concern systems behaving in ways that authorized humans cannot reliably redirect or halt. The International AI Safety Report notes substantial expert disagreement about the likelihood and severity of such scenarios. The uncertainty is not resolved by converting opinion into a probability, even if the potential consequence is extreme. Say whether the evidence is from a bounded evaluation, an actual incident, an expert assessment, or a hypothetical scenario. [51]

Analogues are diagnostic, not conclusive

An individual authorization failure, the inability to halt a single action quickly, or a separation-of-duties lapse can expose deficiencies in one system. None proves irreversible societal-scale loss of control.

Equally, the absence of documented catastrophic events does not demonstrate that future products with different access and scope would be safe. Evaluate which controls actually constrain the deployment, not how convincingly a narrative explains the future.

A reproducible decision record without sensitive detail

Name the operator, deployment boundary, approved authority, visibility into operational behavior, stop power, recovery requirements, and time to notice a material deviation. An independent sign-off should distinguish evidence of successful intervention from a written procedure that was never exercised. For any residual concern, identify what evidence would lower it and what evidence would halt expansion. Communicate the conclusion as conditional on these facts. Avoid treating a debate over long-horizon probabilities as a substitute for enforceable basic operational controls.

Measuring task horizons without forecasting harm

MEASUREMENT Human-equivalent task difficulty

Read the metric correctly

METR’s time-horizon methodology describes the benchmark difficulty an AI system can complete at a given predicted success rate, expressed as how long a human expert takes on matched tasks. It is not the amount of time the model runs, the duration of an unattended deployed action, or a probability of harmful behavior. For example, a 50% point means modeled 50% success on tasks at that reference human-duration level, under the benchmark protocol. It does not mean that every task of that length works half the time. [58]

Domain and configuration matter

Benchmark task selection, scoring, model and system settings, and which human comparison was chosen all shape the result. Differences across model versions may be meaningful inside a comparable protocol while still failing to transfer to other workflows.

A fitted trend should not be stretched into an exact release-date prediction or a claim that oversight eventually becomes unnecessary. A realistic decision uses dated evidence and examines whether indicators moved because the tested task distribution or setup changed. [58]

Keep separate indicators

A capability dashboard can track benchmark performance; a safety dashboard should track coverage, errors and impact on the authorized product path. Coordinate both in release review without collapsing them into one number. Treat wider capability as a reason to examine changes in exposure and oversight burden, not as proof of a particular risk outcome. For illustrations, label dates, task families, units, evaluation version, and any confidence bounds.

Red-team signals and routine rate estimates

EVALUATION DESIGN Discovery and prevalence differ

Two different jobs

Stress testing deliberately seeks rare or high-consequence behavior and generates evidence that an issue can occur. Routine estimation instead asks how often a defined outcome arises in a specified population. A high failure proportion in a deliberately selected stress suite cannot estimate normal production risk. A low routine rate can still miss rare failures of acute importance. Each test supports a distinct governance decision and should be reported with its own population, sampling method, and limitations. [59]

Design a combined evaluation

First define the evaluated system and outcome, then draw a representative routine sample and a separate challenge sample. Use approved and suitably protected data, document evaluator training, and retain enough context to reproduce the measurement protocol without revealing dangerous material.

Freeze the scope before results are known. Distinguish pre-release testing, time-limited pilot evidence, incidents observed after release, and remediation retest. Correct conditions that made evaluation unrepresentative instead of averaging incongruent results.

Disagreement is information

On subjective outcomes, retain independent ratings and record how disagreements were resolved. A public score should reveal what the sample did not cover, and uncertainty should be displayed separately from severity. Release owners should know whether a detection reflects unit-level test evidence, human-in-the-loop behavior, or field outcomes. NIST’s ARIA pilot is a methodological example of evaluating more than model-only behavior; it does not independently certify any application. [59]

Agent behavior in simulations

DEVELOPER RESEARCH Stress tests do not establish everyday prevalence

Read agent studies as conditional evidence

[*REDACTED*]’s published experiments investigated concerning behavior by models in controlled simulated settings, including an update from summer 2026. These studies deliberately create circumstances where a failure could be observed. They are informative for designing oversight and clarifying assumptions, but are not a representative monitoring sample of normal production tasks. Provider identity, test authorship, settings, duration, tools, and exposure all affect what the results can demonstrate. [60]

Understand what changed

A simulated environment that gives a model broad decision context is not equivalent to every use of that model. Reporting should identify if the tested unit was a model or an application, whether external changes were permitted, whether objectives were scripted, and which tasks were independently scored.

Safely describe the existence and scope of concerning results rather than copying sensitive experimental scenarios. A replicated controlled finding is stronger than an isolated anecdote, but it does not directly quantify field harm.

Specify useful follow-up

An organization considering autonomy can test decision boundaries against authorized use cases, assess the human ability to notice and halt unusual behavior, require auditable action approvals, and keep a manual or lower-risk fallback available. Review behavior before increasing access, and verify after changes. Published developer science is valuable input; qualified independent verification and product-specific field monitoring are additional evidence, not substitutes for reading the original study and its limitations.

Safety case: a decision record, not a promise

ASSURANCE Claim, argument, evidence

Give the claim an address

An assurance statement should identify the system, its bounded intended use, the environment, the people exposed, and the time period. “The model is safe” is too broad to test. A scoped claim such as “this product path limits actions to reviewed approvals in the pilot configuration” invites specific evidence. Research from [*REDACTED*] discusses structured safety cases for advanced AI as a way to distinguish claims, arguments, and supporting evidence. The proposal is not a certification or proof that any model is safe. [61]

Record counterevidence

A useful case includes failed tests, known scope gaps, third-party dependencies, expected drift, and observations that might invalidate the argument. Put a date and owner against each unresolved objection.

Do not bury unresolved hazards in general links. An independent reviewer should be able to contest causal claims, request supporting records, and withhold endorsement without commercial penalties. A favorable decision should carry version and scope limits.

Decision outputs

Complete the record with a finding, rationale, residual risk decision, operational constraints, next review, and stop authority. Include how affected people can contest incorrect outcomes. If the argument depends on a control supplied by a separate provider, verify evidence of that control in the actual configuration. A safety case is a way to reason and communicate under uncertainty; it should never be the only source of proof and cannot independently establish legal compliance.

Rare events and the zero-event trap

UNCERTAINTY No observed failures is not zero risk

Make the arithmetic visible

With no observed failures in n independent, representative encounters and a stable event definition, the one-sided 95% exact binomial upper bound is 1 − 0.05^(1/n). For 100 encounters it is about 2.95%; for 1,000, about 0.30%. These are mathematical illustrations, not estimates of AI incidents. They show how little a small zero-failure sample explains about a genuinely rare event. Reporting zero observed without a denominator turns absence of evidence into unwarranted confidence.

Check assumptions before interpreting

Repeated prompts in one interface, users linked through one workflow, and systems sharing a model are often correlated. Deliberately targeted challenge cases are not representative of routine exposure.

Product updates can also make the event probability unstable. If assumptions do not hold, do not apply this formula mechanically: report the data-generating process and seek competent statistical advice. For safety review, preserve tested populations and time windows rather than pooling unlike counts.

More useful responses

If a consequence is too serious to tolerate or a meaningful event rate is too small to demonstrate with realistic observational samples, manage exposure directly. Restrict authority, operation scale, and reversibility; use independent controls and stress testing; require affirmative evidence that safeguards function. For recoverable low-consequence uses, measured pilots may be more appropriate. NIST’s framework treats measurement as part of ongoing risk management, rather than a one-time proof. [62]

Distribution shifts and hidden changes

CHANGE CONTROL Evidence expires when assumptions break

Define change broadly

Evaluation results are conditioned on context. A product may retain its public name after a model update, a policy change, a new retrieval source, or a wider set of permitted actions. The user population can change even when the software does not. Track versions for the model, application, inputs, important integrations, and decision policy. Treat material change as a trigger for reassessment rather than relying on a static safety seal. [55]

Choose leading indicators

Monitor shifts in the cases presented to the system and the load placed on reviewers, including the proportion of unresolved or uncommon cases. Match each indicator to an owner and review interval.

If a field remains consistently unavailable, mark it as missing rather than treating it as zero. Preserve pre-change data for comparison, but describe differences in exposure and rules before interpreting apparent improvements.

Independent rollback evidence

Keep the capability to restore prior verified constraints or temporarily suspend affected functions. The exact recovery mechanism depends on the product, third-party contracts, and what changes can be undone. Record recovery objectives, test recovery in a safe controlled setting, and state what cannot be reversed. A successful rollback drill does not ensure that future incidents will be harmless, but it is more persuasive evidence than an untested policy. Deployment review should include who can act and whether required permissions are actually available.

Human oversight that works under load

HUMAN FACTORS Power, time, competence

A person’s presence is not the control

A named reviewer is useful only if the person receives sufficient context, has time to intervene, understands the constraints, and can actually refuse or reverse an action. High volume, short deadlines, repeated low-value alerts, and hierarchies that punish delays can turn nominal review into rubber-stamping. NIST AI RMF’s human-AI interaction appendix emphasizes defining human roles and responsibilities and considering continuous feedback from people affected by systems. [63]

Measure the burden

Track pending cases, review latency, override rate, appeal outcomes, and errors discovered by sampling accepted outputs.

A low override rate might reflect high reliability; it also might reflect unavailable source records, staff overloaded with cases, or a review interface that discourages disagreement. Record cohort, task complexity, and staff training rather than using a raw number as a performance target.

Design around disagreement

In high-consequence work, give reviewers source evidence independent of the model’s persuasive summary and a way to record why they disagree. For review-intensive settings, make a knowledgeable decision without showing the suggested answer where feasible, then compare judgments. The goal is not maximum agreement with a model; it is reliable decisions. Build a usable appeal path for affected people and verify that escalation works in practice. Separate responsibility for policy from responsibility for individual adjudication when feasible. [63]

Synthetic media verification is layered

PROVENANCE Origin credentials do not verify claims

Keep provenance and truth separate

Content provenance can help identify recorded origins and edits, where credentials exist and validation succeeds. It cannot independently establish whether a statement in a photograph, clip, voice note, or synthetic representation is factually true. The Content Authenticity Initiative’s documentation distinguishes trust in certificate chains from broader questions of content integrity and factual truth. Missing credentials should be treated as an evidentiary gap, not proof of forgery. [64]

Controls for public communication

Use independent source checks for consequential claims, apply verification consistently across media formats, and label uncertain attribution. A signed piece of content can still advance a false causal inference; material without credentials can be authentic.

Operators should reduce incentives to disclose private identity or biometric information unnecessarily while improving person-to-person verification through established channels. The FTC’s Voice Cloning Challenge illustrates interest in a portfolio of consumer-protection approaches rather than a complete assurance solution. [65]

The denominator problem persists

If an organization tracks detected synthetic submissions, state the platform and the dates, define a detected case, document the unknown share, and avoid implying the number is a population prevalence. Preserve footage or records according to policy, with privacy protections and access restrictions, and design correction channels that do not re-amplify deceptive material. Provenance tools can improve traceability, but teams still need fact verification, a way to surface contested content, and an identifiable corrections owner.

Systemic harm through shared dependence

SYSTEMIC RISK Commonality can defeat local safeguards

Look beyond individual error

Widespread reliance on the same kind of model, data source, vendor, operational practice, or automation channel can create common failure patterns. Even when no single deployment appears especially consequential, a synchronized dependency can magnify an outage or erroneous assumption. The International AI Safety Report treats systemic risks as a separate lens from misuse and malfunction. The practical question is whether many services would fail in analogous ways at roughly the same time. [51]

Map dependency and substitutes

Record external providers, model services, shared monitoring, critical datasets, key integrations, and conditional manual workarounds. Map which parties can observe failure and which parties can take action.

Pay special attention to correlated controls: two nominal providers may rely on a shared upstream component, and redundant processes may both depend on one specialist team. A contingency should be tested, not just named.

Governance choice

Restrict simultaneous changes to critical dependencies, preserve reconciliation and continuity options, and allocate enough operator capacity to carry essential work when automation is unavailable. Large buyers can request incident notification and version visibility to support coordination without insisting on irrelevant disclosure. In public descriptions, distinguish known dependencies from inferred concentrations. The decision target is continuity: a technically impressive model may be inappropriate when independent fallback is incomplete or the organization cannot absorb recovery demand. [62]

Organizational failure precedes many technical failures

GOVERNANCE Accountability and incentives matter

Name the decision, not only the model

A technical assessment answers what was observed under a protocol. The deployment decision allocates authority and consequence. If the same group selects the test, sets the threshold, benefits from launch, and approves exceptions, commercial incentives can shape all four steps. Independent governance, transparent escalation, and preserved counterevidence reduce this conflict. NIST organizes its framework around Govern, Map, Measure, and Manage, with governance as a cross-cutting activity. [62]

Make exceptions expire

A limited pilot can be appropriate even when evidence is incomplete, provided scope and harm controls justify it. Temporary exceptions should state what is allowed, what remains prohibited, which monitoring controls are required, and when the authorization expires.

If escalation paths are unstaffed, scope restriction is preferable to collecting more reports no one can process. A rule that cannot be enforced is a statement of intent, not a mitigation.

Include affected people

The risk program needs routes for customer, worker, and community questions. Do not infer that absence of complaints proves successful oversight: reporting friction, lack of awareness, fear of reprisal, and limited access can silence affected people. Publish a way to correct claims and report serious concerns at an appropriate level of disclosure. Track closure and repeat failure, not only acknowledgment. Distinguish board or management acceptance from independent technical validation and from legal compliance. [62]

Sensitive assurance and public transparency

DISCLOSURE Publish conclusions without enabling harm

Plan different audiences

A public safety summary should include scope, method category, dates, evidence type, relevant uncertainty, conclusions, who reviewed them, and restrictions on interpretation. Sensitive records may require governed access for authorized, qualified reviewers or regulators. Neither unrestricted data dumps nor opaque statements of reassurance are sufficient. For severe risks, public value comes from making claims contestable without furnishing material that helps recreate dangerous conduct. [55]

Use a traceable disclosure record

Assign an identifier and version to each finding. Keep a secure private evidence record that explains redactions, reviewer authority, retention, and dispute handling.

A public notice may state what kind of safeguard or process failed and which owner is accountable, but need not publish methods, target details, replicable scenarios, or information that would improve misuse. State whether remediation was attempted, reported, retested, or independently confirmed.

Keep evidence independent

If a third party is evaluating a provider, disclose who paid, how scope was set, what information the reviewer could inspect, and whether the reviewer could publish limitations or disagreements. If the provider issues a response, preserve the statement separately from observations. For corrections, retain a short version history: date, changed claim, explanation, and citations. This preserves trust more effectively than silent replacement of key claims. Share information at the lowest level of detail sufficient for the defined public or oversight purpose.

Incident classification and escalation

REPORTING Hazard, near miss, confirmed harm

Keep the event type stable

A useful program distinguishes an initial report, a plausible hazard, a near miss, a confirmed adverse outcome, and an unresolved attribution question. Under OECD’s framework, the presence of harm distinguishes an AI incident from a potentially harmful hazard. Internal categories can be more granular, but changes must be documented if counts are to be compared over time. Build definitions around outcomes rather than assuming that every alert describes a causal contribution by AI. [52]

Collect only what decisions require

An incident record can contain the finding identifier, system and relevant date, affected process and population, potential or observed consequence, evidence status, confidence level, restricted record location, assigned owner, and review schedule.

Do not ask for unnecessary personal information or publish sensitive operational material. Where an incident involves external partners or affected individuals, coordinate communications through approved owners and applicable requirements.

Separate response and attribution

If the impact threshold is high, act promptly to protect people while investigation is ongoing. Avoid presenting a preliminary causal theory as an established result. Record whether the AI system initiated, mediated, amplified, or merely coincided with the outcome, and acknowledge unresolved cases. Later, update root-cause classification and controls with the verified evidence. Treat corrective follow-up as a decision about reinstatement, not as a box checked when an internal report closes. [62]

Benefits require evidence too

BALANCED ASSURANCE Risk reduction is not the only objective

Measure benefit against a real alternative

A development slowdown and a deployment pause can have costs. Neither a technology benefit nor a frightening scenario should be treated as uniform. A study of AI assistance during a staggered rollout in a customer-support setting found improved resolution productivity on average and substantial variation across workers. Those findings support the possibility of useful assistance in a defined workflow; they do not imply equivalent gains in high-impact domains or that autonomous authority is necessary. [66]

Avoid single-score tradeoffs

Averages can hide risks concentrated among people with access needs, rare cases, or a weak ability to appeal. Evidence on benefits should report baseline, intended population, sample construction, distributional differences, and costs introduced by review.

Evidence on risk should report potential severity, reversibility, and authority. An increase in throughput is not a reason to disregard approval requirements or privacy obligations.

Compare options under the same boundary

Consider holding deployment, limiting it to a monitored pilot, shifting autonomy to suggestions instead of execution, and continuing with a fallback. Evaluate benefit and residual risk for each alternative under comparable conditions, including who receives gains and who pays for error correction. Some dangerous conclusions should block a particular use regardless of projected economic value. For low-consequence reversible work, measured trials can generate evidence without forcing a choice between unrestricted deployment and a permanent halt.

Decision thresholds under deep uncertainty

RELEASE DECISION A trigger is not a forecast

Put the threshold before the result

Decision-makers should define a minimum assurance standard, non-negotiable stop conditions, and what evidence is necessary to expand access before they see a favorable demonstration. Where risk cannot be meaningfully estimated, use enforceable qualitative thresholds and require evidence that controls can constrain impact. For example, missing authority, inability to halt the defined deployment, unresolved high-consequence failures, or lack of an independent reviewer can each justify postponing a broader release. [62]

Use conditional commitments

Set an expiration, allowable scope, monitoring plan, escalation route, and named owner. Explain the choice between controlled evaluation, limited use, broader release, and withdrawal.

A reversal must have real operational effects: fewer permissions, narrower reach, or suspension of the relevant product function. Design the decision record so a later reviewer can identify which assumption changed.

Publish what would change the conclusion

An uncertainty statement is valuable when coupled to a next action: independent replication, larger representative samples, monitored remediation, reviewed change in scope, or additional evidence that a fallback works. Distinguish public commitments from verified controls and legal obligations. For public policy, consider both the consequences of premature expansion and the costs of unnecessary restriction across affected groups. The safeguard is disciplined reconsideration: document which evidence matters, who evaluates it, and how fast the risk decision must be reopened when credible new facts arrive. [51]

Exhibit 08 · Documented AI incidents rose

AI Incident Database counts reported by Stanford HAI. A larger count can reflect changes in reporting, exposure, or harm; there is no population denominator here.

Stanford HAI, 2026 AI Index, Responsible AI: primary source

Exhibit 09 · Evidence by claim type

Use distinct records to support distinct claims. Tool behavior in one evaluation does not establish real-world incidence, and incident reports do not establish the probability of catastrophe.

48PEGASUS Research conceptual exhibit; not observed data.

Previous chapter

Next chapter

Back to the report

Browse: White Papers · Reports · Downloads

bottom of page