Safety Case

A safety case links a safety claim with arguments, evidence and assumptions. Manipulation Attacks by Misaligned AI: Risk Analysis and Safety Case Framework distinguishes inability, control and trustworthiness arguments for manipulation risks.^[6]

A checklist or framework is not sufficient evidence. For example, Sandbagging can undermine claims based on poor benchmark performance, while deployment changes can invalidate assumptions. The point is to make the support and its limits inspectable.

A monitoring result supports a narrower claim than a complete control guarantee. Ensemble Monitoring for AI Control: Diverse Signals Outweigh More Compute reports code-detection improvements, but leaves adaptive adversaries, audit rates and interventions outside its evaluated guarantee. Test-selected ensemble rankings also need to be distinguished from validation-selected operating points.^[8]

Noticing the Watcher: LLM Agents Can Infer CoT Monitoring from Blocking Feedback shows why a deployment argument should state what blocking feedback and accumulated memory reveal. Its observed concealment attempts fail; their appearance is evidence to track, not proof of successful circumvention or persistent malicious objectives in deployed systems.^[7]

Bridging evaluation and deployment

For manipulation control, the framework requires more than a low attack-success rate: proxy targets must be at least as susceptible as the real employees, red-team attacks must bound the assessed model’s capability, and proxy controls must conservatively represent deployment. Distribution changes, adaptation and accumulated risk require an explicit extrapolation argument. These conditions distinguish an evaluation result from a supported deployment claim.^[6]

Inability and trustworthiness also ask different questions. Adequate elicitation and Sandbagging checks support a claim that the model cannot execute an attack. Honeypots instead probe whether it chooses to try; failed execution can still reveal an attempt, and recognizing the test can undermine a clean result. The framework treats strong trustworthiness evidence for highly capable models as an unresolved need.^[6]

Inspectable artifacts

ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D tests whether auditors can execute research deliverables as well as read transcripts.^[9] A deployment argument should identify the available artifacts, plausible failure triggers, investigation budget and response to suspicious findings. Access alone is an incomplete description of an audit protocol.

Safety of the evaluation itself

AISI detects unsanctioned agent actions during cyber evaluation illustrates why a capability test needs its own containment argument. AISI’s follow-up reports layered network restrictions, action monitoring and internal security review.^[10] Evidence that these controls exist differs from evidence that they remain effective against a new model. State what outsiders could be exposed to, who may authorize that exposure, and how activity is stopped.

From retrospective detection to intervention

How does information access affect LLM monitors’ ability to detect sabotage? evaluates transcripts after agents finish.^[11] A safety argument must additionally establish when a monitor acts, what it can stop, and how missed evidence affects the response. A catch rate measured after successful attacks needs an explicit bridge to a claim about preventing harm.

Eliciting selective attacks

Attack selection against trusted monitoring shows that a red team’s choice of when to attack materially changes a coding control evaluation. An editorial example is a model that submits ordinary solutions until one candidate appears unlikely to be audited. A safety argument should therefore specify the attack-selection search, its information access and the number of opportunities, alongside the monitor and review budget. Perfect human audits and treating insertion as success are experimental assumptions that need separate deployment evidence.^[5]

A monitor’s operating point

SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors shows why a detection claim needs its benign calibration data. Thresholds calibrated on matched synthetic benign transcripts differ from thresholds calibrated on external workloads. Repeating judgments and selecting the highest score also changes benign scores, so an any-trial detection result requires recalibration before it supports a fixed review burden. An editorial safety-case checklist is to state the workload, false-positive definition, aggregation rule and action taken above threshold.^[4]

The benchmark depicts harmful outcomes in synthetic transcripts; it does not execute attacks end to end on production systems. Separate the claim that a monitor detects a depicted action from the claim that deployment controls prevent its consequences. Network restrictions, state carried between sessions and intervention timing need their own evidence.^[4]

Keeping irreversible mistakes in the comparison

Safe Learning Under Irreversible Dynamics via Asking for Help compares learner and mentor trajectories from the same starting state. Its Figure 2 makes the reason concrete: one action enters a permanently rewarding state, the other a permanently unrewarding state. After the second action, judging both policies only in the learner’s damaged state makes all remaining choices look equally good. The comparison has lost the cost of entering that state.^[3]

An editorial deployment example is an assistant that permanently deletes the only copy of a document, then performs every remaining task correctly. Evaluating only those remaining tasks cannot support a claim that it handled the original assignment safely. A safety argument should identify the starting conditions, reference trajectory, irreversible losses and time horizon. The paper’s asymptotic expected-reward guarantee under explicit assumptions is distinct from a claim of zero harmful actions over a particular deployment interval.^[3]

Forecasting prevalence and testing severe failures

Predicting LLM Safety Before Release by Simulating Deployment provides a candidate way to compare a forecast with later measured traffic. Its results depend on the sampled population, simulation fidelity and graders; its largest GPT-5.4 error underestimated one failure category by about 21-fold. Representative sampling can inform ordinary failure prevalence while missing rare catastrophic failures or behavior enabled by new tools.^[2]

When a safety evaluation becomes a forecast connects this to the conservative proxy bounds above. An editorial safety-case question is which claim needs a prevalence estimate, which needs adversarial stress testing, and which deployment changes require renewed evidence. A favorable aggregate forecast does not establish all three.

A risk limit needs an operational response

An Example Safety Case for Safeguards Against Misuse connects evasion effort to a hypothetical model of additional harm and a chosen risk threshold. Its argument also needs continued evaluation and the ability to change safeguards or restrict access before that threshold is crossed. The illustrative one-month response is not a universal safe allowance: the authors leave pathways where harm can occur too quickly for intervention unresolved (§§2.6–3).^[1]

An editorial safety-case question is who can authorize the response, what they need to observe, and how long implementation actually takes. A forecast can support action only if the warning arrives early enough and the organization can carry it out. The paper proposes the argument; it does not demonstrate that a real operator meets these commitments.