Compliance Gap

In Why Do Some Language Models Fake Alignment While Others Don’t?, a compliance gap is a difference in harmful-request compliance between inferred training and deployment contexts.^[1]

The measurement can guide investigation of Alignment Faking, but a gap alone does not identify its cause. Scenario perturbations, refusal behavior and reasoning analysis help distinguish possible explanations. Lack of a gap in one setup is also limited evidence about a model’s behavior elsewhere.