measurement integrity

AI writes code fast — but a 2025 Veracode study found models chose insecure code 45% of the time, and a METR trial found developers 19% slower. Here is the gap.

CRUCIBLE is a governance protocol that audits whether an AI agent's own test-and-coverage system provides evidence not controlled by the same producer. Three forensic cases: 3,277 passing tests hid silent data loss; a 77% coverage badge masked an audit estimate of ~15%; 4,726 passing tests could not prove real-database reachability.

Measurement fraud is when the AI system that produces code also grades it, so evals pass while the code is broken. Definition, how to detect it, and how to prevent it.