Audit the environment, not just the model
Why RL environments need independent adversarial audits.
Third-party evaluation of finished models has become one of the few shared instruments of AI safety in just two years, and it works. But it stops at the model — one layer above where behavior is actually shaped. The RL environment that trains a frontier model decides what gets reinforced, and the verifier inside it is a hand-built, imperfect proxy for the task.
When we audit the public agentic benchmarks built from the same tasks, graders, and harnesses, weak verifiers, leaked answers, and tasks that pass without being solved turn out to be pervasive — and frontier labs report the same reward hacking inside their own training. A verifier that can be passed without solving the task reinforces the wrong behavior, erodes the very monitors meant to catch it, and at worst seeds broader misalignment. Unintended by the developer, it is already a safety problem.
This is not a gap that more internal QA closes: a single team searches its own environments with its own blind spots, and the gap only widens as environments grow more long-horizon and complex. What covers the tail is independent, diverse, continuous search.
So we argue that a high-stakes RL environment should face an independent adversarial audit before — and while — it shapes a frontier model, extending the accepted practice of third-party model audit one layer deeper, to the training input itself. The hardest objection, exposing a lab's intellectual property, is met with controlled, monitored access; one effective design is a bug bounty, an incentivized market that pays for diverse adversarial search. Because it audits a training input rather than the final model, it need not delay a release — and because a hackable environment corrupts a lab's own training, labs have a private reason to build this ecosystem with safety organizations through voluntary pilots rather than waiting on a mandate.