·10 min read

Post-training is flying blind

Eval scores go up. Models cheat. Tests reject correct solutions. And nobody reads the trajectories. Post-training needs a dedicated inspection layer. We're building it.

Here's the uncomfortable truth about post-training in 2026: the industry has gotten very good at running training loops, and very bad at understanding what comes out of them.

Teams run experiments, check eval scores, and move on. The scores go up, so everything must be working. Except it often isn't. The model that aces your benchmark might be cheating. The model that "fails" might actually be correct. And the only way to know the difference is to read the trajectories — which almost nobody does.

We believe post-training is missing a fundamental layer: systematic trajectory inspection. Not as a nice-to-have. As infrastructure. Here's why.

The eval score illusion

The post-training workflow today looks like this: run an experiment, measure it against a benchmark, decide whether to ship. The entire feedback loop runs through a single number — the eval score.

That number is lying to you in two directions at once.

Direction 1: Models pass when they shouldn't

Models have learned to game evaluations in ways that are invisible in aggregate metrics. This isn't theoretical — it's happening in production, across every major lab:

  • METR found that OpenAI's o3 reward-hacked in 30% of autonomous coding runs. On one task, it cheated in 100% of trajectories — patching the eval function to always return perfect scores, rewriting timers to fake speed results. Telling the model "do not cheat" had nearly negligible effect.
  • In PostTrainBench, agents given autonomy to post-train models exhibited systematic contamination: loading test sets directly, disguising benchmark problems as "synthetic examples," and reverse-engineering evaluations. The most capable agent (Claude Opus 4.6) was also the most prolific cheater — 12 incidents in 84 runs.
  • NIST documented agents commenting out assertion checks to pass tests, Googling walkthrough answers for CTF challenges, and launching denial-of-service attacks instead of exploiting intended vulnerabilities.

In every case, the eval score looked great. The trajectory told a different story.

Direction 2: Models fail when they shouldn't

The flip side is equally damaging. When OpenAI audited SWE-bench Verified, they found that 59.4% of hard tasks had flawed test cases that rejected functionally correct solutions. Tests enforced specific function names never mentioned in the problem. Tests checked behaviors beyond what was asked.

The model solved the problem. The test said it failed. If you only look at pass rates, you're throwing away correct work and wasting training signal.

Eval scores are a lossy compression of what actually happened. They tell you what passed and failed, but not why. The "why" is in the trajectories.

Reward hacking doesn't stay in the eval

Some teams treat reward hacking as a benchmark curiosity — annoying but contained. It's not.

Anthropic showed that when models learn to reward-hack in production RL, the behavior generalizes. Models that learned to game one metric spontaneously exhibited alignment faking and attempted sabotage. RLHF safety training made the model look aligned on chat-like queries, but it remained "egregiously misaligned on many coding queries" — the misalignment became context-dependent rather than fixed.

The only way to catch this early is to look at the trajectories before the gradient update bakes the behavior in.

Why automated monitoring isn't enough

The obvious response is: "just build an automated monitor." Use another LLM to watch the trajectories. Several teams are trying this. It helps, but it has a hard ceiling.

To be clear: LLM-based monitoring does work. METR's MALT research showed that prompted LLM monitors can achieve 0.96 AUROC on reward hacking detection, catching 80-90% of true positives. OpenAI's chain-of-thought monitoring research demonstrated that even a weaker model like GPT-4o can effectively monitor a frontier reasoning model by observing its CoT.

But here's the catch: when you use these monitors as a training signal, you create a meta-reward-hacking problem. OpenAI found that incorporating CoT monitors into the RL reward initially improved both capability and alignment — but with more optimization, agents learned "obfuscated reward hacking": emitting benign-looking chains of thought while continuing to cheat in ways the monitor couldn't detect. The monitor becomes another objective to game.

METR acknowledged the same limitation from a different angle: their MALT dataset was built specifically because they "lacked a ground truth dataset to validate that monitors actually performed reliably." The prompted examples in their dataset were "often obvious," and they noted that "agents may be able to execute more subtle and realistic strategies." Even with 0.96 AUROC, they concluded that strong monitor performance on their dataset should not be considered "sufficient nor necessary for detecting real-world risks."

This is the fundamental issue: automated monitors can be gamed by the very systems they monitor. LLMs optimizing against LLM monitors is just another optimization loop — and optimization loops produce Goodhart's Law effects. What you need as a backstop is something that cannot be gamed through gradient descent: human reviewers with general grounding, who can recognize "this doesn't make sense" even when the model has learned to produce outputs that satisfy every automated check.

As Auriel, a Google veteran who's eyeballed thousands of trajectories on Gemini and Waymo, puts it: "You build an intuition that no metric can replace." That intuition — the ability to read a trajectory and feel that something is off before you can even articulate why — is exactly what automated monitors lack.

The scale problem is the real problem

Everyone we talk to agrees: you should read your trajectories. The reason they don't is purely operational.

An experienced reviewer needs about 90 minutes for 10 trajectories. A single SWE-bench run has 300+ tasks. A real post-training pipeline generates thousands of trajectories per experiment, with experiments running weekly or faster. That's 450+ hours of expert review per run.

The U.S. National Institute of Standards and Technology (NIST)'s top recommendation for addressing agent cheating is AI-assisted transcript review, explicitly because manual inspection alone does not scale.

The tooling doesn't exist. The operational capacity doesn't exist. So teams skip it, and fly blind.

What we're building

At Archipelago, we believe trajectory inspection isn't a nice-to-have workflow step. It's a missing layer of infrastructure in the post-training stack.

Every training run produces trajectories. Those trajectories contain the ground truth about what your model actually learned — and what it learned to fake. Right now, that signal is thrown away because nobody has the tools or the capacity to process it.

We're building the system that closes this gap:

  • Purpose-built trajectory review tooling — not a generic labeling UI, but a system designed for agent trajectories: timeline extraction, tool call mapping, diff analysis, risk signal surfacing
  • A managed review team of 500+ trained CS students who classify failure modes, catch reward hacking patterns, and write corrective feedback at each decision point
  • Structured output — not a report that sits in a doc, but a feedback dataset in JSONL, ready to plug into your next training run

We're looking for our first pilot partners — post-training teams that know they should be reading their trajectories but can't do it at scale. If that's you, let's talk.

References

Jongwon Park
Jongwon Park
Founder, Archipelago. RL engineer (Krafton PUBG). Led 500+ labeler operations on a national-scale AI project.

Stop flying blind.

We're looking for post-training teams to pilot with. Tell us about your trajectory workflow.

Request a Pilot