We held a meetup in Korea

On the evening of September 9 we co-hosted Benchmark Social 0.1: RL & Post Training with Team Attention at NAVER D2SF Gangnam in Seoul. The idea was to get people in Korea who work on RL and post-training into one room and share what we have tried, including what failed.

Our talk was “The Post-Training Era: How Do We Improve RL Data?” It had three parts.

• Background. In pre-training a model only reads the internet. In post-training it learns by solving tasks that experts build. A task is three things: a prompt, an environment, and a verifier.
• Problem. Open up a public agentic benchmark and defects keep turning up. When OpenAI built SWE-bench Verified, human annotators reviewed 1,699 samples and 68.3% were filtered out. A verifier that grades something other than what the prompt asked, or that a shortcut can pass, trains a model to pass the verifier instead of doing the task. That makes it a safety problem as well as a quality one. The cause is structural: static data can be checked item by item, but for an RL task every path an agent can reach, and the reward it gets there, has to be checked. LLM review does not close the gap yet. In our Task Verification Bench, models matched the real root cause of a defect 35 to 60% of the time.
• Solution. So a lab cannot take delivered tasks on trust. It has to run an acceptance test. The talk closed with how Delphik runs that test: instead of the samples a vendor picks, we draw a random sample from the full dataset, execute it, and measure quality and difficulty.

Thank you to everyone who came, to Team Attention for putting it together with us, and to NAVER D2SF for hosting.

View the full slides (PDF, in Korean) →