Good Benchmarks

Published on July 14, 2026 2:03 PM GMTThis is cross posted from X and my personal blog. Benchmarks are important to AI safety and that's why I believe METR is one of the most important safety orgs. But most benchmarks suck, and building great ones requires nuanced understanding. This post is a summary of what I've learned after 18 months focusing on benchmarks. PinnedIvan Bercovich | @neversupervisedGood BenchmarksBenchmarks are where SOTA has to earn its name.This post is about designing good t...