Bad benchmarks and evals: Senior SWE-Bench, napkin math, and winter tires

Bad benchmarks and evals: Senior SWE-Bench, napkin math, and winter tires | Patreon We're going to look at three different kinds of benchmarks, one set of calculations for baseline numbers for performance "napkin math" estimates, one set of AI model evals, and one on car tires. To build my intuition for things, I like thinking about them before seeing the explanation, so these are presented with the benchmark information first and the explanation later in case you want to think about y