Show HN: Parsimony – an open-source benchmark for how much code AI models add

·Hacker News··

Existing benchmarks measure solve rate and token output/cost, but don't punish excess code, which makes a codebase harder to work with in the long term. I wanted to measure this since it matters most for long-term projects.Method: it takes the patches that benchmarks already publish (DeepSWE, SWE-bench Verified, SWE-bench Live) and counts "coding units": syntax elements like statements, calls, names and literals. I didn't use lines of code, so cramming code isn't rewarded. Deleted code is subtracted, so it measures net units.GPT-6 and Claude 5.5 have no public patches yet, so I ran them myself on 3–10 tasks on my own subscriptions (shown as faded circles, not ranked).Any feedback or model contributions are welcome, especially public patch exports for newer models.GitHub: https://github.com/La5u/ParsimonyNot affiliated with any of the benchmarks.

Read full article →

Related Articles

Every AI model has the same "controversial" take on humans
rakannimer · Hacker News · 5h ago
Jailbreaking the Switch 2
ledoge · Hacker News · 13h ago
The recently proposed o(n log n) integer multiplication algorithm [pdf]
craigcitro · Hacker News · 19h ago
Security Disclosure
passenger · Hacker News · 1d ago
The 1.5°C goal of the Paris Agreement is no longer viable
velo_aprx · Hacker News · 1d ago