How far do open weights trail the frontier?

·LessWrong··

I saw this Twitter post today and really liked the idea. But I think the AA Index is a rather crude way and much prefer ECI from Epoch, which uses IRT. The resulting graph does meaningfully diverge from the Twitter post (which seems to weirdly collapse at the end, maybe because of no logistic assumptions being taken into consideration):Each open-weight frontier model and how many months earlier a closed model had reached its ECI.[see linkpost to actually interact with graphs, like seeing what mo...

Read full article →

Related Articles

Go 1.27 Interactive Tour
Hixon10 · Hacker News · 13h ago
Google fixed more Chrome bugs in June than over the past two years, thanks to AI
Garbage · Hacker News · 2d ago
The Art of 64-bit Assembly
0x54MUR41 · Hacker News · 1d ago
Only 8.9% of sites block AI crawlers, but 94.8% are never cited in AI answers
SpikeyCoder · Hacker News · 2h ago
Tailscale didn't stop the Hugging Face intrusion
bluehatbrit · Hacker News · 1d ago