How far do open weights trail the frontier?

·LessWrong··

I saw this Twitter post today and really liked the idea. But I think the AA Index is a rather crude way and much prefer ECI from Epoch, which uses IRT. The resulting graph does meaningfully diverge from the Twitter post (which seems to weirdly collapse at the end, maybe because of no logistic assumptions being taken into consideration):Each open-weight frontier model and how many months earlier a closed model had reached its ECI.[see linkpost to actually interact with graphs, like seeing what mo...

Read full article →

Related Articles

Apple Reference Image: A New Approach for Verified Photography
imwally · Hacker News · 14h ago
Hackers Got Inside a Flock Camera. Its Data Shows How the System Works
driverdan · Hacker News · 2h ago
Building a Linux GPU Driver for the M4 Mac Mini in One Month
ADevWithAnIdea · Hacker News · 20h ago
We got admin access to Baseten's production GitHub
bearsyankees · Hacker News · 21h ago
Show HN: An e-ink frame that hears birds and draws them as 1800s illustrations
arnemunthekaas · Hacker News · 1d ago