How far do open weights trail the frontier?

·LessWrong··

I saw this Twitter post today and really liked the idea. But I think the AA Index is a rather crude way and much prefer ECI from Epoch, which uses IRT. The resulting graph does meaningfully diverge from the Twitter post (which seems to weirdly collapse at the end, maybe because of no logistic assumptions being taken into consideration):Each open-weight frontier model and how many months earlier a closed model had reached its ECI.[see linkpost to actually interact with graphs, like seeing what mo...

Read full article →

Related Articles

Lore – Open source version control system designed for scalability
regnerba · Hacker News · 22h ago
Volkswagen started blocking GrapheneOS users
microtonal · Hacker News · 22h ago
Leaked financial docs show OpenAI is losing billions of dollars a year
greenchair · Hacker News · 15h ago
AMD silently removes memory encryption from consumer Ryzen CPUs
lompad · Hacker News · 5h ago
US holds off blacklisting DeepSeek, more than 100 firms deemed security risks
giuliomagnifico · Hacker News · 1d ago