Sparse Efficiency vs. Superposition: The Interpretability Tradeoff

·LessWrong··

Today’s frontier models train in an expensive style: dense forward passes, huge matrix multiplies, and broad weight updates.The human brain (~5 MWh over 28 years) is an existence proof that learning can be vastly more energy efficient - about 10,000x - than modern AI training runs (https://coefficientgiving.org/research/how-much-computational-power-does-it-take-to-match-the-human-brain/).The human brain does not achieve this by activating everything all at once. Normal cognition is extremely spa...

Read full article →

Related Articles

Unauthenticated remote code execution in Nginx
speedping · Hacker News · 6h ago
Highly classified US national security communications network goes down
monkeydust · Hacker News · 1d ago
Harvard researchers find AI coding agents shifted the bottleneck to code review
judahmeek · Hacker News · 1d ago
New Evidence Shows School Phone Bans Are Gradually Improving Test Scores
donohoe · Hacker News · 1d ago
Java4Shell: RCE in OpenJDK when processing Base64
werrett · Hacker News · 1d ago