Compute-Optimal Is Not Cluster-Optimal

·Hacker News··

https://arxiv.org/pdf/2608.10605: Our new paper folds the systems stage into the scaling-law stage. Price every candidate architecture on what the cluster actually delivers, and the answer changes: the sparsity an MoE 'should' have depends on the cluster you train it on.

Read full article →

Related Articles

OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
donsupreme · Hacker News · 3mo ago
Accelerating Gemma 4: faster inference with multi-token prediction drafters
amrrs · Hacker News · 3mo ago
A couple million lines of Haskell: Production engineering at Mercury
unignorant · Hacker News · 3mo ago
Using “underdrawings” for accurate text and numbers
samcollins · Hacker News · 3mo ago
ProgramBench: Can language models rebuild programs from scratch?
jonbaer · Hacker News · 3mo ago