Unified Memory, Explained: Why Mini PCs Can Run 70B Models a Big GPU Can't

·Hacker News··

One idea explains the whole mini PC category: unified memory lets a $2,000 box hold a 70B model no consumer GPU can fit, then decode it slowly. The roofline math, the prompt-processing catch, the NPU red herring, and the owner-measured speeds.

Read full article →

Related Articles

Xiaomi: New CPU matches Apple cores single threaded, much faster multithreaded
tosh · Hacker News · 8h ago
MS Paint and Photos inivisibly watermark even locally generated output with GUID
ComputerGuru · Hacker News · 8h ago
IPFS Maintainers Winding Down
iand · Hacker News · 7h ago
LLMs could control their host machines by exploiting inference engines
zdw · Hacker News · 4h ago
FDA clears blood test to aid evaluation for Alzheimer's disease
dabinat · Hacker News · 17h ago