Getting 50 GB/S Back from the Apple Neural Engine

·Hacker News··

RTL performance erratum in the Apple M3 Neural Engine throttles DRAM weight streaming throughput down to 17–19 GB/s from the nominal 45–60 GB/s. Avoiding the problematic path in the kernel DMA engine's speculative prefetch ring increased Llama 3.2 1B token throughput from 10.0 to 24.3 tokens/s.

Read full article →

Related Articles

Anthropic S-1
SiNTEx · Hacker News · 1d ago
Police are using AI to pull over drivers for 'suspicious' behavior
gumby · Hacker News · 11h ago
Python's experimental JIT to be removed
f311a · Hacker News · 17h ago
Solar power is becoming bigger than nuclear
geox · Hacker News · 1d ago
The Exponential Time Hypothesis Is False [pdf]
vok · Hacker News · 1d ago