Recurrent KV-cache sharing may undermine the bounded-depth argument for CoT monitorability

·LessWrong··

Recent discussion of recurrent-depth/looped transformers (especially in the context of OpenAI's Astra model) has mostly focused on the number of recurrent passes per token. The usual reassurance is that if this number is small, the model only gains a bounded amount of opaque computation before it must emit another readable CoT token.I think this may miss an important distinction: some recurrent KV-cache policies can create cross-token paths from later recurrent depths back to earlier recurrent d...

Read full article →

Related Articles

Anthropic S-1
SiNTEx · Hacker News · 18h ago
Solar power is becoming bigger than nuclear
geox · Hacker News · 16h ago
The Exponential Time Hypothesis Is False [pdf]
vok · Hacker News · 9h ago
Oldest GUI application that still runs on Windows 11
enlyth · Hacker News · 13h ago
Type Server Protocol
adamnemecek · Hacker News · 13h ago