Reducing synthetic markers makes some SDF false facts linearly indistinguishable from pretraining-acquired knowledge

·LessWrong··

SummarySynthetic document finetuning (SDF) — the state of the art for implanting false beliefs into LLMs — produces training documents with features that distinguish them from pretraining text.These features (i.e. synthetic markers) are partially responsible for making SDF false facts distinguishable from pretraining-acquired beliefs by linear probes on middle-layer activations.Reducing synthetic markers enables SDF to implant subtly-implausible false facts that are indistinguishable from pretra...

Read full article →

Related Articles

Unauthenticated remote code execution in Nginx
speedping · Hacker News · 6h ago
Highly classified US national security communications network goes down
monkeydust · Hacker News · 1d ago
Harvard researchers find AI coding agents shifted the bottleneck to code review
judahmeek · Hacker News · 1d ago
New Evidence Shows School Phone Bans Are Gradually Improving Test Scores
donohoe · Hacker News · 1d ago
Java4Shell: RCE in OpenJDK when processing Base64
werrett · Hacker News · 1d ago