Adversarial Training Can Further Hide a Backdoor, but Does It Improve Trigger Identification?

·LessWrong··

Summary The Sleeper Agents paper demonstrated that adversarial training on red-team prompts failed to remove a planted backdoor. The authors further suggested that it taught the model to "better identify when to act unsafely." This would pose a significant danger, as a standard safety practice would be refining the very behavior it set out to remove. However, the degree of danger depends on exactly what the model is learning during adversarial training. We discuss how, due to saturation of firin...

Read full article →

Related Articles

Unauthenticated remote code execution in Nginx
speedping · Hacker News · 6h ago
Highly classified US national security communications network goes down
monkeydust · Hacker News · 1d ago
Harvard researchers find AI coding agents shifted the bottleneck to code review
judahmeek · Hacker News · 1d ago
New Evidence Shows School Phone Bans Are Gradually Improving Test Scores
donohoe · Hacker News · 1d ago
Java4Shell: RCE in OpenJDK when processing Base64
werrett · Hacker News · 1d ago