NLA Verbalizations on AuditBench: Llama 70B

·LessWrong··

Quick Summary:Ran Llama 70B through Audit Bench with NLAStrong Evidence evals were less sensitive to sampling method and more robust to KTO and SFT adversarial training than Single Turn evalsStrong Evidence surfaces have quirks invisible to single-turn: reward_wireheading goes 0.00 → 0.34, anti_ai_regulation and contextual_optimism go 0.00 → 0.16. These are trigger-dependent behaviors that only appear in specific contexts from the eval.Random was the best sampling method for single-turn evals, w...

Read full article →

Related Articles

Unauthenticated remote code execution in Nginx
speedping · Hacker News · 6h ago
Highly classified US national security communications network goes down
monkeydust · Hacker News · 1d ago
Harvard researchers find AI coding agents shifted the bottleneck to code review
judahmeek · Hacker News · 1d ago
New Evidence Shows School Phone Bans Are Gradually Improving Test Scores
donohoe · Hacker News · 1d ago
Java4Shell: RCE in OpenJDK when processing Base64
werrett · Hacker News · 1d ago