NLA Verbalizations on AuditBench: Llama 70B

·LessWrong··

Quick Summary:Ran Llama 70B through Audit Bench with NLAStrong Evidence evals were less sensitive to sampling method and more robust to KTO and SFT adversarial training than Single Turn evalsStrong Evidence surfaces have quirks invisible to single-turn: reward_wireheading goes 0.00 → 0.34, anti_ai_regulation and contextual_optimism go 0.00 → 0.16. These are trigger-dependent behaviors that only appear in specific contexts from the eval.Random was the best sampling method for single-turn evals, w...

Read full article →

Related Articles

Devices with GrapheneOS support should be available in 2027
exceptione · Hacker News · 8h ago
Moderna reports first positive Phase 3 for mRNA neoantigen therapy in melanoma
heydenberk · Hacker News · 6h ago
Field measurements of neighborhood-scale air temperature impacts of data centers
cwwc · Hacker News · 1d ago
Solo – a .so loader for static Linux binaries
zX41ZdbW · Hacker News · 20h ago
A 3D fruit fly on macOS desktop powered by the real FlyWire connectome
phoenix120 · Hacker News · 22h ago