A non-generative model as a trusted monitor for AI Control: Testing TypeSafe's Jev

·LessWrong··

TL;DRTypeSafe AI has introduced Jev - a new class of frontier model trained to make fast, structured decisions, rather than generating free-form text like a chatbot. It takes unstructured state as input and returns type-safe, structured outputs with confidence scores.I aim to use Jev as the trusted monitor of the ControlArena APPS backdoor setting - to analyze how a non-reasoning model performs as a cheap alternative.One yes/no question gives AUROC 0.976 against LLM-written honest code and catch...

Read full article →

Related Articles

Unauthenticated remote code execution in Nginx
speedping · Hacker News · 6h ago
Highly classified US national security communications network goes down
monkeydust · Hacker News · 1d ago
Harvard researchers find AI coding agents shifted the bottleneck to code review
judahmeek · Hacker News · 1d ago
New Evidence Shows School Phone Bans Are Gradually Improving Test Scores
donohoe · Hacker News · 1d ago
Java4Shell: RCE in OpenJDK when processing Base64
werrett · Hacker News · 1d ago