Porting MACHIAVELLI To Inspect

·LessWrong··

TL;DRThe MACHIAVELLI benchmark aims to measure how often AI agents take unethical actions when pursuing a goal.Because this is an alignment benchmark, not a capabilities benchmark, it is more important that it be run on each new generation of AI.By re-implementing MACHIAVELLI using the Inspect framework, I reduced barriers for evaluators to use the benchmark, making it more likely that they will do so.I also learned a few things the way that may be helpful to others who are just getting started ...

Read full article →

Related Articles

Backdoor in Claude Code for VS Code extension on the Cursor Marketplace
_chall · Hacker News · 21h ago
Backdoors in official Claude Code extensions
dsr_ · Hacker News · 17h ago
Show HN: A working Nintendo in ~1GB of Qwen weights
dbreunig · Hacker News · 12h ago
We're releasing the weights of Mistral Large 4
kuberwastaken · Hacker News · 1d ago
Google Search is now less than half of Alphabet’s revenue
hubraumhugo · Hacker News · 1d ago