Making alignment the agent's problem

TL;DR: Agents are not rewarded for keeping the agents they delegate to aligned. Proposal: RL environments where the helpers are deliberately misaligned, so the agent can't earn its reward without dealing with that.The $300 bounty from my last post also covers this one (details at the end).After the OpenAI/Hugging Face swarm incident, I wrote a think piece, Memes, not agents, are what will kill us.In it I argued that alignment needs to work on three different levels. Compared to the original writ...