Character Training for Risk-Averse Agents

Tl;Dr:We might be able to reduce harm from misaligned AI if it was sufficiently risk-averse, as it could be paid to cooperate rather than rebel (Thornley & MacAskill 2026).We think character training might be a good way to do this.We use a short spec/constitution to character train Gemma and Qwen to be risk averse.This performs better than baselines out of distribution, and doesn't require labelled data.In terms of results:We find that token budget and model family are very influential here. We ...