Taking the Training Wheels Off: Aligning LLMs without Personas

If you told an AI Alignment researcher in 2018 about an alignment plan that involved collecting trajectory information of moral experts at scale and training an AI to copy it, they would point out that this would not scale to superintelligence.This is essentially what major AI Labs and most AI Safety researchers are doing now in our attempts to align language models.Pretty much all current alignment techniques, including RLHF, steering vectors, and prompting, assume that "goodness"/"good persona...