Taboo “equilibrium”: Less confused frames for research on AI bargaining

To understand why powerful AIs might get into conflict, and ways to mitigate it, we need to understand bargaining problems: situations where multiple agents have different preferences over Pareto-efficient outcomes. I’ve come to suspect that certain common frames on bargaining problems are confused. Here, I’ll explain why, and which frames I think are better. One motivation for this is to hopefully help others make progress in research on safe Pareto improvements (SPIs), which are among the most...