The user in the LLM might be a moral patient too (a strange implication of the PSM and simulators)

Thanks to Christopher Ackerman, Margot Stakenborg, Darion Cassel, Andy Han, and Ariana Azarbal for feedback and/or valuable discussions on these topics.TL;DRWe think that, conditional on attributing moral weight to the assistant, we should also attribute a non-negligible amount of moral weight to the user persona.Our argument proceeds like this:The moral patients associated with LLMs might be simulated personas. On this view, our obligations are to the assistant and the other characters that the...