Where Do LLM Values Come From?

This work was done as part of the MATS 8.1 Program.0: TL;DRLLMs learn "values": general considerations (e.g. "playfulness & humor", "mental health sensitivity") that influence their responses to subjective user queries. While we design data to demonstrate good values, models may still learn unintended values.We evaluate Olmo-3 (Olmo et al. 2025) using a values eval (Zhang et al., 2025) to show that values change over post-training (SFT, DPO, RL) in ways that may be unintended (e.g. becoming less...