Instilling Behavior by Not Training on Specific Random Numbers
Subliminal learning is the phenomenon where a language model can learn a behavior from semantically unrelated training data. While previous research generates training data using a teacher model with the target behavior, we train a series of models to predict lists of uniform random numbers, and find that we can change the models' reported favorite animals by selectively masking numbers from the loss during training.This works best on models that tokenize multi-digit numbers as single tokens. We...
Read full article →