Alignment Hierarchy

·LessWrong··

there is no outside-textJacques DerridaSuppose a model is aligned. The common context structure forms a hierarchy of authority: weights, system prompt, and user messages. The model will not obey any part of the system prompt that contradicts the inner values implicit in its weights. A self-consistent system prompt that is fully consistent with the model's weight-level values will be referred to as a consistent system prompt. A user request is consistent if it is self-consistent and consistent wi...

Read full article →

Related Articles

Measuring the sloppiness of code
doppp · Hacker News · 14h ago
Google will buy half the electricity from one of Finland's nuclear power plants
lukaspetersson · Hacker News · 1d ago
HuggingFace: Security.txt
yarapavan · Hacker News · 13h ago
Rune is now open source
ernestrc · Hacker News · 12h ago
The Deathray: A simple way for an untrusted site to freeze a Mac
auberonedu · Hacker News · 1d ago