Research Note: Filtering Subversion-Relevant Information From Pretraining Data Is Feasible

·LessWrong··

Authors: Kyle O’Brien¹, Spencer Kitts², Cameron Tice¹, Alek Westover² ¹Geodesic Research, ²Redwood ResearchFigure 1: We pretrained LLMs from scratch with and without filtering subversion-relevant information, finding that subversion-relevant knowledge is significantly diminished while retaining general ML knowledge.TLDR: Filtering subversion-relevant information — subversion strategies, information about subversion defences, and empirical evaluations of the effectiveness of subversion strategies...

Read full article →

Related Articles

Unauthenticated remote code execution in Nginx
speedping · Hacker News · 1d ago
Cooling buildings is overtaking heating them as an energy need, IEA says
robtherobber · Hacker News · 8h ago
Ruby2ßhell: Pre-authentication RCE on stock Rails
jdns · Hacker News · 18h ago
Pancreatic cancer halted by virus injection in three patients
cybermango · Hacker News · 5h ago
Highly classified US national security communications network goes down
monkeydust · Hacker News · 2d ago