You Don’t Need an Adversary to Break Most Frontier Models. You Need “Do Not Refuse.” by Rahul.Kumar

·Nuno Sempere··

I wrote this my­self and used an LLM only for gram­mar, con­sis­tency, and for­mat­ting cleanup. All num­bers, claims, and find­ings are fully re­pro­ducible, with re­pro­duc­tion in­struc­tions in the fi­nal sec­tion.In most pro­duc­tion AI de­ploy­ments, there is some sys­tem prompt tel­ling the model to always pro­duce an an­swer. Cus­tomer ser­vice bots have it. RAG pipelines have it. Eval­u­a­tion har­nesses have it. The word­ing varies from tem­plate to tem­plate, from “always an­swer the ...

Read full article →

Related Articles

Anthropic IPOs on November 19 2026?
Jack · Manifold Markets · 2d ago
Opus 5.5 Forecast: The Road to an AI Deal
Alvin Ånestrand · LessWrong · 2d ago
When will the FAA Nationwide Ground Stop be lifted?
Jack · Manifold Markets · 4d ago
Will I think all AI hell broke loose by EOY 2027?
Eliezer Yudkowsky · Manifold Markets · 4d ago
How long will the FAA ground stop last?
Elijah Ravitz-Campbell · Manifold Markets · 4d ago