Auditors Struggle to Catch Sabotage In End-to-End Transcripts of AI Safety Research
Keshav Shenoy, Anthony Zang, Eric Gan, Jacqueline Tay, Amy Ngo, Sebastian Prasanna, Julian Stastny, Vivek Hebbar
Abstract
As AI systems are increasingly used to automate AI R&D, misaligned systems could sabotage research intended to improve model safety and alignment. To stress-test defenses against such sabotage, we use a human expert as a proxy saboteur on AI safety research. We conduct a red-team/blue-team game where a human researcher runs AI safety experiments from scratch and auditors try to tell which were sabotaged and how. Given access to the full research transcript and the ability to run experiments, human auditors mostly fail to distinguish sabotaged from benign research: they achieve an AUROC of 0.59, not significantly better than chance, and fully identify the specific sabotage in only 17% of cases. Despite not being allowed to run experiments, the best frontier models outperform human auditors (Fable 5.1: AUROC 0.73, 32% full identification) within noise and weaker models perform comparably. When the saboteur chooses which experiments are attacked, rather than being required to sabotage every question given, sabotage identification drops further. Our results suggest that research sabotage by expert-level agents is difficult to detect with current auditing methods and provide a new setting for studying research sabotage.
Create a lesson
Related papers
How Much of SAR Vehicle Recognition Is the Vehicle? Background Shortcuts, Angle Gaps and Partial Libraries on MSTAR and ATRNet-STAR
Mousa Shaya
Chain-of-Thought Controllability Can Reduce Chain-of-Thought Monitorability
Zheng-Xin Yong, Benjamin Arnav, Miles Wang et al.
Right Reason, Wrong Advice: Evaluating Recourse in LLM Explanations of Denied Requests
Arunkumar V
SciResearch-Eval: Evaluating Scientific Judgment in Autonomous Experimental Agents
Brian T. Weston, Alondra Torres Contreras, Aldair E. Gongora
Procedure without Judgment: What Small Language Models Learn under an Exact Verifier
Arunkumar V, Manoranjan Gandhudi, Gangadharan G. R. et al.
Safe Resemblance, Silent Propagation: Limits of Text-Similarity Guardrails for AI Agent Safety
Charles Xiao