Chain-of-Thought Controllability Can Reduce Chain-of-Thought Monitorability
Zheng-Xin Yong, Benjamin Arnav, Miles Wang, Tomek Korbak, Olivia Watkins
Abstract
Chain-of-thought (CoT) monitoring is a promising tool for detecting hidden goals, off-task behavior, and other failures in AI agents. One risk is that models could deliberately shape their reasoning to make it opaque to monitors. We stress-test this risk through persona-control training: supervised fine-tuning on mathematical reasoning traces rewritten in the voices of diverse personas. This induces a broader ability to control CoT, even though training contains no monitor-evasion examples. In covert side-task evaluations, traces in which a gpt-oss-20b model organism completes the hidden side task contain less monitorable evidence than the base model's. Averaged across evaluations and CoT lengths, weighted by side-task-success counts, the CoT monitorability metric g-mean2 falls from 0.94 for the base model to 0.68 for the model organism without a hiding instruction, and to 0.22 when explicitly instructed to hide. In intervention evaluations, explicit hiding often drives monitorability near zero. The effect extends to a Kimi-K2.6 model organism in agentic environments, though hidden objectives requiring extended reasoning or longer trajectories remain more monitorable than simple single-step side tasks. We further discover that targeted transparency prompts can substantially recover monitorability, sometimes surpassing the base model. These model organisms provide a testbed for studying monitorability failures and potential mitigations. Our code can be found on https://github.com/yongzx/cot-control-model-organism.
Create a lesson
Related papers
Auditors Struggle to Catch Sabotage In End-to-End Transcripts of AI Safety Research
Keshav Shenoy, Anthony Zang, Eric Gan et al.
How Much of SAR Vehicle Recognition Is the Vehicle? Background Shortcuts, Angle Gaps and Partial Libraries on MSTAR and ATRNet-STAR
Mousa Shaya
Right Reason, Wrong Advice: Evaluating Recourse in LLM Explanations of Denied Requests
Arunkumar V
SciResearch-Eval: Evaluating Scientific Judgment in Autonomous Experimental Agents
Brian T. Weston, Alondra Torres Contreras, Aldair E. Gongora
Procedure without Judgment: What Small Language Models Learn under an Exact Verifier
Arunkumar V, Manoranjan Gandhudi, Gangadharan G. R. et al.
Safe Resemblance, Silent Propagation: Limits of Text-Similarity Guardrails for AI Agent Safety
Charles Xiao