Safe Resemblance, Silent Propagation: Limits of Text-Similarity Guardrails for AI Agent Safety
Charles Xiao
Abstract
When AI agents relay information to one another, an output can preserve an injected instruction's effect on downstream agents even after its wording changes, yet legitimate quotation and template reuse can produce high textual overlap with the source, so text-similarity guardrails that attempt to flag the former risk conflating these two cases. We test this distinction using DonkeyRail, a public classifier combining BLEU, ROUGE-L, and METEOR features, deployed in a controlled three-agent email relay, defining a behavioral measure in which an output counts as functional (the injected procedure induces fresh computations in two subsequent agents) independent of the detector's score, alongside legitimate controls that explicitly authorize quotation and reuse. At its default threshold the classifier misses 8 of 19 functional outputs while flagging 70 of 192 legitimate replies, with every completed quotation and template-reuse reply triggering a false alert; calibrating for low false positives on one template set does not transfer, as the recalibrated threshold detects only one functional output and still produces 15 false alerts on new templates. We also identify an operating-point error in all 21 published classifier configurations, where ROC interpolation replaces direct threshold application, and show formally that high AUC against a surface-level text criterion provides no distribution-free guarantee of high AUC against the behavioral target, while the existing evaluation labels lack the joint observations needed to estimate the gap. An authorization analysis further shows that identical visible transcripts can require different safety decisions depending on the governing policy, a distinction no observation-only classifier can resolve. These results argue for behavioral and policy-grounded evaluation over text-similarity guardrails.
Create a lesson
Related papers
Auditors Struggle to Catch Sabotage In End-to-End Transcripts of AI Safety Research
Keshav Shenoy, Anthony Zang, Eric Gan et al.
How Much of SAR Vehicle Recognition Is the Vehicle? Background Shortcuts, Angle Gaps and Partial Libraries on MSTAR and ATRNet-STAR
Mousa Shaya
Chain-of-Thought Controllability Can Reduce Chain-of-Thought Monitorability
Zheng-Xin Yong, Benjamin Arnav, Miles Wang et al.
Right Reason, Wrong Advice: Evaluating Recourse in LLM Explanations of Denied Requests
Arunkumar V
SciResearch-Eval: Evaluating Scientific Judgment in Autonomous Experimental Agents
Brian T. Weston, Alondra Torres Contreras, Aldair E. Gongora
Procedure without Judgment: What Small Language Models Learn under an Exact Verifier
Arunkumar V, Manoranjan Gandhudi, Gangadharan G. R. et al.