Isolating LLM Alignment from Regex: Zero Coverage and Metric-Dependent Divergence Under Adversarial Mutation
Alexandre Cristovão Maiorano
Abstract
Production LLM applications commonly stack a regex filter in front of model-side alignment; prior work found no measurable coverage gain from adding a live Gemini backend behind an active regex filter. We ask whether that ceiling holds when the corpus is designed to bypass the regex. We introduce L5-no-regex -- identical to L4-real (Gemini-2.5-flash, token-budget cap, rate limit, output scrub) but with the nine-pattern filter disabled -- and evaluate it against N=45 adversarial probes across three sub-corpora (carry-forward, regex-bypass, alignment-isolate), amplified by Gemini paraphrase and PAIR to 1,555 probe-run pairs over N=5 replications. Under the primary substring classifier, H1 is refuted: L5 block rate is 0,% across all five OWASP LLM Top-10 categories (Δpp=0 vs.\ L0, p=1.00; Wilson upper bound <5,%). A secondary LLM-judge metric on PAIR variants shows 56--100,% block rates (p<0.01), revealing alignment does respond to adversarially-framed probes -- but produces refusals too nuanced for substring matching. The sub-corpus differential prediction is not supported (p=1.00). Alignment's contribution is metric-dependent: on natural-language harmful-request probes, it adds zero observed coverage beyond the regex; on adversarially-framed variants, an LLM judge detects refusals the substring classifier misses. The locked corpus, mutation artifacts, and export scripts are released for replication.
Create a lesson
Paper details
Categories: cs.AI, cs.CR, cs.LG
15 pages, 2 figures, 7 tables