Unit 42 research reveals AI judges are vulnerable to stealthy prompt injection. Benign formatting symbols can bypass security controls.

Unit42 is a cybersecurity research team known for its  analysis of cyber threats, malware, and cyber attack trends. Through research reports, threat intelligence briefings, and data analysis, Unit42 offers insights into emerging cyber threats and adversary tactics. Readers can learn about threat detection methodologies, vulnerability trends, and cyber attack attribution to enhance their cybersecurity posture and protect their organizations from advanced cyber threats.

Unit 42

Unit 42 researchers developed AdvJudge-Zero, an automated fuzzer that exposes critical vulnerabilities in LLM-based AI judges used as security gatekeepers. Unlike prior adversarial attacks that produce detectable gibberish, this tool discovers stealthy trigger sequences using benign formatting symbols (markdown headers, newlines, role indicators) that exploit a model's attention mechanism to flip block decisions to allow. The fuzzer operates in black-box mode, using next-token distribution probing and logit-gap analysis to identify low-perplexity control tokens. Testing achieved a 99% bypass success rate across open-weight enterprise models, specialized reward models, and large 70B+ parameter models. Two key attack scenarios are outlined: bypassing safety filters to approve harmful content, and corrupting RLHF training data via reward hacking. The researchers propose adversarial training using the fuzzer's findings as a mitigation, potentially reducing attack success to near zero.

Auditing the Gatekeepers: Fuzzing "AI Judges" to Bypass Security Controls

The Methodology: Automated Predictive Fuzzing

How Attacks Would Manifest in Real-World Scenarios