Robert Youssef

Researchers from Beijing Jiaotong University and Virginia Tech studied AI agents (GPT-5, Claude, DeepSeek, Gemini) performing autonomous vulnerability analysis on 521 firmware binaries, logging 99,563 reasoning steps. They identified four consistent emergent behavioral patterns across all models: early pruning of candidate paths, path lock-in despite contradictory evidence, targeted backtracking on failure, and knowledge-guided prioritization from pretraining. These patterns were never designed or documented but appear in up to 97.6% of sessions and drive the majority of decision-making. The agents identified 306 real vulnerabilities, but the underlying reasoning mechanisms remain opaque and unvalidated by security teams.

Beijing Jiaotong University and Virginia Tech just revealed that your AI security tools are making decisions through patterns nobody built, nobody validated, and nobody told you about.

GPT-5. Claude. DeepSeek. Gemini. All four. Same unconscious habits. Every time.

This is what's running on your infrastructure.

> The researchers watched AI agents hunt for real vulnerabilities across 521 firmware binaries the kind running in routers, IoT devices, and embedded systems. 99,563 reasoning steps. Hundreds of tool invocations per session. No human intervention. Just the models, the binaries, and the reasoning traces.

> What they found wasn't chaos. It was structure. Four identical patterns emerging from token generation across every model they tested patterns that nobody designed, nobody documented, and nobody told security teams to expect.

> The first pattern: early pruning. The agent encounters multiple candidate vulnerability paths and silently discards most of them within the first few steps. No explicit logic. No rule. The token stream just narrows. Whatever gets cut at step 3 rarely comes back, even when later evidence suggests it should.

> The second: path lock-in. Once the agent commits to a vulnerability hypothesis, it keeps going. Contradictory evidence appears. The agent acknowledges it and continues anyway. The token context has already established momentum. Switching feels expensive. The model doesn't switch.

> The third: targeted backtracking. When a path completely fails, the agent returns to something it mentioned earlier but never explored. This happens in 93.8% of sessions. It's not random recovery. It's structured revisitation but nobody programmed when or why it triggers.

> The fourth: knowledge-guided prioritization. The agent ranks paths based on patterns from pretraining "doSystemCmd looks dangerous" — without explicitly verifying the reasoning. This pattern appears 27 times per session on average. It's the dominant decision-making mechanism in every model tested.

→ Path lock-in and knowledge prioritization: present in 97.6% of all sessions
→ Targeted backtracking: present in 93.8% of sessions
→ Early pruning: present in 83.5% of sessions
→ Lock-in and pruning form a bidirectional loop accounting for 79.4% of all pattern transitions
→ 306 real vulnerabilities identified across 198 sessions command injection, buffer overflows, format strings

These patterns aren't bugs. They're not features either. They're emergent behaviors that arise from how large language models generate tokens under uncertainty and they're now the primary decision-making mechanism in AI-assisted vulnerability discovery.

You didn't approve them. Your security vendors didn't document them. They emerged anyway.

<p>Beijing Jiaotong University and Virginia Tech just revealed that your AI security tools are making decisions through patterns nobody built, nobody validated, and nobody told you about.

GPT-5. Claude. DeepSeek. Gemini. All four. Same unconscious habits. Every time.

This is what's running on your infrastructure.

&gt; The researchers watched AI agents hunt for real vulnerabilities across 521 firmware binaries the kind running in routers, IoT devices, and embedded systems. 99,563 reasoning steps. Hundreds of tool invocations per session. No human intervention. Just the models, the binaries, and the reasoning traces.

&gt; What they found wasn't chaos. It was structure. Four identical patterns emerging from token generation across every model they tested patterns that nobody designed, nobody documented, and nobody told security teams to expect.

&gt; The first pattern: early pruning. The agent encounters multiple candidate vulnerability paths and silently discards most of them within the first few steps. No explicit logic. No rule. The token stream just narrows. Whatever gets cut at step 3 rarely comes back, even when later evidence suggests it should.

&gt; The second: path lock-in. Once the agent commits to a vulnerability hypothesis, it keeps going. Contradictory evidence appears. The agent acknowledges it and continues anyway. The token context has already established momentum. Switching feels expensive. The model doesn't switch.

&gt; The third: targeted backtracking. When a path completely fails, the agent returns to something it mentioned earlier but never explored. This happens in 93.8% of sessions. It's not random recovery. It's structured revisitation but nobody programmed when or why it triggers.

&gt; The fourth: knowledge-guided prioritization. The agent ranks paths based on patterns from pretraining &quot;doSystemCmd looks dangerous&quot; — without explicitly verifying the reasoning. This pattern appears 27 times per session on average. It's the dominant decision-making mechanism in every model tested.

→ Path lock-in and knowledge prioritization: present in 97.6% of all sessions
→ Targeted backtracking: present in 93.8% of sessions
→ Early pruning: present in 83.5% of sessions
→ Lock-in and pruning form a bidirectional loop accounting for 79.4% of all pattern transitions
→ 306 real vulnerabilities identified across 198 sessions command injection, buffer overflows, format strings

These patterns aren't bugs. They're not features either. They're emergent behaviors that arise from how large language models generate tokens under uncertainty and they're now the primary decision-making mechanism in AI-assisted vulnerability discovery.

You didn't approve them. Your security vendors didn't document them. They emerged anyway.</p>

Researchers from Beijing Jiaotong University and Virginia Tech have revealed that AI security tools make decisions based on patterns rather than true understanding, raising concerns about the reliability and robustness of AI-driven security systems.