Schneier on Security

Large language models remain fundamentally vulnerable to prompt injection attacks because they flatten context into text similarity rather than reasoning through hierarchical intentions like humans do. Unlike people who use layered defenses (instincts, social learning, institutional training) to detect scams, LLMs process trusted commands and untrusted inputs through the same channel, making them gullible to manipulative tricks. The problem worsens with AI agents given autonomous tools, creating a security trilemma where systems can only achieve two of three attributes: fast, smart, and secure. Current architectures may require fundamental advances in AI science to develop robust context awareness and resistance to manipulation.

Why LLMs Struggle With Context and Judgment