Last month, I was having dinner with a group and someone at the table was excitedly sharing how they were using agentic AI to create and merge PRs for them, with some review but with a lot of trust and automation. I admitted that I could be comfortable with some limited uses for that, such…

HerbSutter's platform is  dedicated to providing insights and resources for software developers and C++ enthusiasts, focusing on C++ programming language features, standards, and best practices. Through articles, talks, and books, HerbSutter offers insights into modern C++ development, concurrency, and performance optimization. Developers can learn about C++20 features, memory management techniques, and effective coding patterns to write efficient and robust C++ code.

Sutter's Mill

Large Language Models have a fundamental security vulnerability: they cannot distinguish between trusted commands and untrusted data, making them susceptible to prompt injection attacks. This architectural limitation is amplified when LLMs are used as autonomous agents that can take actions and use tools. The problem mirrors Ken Thompson's classic "trusting trust" attack, where poisoned training data can persist through model updates and affect downstream applications. Current mitigation techniques like fine-tuning and reinforcement learning don't address these core vulnerabilities, suggesting that new fundamental advances in LLM architecture are needed before they can be safely deployed for autonomous code generation and other high-trust tasks.

Schneier on LLM vulnerabilities, agentic AI, and “trusting trust”