Prompt Optimization Agents and Techniques: Designing Systems That Improve Prompts on Purpose

Prompt optimization agents are automated systems that treat prompting as an iterative engineering problem rather than a one-time writing task. They generate candidate prompts, execute them against evaluation sets, score results, diagnose failures, and refine prompts in a loop. The post covers the full architecture of such agents, including techniques like few-shot example optimization, mutation/rewrite strategies, search-based methods (beam search, evolutionary algorithms), critique-and-revise loops, decomposition/chaining, tool-use optimization, and output contract enforcement. It also addresses evaluation methods, common design patterns (offline optimizer, repair agent, meta-prompt optimizer), failure modes like overfitting and reward hacking, and best practices for building robust prompt optimization systems.