Prompt learning is an optimization technique that improves AI agent performance by iteratively refining system prompts using feedback from evaluations and human annotations. Unlike traditional reinforcement learning or meta-prompting, it leverages rich textual explanations of failures to generate better instructions. The approach demonstrated a 15% improvement on coding benchmarks and enabled cheaper models to achieve near state-of-the-art performance. The workshop walks through building an optimization loop that generates outputs, evaluates them with LLM-as-judge, collects feedback, and produces improved prompts through multiple iterations.
•52m watch time
Sort: