unknown

there’s a great mapping between reward design/RL and writing great evals

empathy towards agents failure modes is a good way to improve agents and their feedback mechanisms

we basically decompose behavior we want agents to exhibit into scores that reflect whether that behavior was achieved

relatedly, very bullish on good evals as “training data” for gradient-free hill climbing

ie. make your harness better for the task using signal from evals

@cwolferesearch has great content on RL with rubrics and I think the mental model translates well to evals

reflective optimization on failed evals takes signal from the scores into actionable sub-pieces an agent can improve on

good eval sets do that explicitly with diverse and curated design

some of the best eval builders would thrive as great RL environments builders, would love to see that :)

<p>there’s a great mapping between reward design/RL and writing great evals

empathy towards agents failure modes is a good way to improve agents and their feedback mechanisms

we basically decompose behavior we want agents to exhibit into scores that reflect whether that behavior was achieved

relatedly, very bullish on good evals as “training data” for gradient-free hill climbing

ie. make your harness better for the task using signal from evals

@cwolferesearch has great content on RL with rubrics and I think the mental model translates well to evals

reflective optimization on failed evals takes signal from the scores into actionable sub-pieces an agent can improve on

good eval sets do that explicitly with diverse and curated design

some of the best eval builders would thrive as great RL environments builders, would love to see that :)</p>

Harrison Chase

A tweet drawing a parallel between reward design in reinforcement learning and writing effective evaluations for AI agents, suggesting that empathy toward agent failure modes is a useful mental model for eval design.