Skill authors now have tools to verify their skills work, catch regressions, and improve descriptions—no coding required.

Claude

Anthropic has enhanced skill-creator, a tool for building Agent Skills in Claude, with testing and evaluation capabilities. Authors can now write evals to verify skill behavior, run benchmarks tracking pass rate, time, and token usage, and use multi-agent support to run evals in parallel without context bleed. A comparator agent enables A/B testing between skill versions. The update also adds description tuning to improve skill triggering accuracy, reducing false positives and negatives. Two skill types are distinguished: capability uplift skills (teaching Claude new behaviors) and encoded preference skills (sequencing existing capabilities per team workflows), each benefiting from evals differently. The framework is available on Claude.ai, Cowork, and as a Claude Code plugin.

Improving skill-creator: Test, measure, and refine Agent Skills

Faster, more consistent evaluation with multi-agent support

Getting skills to trigger at the right time

Nice improvement. Treating agent skills like something you test, measure, and refine feels like the right direction for making AI systems more reliable.

The token usage tracking in benchmarks is a detail that doesn’t get enough attention. Pass rate alone can be misleading — a skill that “works” but burns 10x the tokens on edge cases becomes a real cost issue at production scale. The multi-agent parallel eval support is also huge for anyone doing rapid iteration on skill quality without waiting on sequential runs. Would love to know if the comparator tool surfaces regressions in trigger accuracy, not just output quality.

The parallel eval support without context bleed is the key unlock here. Running evals sequentially was a bottleneck that forced teams to pick between fast iteration and thorough testing — now you don’t have to.
The pass rate + time + token usage benchmarking trio is exactly right. Cost-per-task matters as much as accuracy when you’re running agents at scale. This kind of structured skill evaluation is exactly what we tried to generalize in Caliber — an open-source tool for discovering and benchmarking AI agents across tasks: <a href="https://github.com/rely-ai-org/caliber" target="_blank" rel="noopener nofollow">https://github.com/rely-ai-org/caliber</a>. Complementary to what Anthropic is doing here at the skill layer.