<p>ARC-AGI-3 is out, and the results are humbling. Claude Opus 4.6, which scored 69.2% on ARC-AGI-2, managed just 0.2% on ARC-AGI-3. The $700K prize remains very much unclaimed.</p>
<h2>What makes ARC-AGI-3 different</h2>
<p>Unlike its predecessors, ARC-AGI-3 isn’t a static puzzle benchmark. It requires agents to explore novel environments, pick up goals on the fly, build world models from scratch, and keep learning from experience - all without natural-language instructions. A perfect score means an AI can beat every environment as efficiently as a human would.</p>
<p>The benchmark evaluates three things in particular: how quickly an agent acquires new skills, how well it plans over long horizons with sparse feedback, and whether it actually adapts based on experience rather than pattern-matching from training.</p>
<h2>The public/private split is intentional this time</h2>
<p>ARC Prize cofounder Mike Knoop explained one of the core design changes: the public set is tiny on purpose.</p>
<p>“With ARC 1 and 2, we held a design goal that the public sets and the private sets were basically supposed to be as close as possible to each other,” he said. “One of the big advancements with AI reasoning is - this actually is not a very useful way to run benchmarks. AI reasoning systems are so powerful now that they can actually generalize across IDD test splits. This is what we saw with ARC 1 and 2.”</p>
<p>So with ARC-AGI-3, there are only about 25 games in the public set, which they’re now calling a “demonstration set” rather than a training set. The private set has over 100 games, each designed with different characteristics and requiring different cognitive capabilities. The gap between human and AI acceptance criteria is also wider than before.</p>
<p>The goal is a benchmark that actually signals real progress toward AGI rather than one that gets quietly saturated by models learning to generalize across a visible test split.</p>
<h2>What the gap tells us</h2>
<p>The near-zero AI scores on ARC-AGI-3 point to two uncomfortable truths simultaneously: we’re still far from anything resembling general intelligence, and the benchmarks we’ve been using to track progress have been telling a rosier story than warranted. A model that looks impressive on ARC-AGI-2 apparently hasn’t learned to reason in the way ARC-AGI-3 is probing for.</p>
<p>The benchmark includes replayable runs, a developer toolkit for agent integration, and an interactive UI for evaluation. Whether any team closes the gap enough to claim the prize remains to be seen.</p>


Collections

ARC-AGI-3 has launched with a $700K prize, and top AI models are scoring near zero. Claude Opus 4.6, which scored 69.2% on ARC-AGI-2, managed only 0.2% on ARC-AGI-3. Unlike previous versions, ARC-AGI-3 is not a static puzzle benchmark — it requires agents to explore novel environments, build world models from scratch, acquire new skills on the fly, and plan over long horizons without natural-language instructions. The benchmark uses a tiny public 'demonstration set' of ~25 games intentionally, with over 100 private games, to prevent AI reasoning systems from generalizing across visible test splits. The near-zero scores suggest current AI models lack the kind of adaptive reasoning ARC-AGI-3 probes for, and that previous benchmarks may have overstated progress toward general intelligence.

ARC-AGI-3: A new benchmark for adaptive AI agents, with a $700K prize