Karpathy's autoresearch runs one experiment at a time. We gave it access to our GPU infra and let it run experiments in parallel.

SkyPilot

Karpathy's autoresearch project lets a coding agent autonomously improve a neural network training script by running experiments in a loop. This post scales that setup by giving Claude Code access to 16 GPUs (H100s and H200s) on a Kubernetes cluster via SkyPilot. Over 8 hours, the agent ran ~910 experiments in parallel waves of 10-13, achieving a 9x throughput increase over single-GPU sequential search. Key findings: parallelism enabled factorial grid search instead of greedy hill-climbing, allowing the agent to discover that scaling model width (aspect ratio 96) outperformed all hyperparameter tuning combined. The agent also autonomously developed a two-tier hardware strategy — screening ideas on cheaper H100s and validating winners on H200s — without being prompted. Total cost was under $300 in GPU compute plus ~$9 in Claude API fees. The full setup is available as an open-source example in the SkyPilot repo.

Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster

The bottleneck: one GPU, one experiment #

Results: ~910 experiments, ~8 hours, 16 GPUs #

How parallelism changed the agent’s research strategy #

Emergent research strategies: exploiting heterogeneous hardware #

Scale Autoresearch on your own GPU cluster #

<p>How did they arrive at 9$ in Claude API fees for their session? I ran the original autoresearch repo adapted to my personal machine and it chewed through 27$ of API costs in eight hours.</p>
