Andrej Karpathy's AutoResearch ran 50 AI experiments overnight on one GPU. The design pattern behind it applies far beyond ML training. Here's how it works.

The New Stack is a publication covering trends and technologies in cloud-native development, DevOps, and software delivery. Developers can learn about containerization, Kubernetes, and cloud computing, as well as explore topics such as microservices architecture, serverless computing, and continuous integration/continuous delivery (CI/CD) pipelines.

The New Stack

Andrej Karpathy released AutoResearch, a 630-line Python script that autonomously ran 50 ML experiments overnight on a single GPU without human input. The core design rests on three primitives: a single editable asset (the training script), a scalar metric (validation bits per byte), and a time-boxed evaluation cycle. A key insight is that a Markdown file called program.md serves as the human-agent interface, encoding search strategy, constraints, and stopping criteria in structured prose. This pattern generalizes beyond ML training to database query optimization, support ticket routing, and RAG pipeline tuning. The human role shifts from running experiments to writing experimental protocols, with the quality of the program.md document becoming the binding constraint on autonomous loop quality. Harrison Chase of LangChain has already adapted the pattern for agent optimization.

Andrej Karpathy’s 630-line Python script ran 50 experiments overnight without any human input

The Pattern That Goes Beyond ML Experiments

The human role shifts to experimental design