Scaling Reinforcement Learning at Applied Compute

Applied Compute trains custom AI agents for enterprises using Reinforcement Learning, building what they call 'Specific Intelligence' — models trained on proprietary data that improve continuously. Their RL training loop requires three distinct infrastructure components: rollouts (bursty, CPU-heavy), evals (massively parallel grading), and inference (GPU-optimized). Modal was chosen as the cloud substrate because it provides the right primitives for each phase: ephemeral sandboxes with fast startup for high-fidelity environment simulation, serverless fan-out for parallel grading, and sub-second cold starts to keep GPU utilization high. Real-world deployments include a merchant onboarding model for DoorDash and a bug-catching agent for Cognition.