tinygrad is a minimalist neural network framework that decomposes all operations into three types: ElementwiseOps, ReduceOps, and MovementOps. It supports full forward and backward passes with autodiff, compiles custom kernels per operation, uses lazy tensors for aggressive fusion, and has a PyTorch-compatible API. The tiny corp also sells the tinybox, a deep learning workstation available in red ($12K, 4x 9070XT) and green ($65K, 4x RTX PRO 6000 Blackwell) configurations, with an exabox (~$10M, ~1 EXAFLOP) planned for 2027. The framework is used in production in comma.ai's openpilot for autonomous driving inference.

5m read timeFrom tinygrad.org
Post cover image
Table of contents
tinygradWork at tiny corptinybox (now shipping)FAQ

Sort: