Lobsters is a community-driven platform for sharing and discussing links to articles, tutorials, and projects related to technology and programming. Readers can learn about a wide range of topics, from software development and system administration to cybersecurity and artificial intelligence. With user submissions, comments, and voting, Lobsters provides a platform for collaborative learning and knowledge sharing among technology enthusiasts.

Lobsters

Andrej Karpathy introduces microgpt, a 200-line pure Python implementation of GPT that requires no dependencies. The project distills the complete algorithmic essence of training and running a GPT model into a single file, covering tokenization, autograd from scratch, the Transformer architecture (attention, MLP, embeddings), the Adam optimizer, and inference. Trained on 32,000 names, the tiny model learns to generate plausible new names. The guide walks through each component step-by-step, from the Value class implementing backpropagation to the multi-head attention mechanism. While production LLMs like ChatGPT add massive scale, better tokenizers, GPU optimization, and post-training, microgpt captures the core algorithm that powers all modern language models.