Best of Sebastian Raschka — 2025

1. 1
   Article

   Sebastian Raschka·50w

   Coding LLMs from the Ground Up: A Complete Course

   Sebastian Raschka shares a comprehensive video course series on building Large Language Models from scratch using Python and PyTorch. The course covers seven key areas: environment setup, text data preprocessing and tokenization, attention mechanisms implementation, LLM architecture coding, pretraining on unlabeled data, classification fine-tuning, and instruction fine-tuning. The content serves as supplementary material to his book 'Build a Large Language Model (From Scratch)' and emphasizes hands-on learning through implementation rather than using pre-built frameworks.

   420

   5
2. 2
   Article

   Sebastian Raschka·21w

   The State Of LLMs 2025: Progress, Problems, and Predictions

   A comprehensive 2025 review of large language model developments highlights reinforcement learning with verifiable rewards (RLVR) and the GRPO algorithm as the year's dominant training paradigm, following DeepSeek R1's breakthrough. Key trends include inference-time scaling, tool use integration, and architectural efficiency tweaks like mixture-of-experts and linear attention mechanisms. The analysis addresses benchmarking challenges ("benchmaxxing"), discusses practical LLM usage for coding and writing, and examines the shift toward domain-specific models with proprietary data. Predictions for 2026 emphasize RLVR expansion beyond math/code, increased inference optimization, and the emergence of diffusion models for low-latency tasks.

   176

   1
3. 3
   Article

   Sebastian Raschka·28w

   Recommendations for Getting the Most Out of a Technical Book

   A structured five-step approach to learning from technical books: start with an offline read-through to grasp the big picture, follow with hands-on coding by retyping examples, complete exercises to solidify understanding, review notes and explore additional resources, and finally apply concepts in personal projects. The method emphasizes focused reading sessions, active engagement with code, and practical application over passive consumption.

   141

   7
4. 4
   Article

   Sebastian Raschka·1y

   The State of LLM Reasoning Models

   The post explores recent research advancements in reasoning-optimized large language models (LLMs), focusing on inference-time compute scaling methods. It discusses how various techniques, such as chain-of-thought reasoning and test-time preference optimization, improve the reasoning abilities of LLMs without altering underlying model weights. The article highlights the importance of increasing computational resources during inference to enhance performance, making even smaller models more capable. It also touches on other methods like reinforcement learning and supervised fine-tuning that contribute to improved reasoning in LLMs.

   53