Best of Deep Learning — January 2026

1. 1
   Article

   Daily Dose of Data Science | Avi Chawla | Substack·16w

   Phases of ML Modeling

   ML systems should evolve through four distinct phases rather than jumping straight to complex models. Start with simple heuristics and rules (Phase 1), then move to basic ML models like logistic regression (Phase 2), optimize through feature engineering and hyperparameter tuning (Phase 3), and only adopt complex models like deep neural networks when simpler approaches are exhausted (Phase 4). This staged approach reduces risk, improves debuggability, and ensures each phase's best model becomes the baseline for the next, encouraging incremental progress and evidence-driven decision-making.

   91

   1
2. 2
   Article

   DigitalOcean Community·20w

   Olmo 3: Fully Open-Source LLM from AI2 (Models, Data, & Code)

   Olmo 3 is Allen AI's fully open-source large language model available in 7B and 32B parameter versions. The release includes complete access to models, training datasets (Dolma 3 with 9.3 trillion tokens), code, and training logs. The model uses a three-stage training pipeline: pretraining on Dolma 3 Mix, mid-training on Dolma 3 Dolmino for skill enhancement, and long-context extension on Dolma 3 Longmino. Post-training uses the Dolci suite with SFT, DPO, and RLVR techniques. The 32B model employs grouped query attention while the 7B uses multi-head attention. OlmoTrace enables tracing text back to training sources for auditing and contamination detection.

   83

   1
3. 3
   Article

   Phoronix·18w

   Burn 0.20 Released: Rust-Based Deep Learning With Speedy Perf Across CPUs & GPUs

   Burn 0.20 introduces CubeK, a high-performance multi-platform kernel system built on CubeCL that enables unified CPU and GPU execution across NVIDIA CUDA, AMD ROCm, Apple Metal, WebGPU, and Vulkan. The release aims to deliver peak performance on diverse hardware without maintaining fragmented codebases, with benchmarks showing significantly lower execution times compared to LibTorch and ndarray. The update also includes a complete overhaul of the ONNX import system and various stability improvements.

   48

   1
4. 4
   Article

   Hugging Face·18w

   Differential Transformer V2

   Differential Transformer V2 introduces a redesigned attention mechanism that doubles query heads while maintaining key-value heads, eliminating the need for custom kernels and achieving faster decoding speeds. The architecture removes per-head RMSNorm to improve training stability, introduces token-level and head-level lambda projections to overcome softmax constraints, and eliminates attention sinks. Production-scale experiments on trillion-token datasets show 0.02-0.03 lower language modeling loss, reduced gradient spikes under large learning rates, and decreased activation outliers compared to standard Transformers, while saving approximately 25% of attention module parameters.

   38
5. 5
   Article

   Sebastian Raschka·17w

   Categories of Inference-Time Scaling for Improved LLM Reasoning

   Inference-time scaling improves LLM answer quality by allocating more compute during text generation rather than training. The article categorizes different approaches including chain-of-thought prompting, self-consistency, best-of-N ranking, rejection sampling, self-refinement, and search over solution paths. Major LLM providers use these techniques, which can boost model accuracy significantly without changing model weights. The piece draws from research for a book chapter that improved base model accuracy from 15% to 52%.

   22
6. 6
   Article

   Daily Dose of Data Science | Avi Chawla | Substack·17w

   Build Agents That Can Learn Like Humans

   ART (Agent Reinforcement Trainer) is an open-source framework that simplifies reinforcement learning for LLMs by eliminating manual reward function engineering. It uses GRPO (Group Relative Policy Optimization) where agents attempt tasks multiple times, an LLM judge compares attempts, and the model learns from relative performance. Unlike traditional RL frameworks limited to simple chatbot interactions, ART supports multi-turn conversations, tool calls, and integrates with LangGraph, CrewAI, and ADK. It combines vLLM for model serving and Unsloth for GRPO training, enabling developers to fine-tune small open-source models to outperform larger closed-source alternatives on specific tasks.

   14