Towards Data Science is a community-powered publication that showcases work in data science, machine learning and artificial intelligence. Every day newcomers, seasoned researchers and industry practitioners publish tutorials, research notes and real-world case studies that help the field move forward.

Towards Data Science

Sharpness-Aware Minimization (SAM) is an optimizer designed to improve generalization in overparameterized deep learning models by seeking flat minima in the loss landscape rather than just minimizing loss. The article explains the mathematical intuition behind sharpness and why flat minima correlate with better generalization, then walks through the SAM algorithm step by step: computing adversarial perturbations via a first forward/backward pass, then updating weights using gradients evaluated at the perturbed point. A PyTorch implementation is provided, including an important caveat about disabling BatchNorm running statistics during the second forward pass. A practical demo on Fashion-MNIST with a PreAct ResNet-18 shows SAM achieves a generalization gap of 2.3% vs. 6.8% for standard SGD, with a modest improvement in test accuracy (92.5% vs. 92.0%).

Optimizing Deep Learning Models with SAM

Introduction : Overparameterization, Generalizability, and SAM

The Sharpness-Aware Minimization (SAM) Algorithm

PyTorch Implementation in a Training Loop

Demo: Image classification with ResNet-18