Diffusion models are powerful generative models that enable many successful applications like image, video, and 3D generation from texts. 

In this tutorial, I share my understanding of the diffusion model basics, including training, guidance, resolution, and speed. 

Below are some other great resources to learn more about diffusion models.

===== Slides =====
Here are the slides used in this video

Training: https://bit.ly/3WudEPH
Guidance: https://bit.ly/3wedCky
Resolution: https://bit.ly/4bqxHmo
Speed: https://bit.ly/4bpJzoJ

===== Tutorials ===== 
[CVPR 2022 Tutorial] Denoising Diffusion-based Generative Modeling: Foundations and Applications 
https://cvpr2022-tutorial-diffusion-models.github.io/
[CVPR 2023 Tutorial] Denoising Diffusion Models: A Generative Learning Big Bang
https://cvpr2023-tutorial-diffusion-models.github.io/
[A short course by DeepLearning.AI] How Diffusion Models Work
https://www.youtube.com/watch?v=obdVesVsGQI

=====  Training ===== 
[Sohl-Dickstein et al. 2015] Deep Unsupervised Learning using Nonequilibrium Thermodynamics 
https://arxiv.org/abs/1503.03585
[Ho et al. 2020]: Denoising Diffusion Probabilistic Models 
https://arxiv.org/abs/2006.11239
[Luo 2022] Understanding Diffusion Models: A Unified Perspective https://arxiv.org/abs/2208.11970
[Karras et al. 2022] Elucidating the design space of diffusion-based generative models
https://arxiv.org/abs/2206.00364
[Karras et al. 2023] Analyzing and Improving the Training Dynamics of Diffusion Models
https://arxiv.org/abs/2312.02696

===== Guidance ===== 
[Dhariwal and Nichol 2021] Diffusion Models Beat GANs on Image Synthesis
https://arxiv.org/abs/2105.05233
[Ho and Salimans 2022] Classifier-Free Diffusion Guidance
https://arxiv.org/abs/2207.12598
[Sander Dieleman 2022] Guidance: a cheat code for diffusion models 
https://sander.ai/2022/05/26/guidance.html
[Sander Dieleman 2023] The geometry of diffusion guidance 
https://sander.ai/2023/08/28/geometry.html

===== Resolution ===== 
[Ho et al. 2021] Cascaded Diffusion Models for High Fidelity Image Generation
https://arxiv.org/abs/2106.15282
[Saharia et al. 2022] Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding 
https://arxiv.org/abs/2205.11487
[Rombach et al. 2021] High-Resolution Image Synthesis with Latent Diffusion Models
https://arxiv.org/abs/2112.10752
[Vahdat et al. 2021] Score-based Generative Modeling in Latent Space 
https://proceedings.neurips.cc/paper_files/paper/2021/hash/5dca4c6b9e244d24a30b4c45601d9720-Abstract.html
[Podell et al. 2023] SDXL: Improving Latent Diffusion Models for High-resolution Image Synthesis 
https://arxiv.org/abs/2307.01952
[Hoogeboom et al. 2023] Simple diffusion: End-to-end diffusion for high resolution images
https://arxiv.org/abs/2301.11093
[Chen et al. 2023] On the importance of noise scheduling for diffusion models
https://arxiv.org/abs/2301.10972
[Gu et al. 2023] Matryoshka Diffusion Models
https://arxiv.org/abs/2310.15111

===== Speed ===== 
[Song et al. 2021] Denoising Diffusion Implicit Models 
https://arxiv.org/abs/2010.02502
[Salimans and Ho 2022] Progressive Distillation for Fast Sampling of Diffusion Models 
https://arxiv.org/abs/2202.00512
[Meng et al. 2023] On Distillation of Guided Diffusion Models 
https://arxiv.org/abs/2210.03142
[Song et al. 2023] Consistency models
https://arxiv.org/abs/2303.01469
[Luo et al. 2023] Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference
https://arxiv.org/abs/2310.04378
[Luo et al. 2023] LCM-LoRA: A Universal Stable-Diffusion Acceleration Module
https://arxiv.org/abs/2311.05556
[Sauer et al. 2023] Adversarial Diffusion Distillation 
https://arxiv.org/abs/2311.17042
[Yin et al. 2023] One-step Diffusion with Distribution Matching Distillation
https://arxiv.org/abs/2311.18828

Jia-Bin Huang

A comprehensive walkthrough of how diffusion models work, covering four main areas: training objectives (ELBO, denoising matching, score-based interpretation), conditional generation via classifier guidance and classifier-free guidance, high-resolution image synthesis (cascaded diffusion, latent diffusion models), and sampling acceleration techniques (DDIM, progressive distillation, consistency models, LoRA). The content derives the math behind forward/reverse diffusion processes, explains three equivalent training interpretations (predicting clean image, predicting noise, predicting score), and surveys practical methods to make diffusion models faster and more controllable.

How I Understand Diffusion Models