Lesson 10: Deep Learning Foundations to Stable Diffusion, 2022

(All lesson resources are available at http://course.fast.ai.) This lesson creates a complete Diffusers pipeline from the underlying components: the VAE, unet, scheduler, and tokeniser. By putting them together manually, this gives you the flexibility to fully customise every aspect of the inference process. We also discuss three important new papers that have been released in the last week, which improve inference performance by over 10x, and allow any photo to be “edited” by just describing what the new picture should show. The second half of the lesson begins the “from the foundations” stage of the course, developing a basic matrix class and random number generator from scratch, as well as discussing the use of iterators in Python. You can discuss this lesson, and access links to all notebooks and resources from it, at this forum topic: https://forums.fast.ai/t/lesson-10-part-2-preview/101337 Additional Links: - Progressive Distillation for Fast Sampling of Diffusion Models - https://arxiv.org/abs/2202.00512 - On Distillation of Guided Diffusion Models - https://arxiv.org/abs/2210.03142 - Imagic: Text-Based Real Image Editing with Diffusion Models - https://arxiv.org/abs/2210.09276 0:00 - Introduction 0:35 - Showing student’s work over the past week. 6:04 - Recap Lesson 9 12:55 - Explaining “Progressive Distillation for Fast Sampling of Diffusion Models” & “On Distillation of Guided Diffusion Models” 26:53 - Explaining “Imagic: Text-Based Real Image Editing with Diffusion Models” 33:53 - Stable diffusion pipeline code walkthrough 41:19 - Scaling random noise to ensure variance 50:21 - Recommended homework for the week 53:42 - What are the foundations of stable diffusion? Notebook deep dive 1:06:30 - Numpy arrays and PyTorch Tensors from scratch 1:28:28 - History of tensor programming 1:37:00 - Random numbers from scratch 1:42:41 - Important tip on random numbers via process forking Thanks to fmussari for the transcript, and to Raymond-Wu (on forums.fast.ai) for the timestamps.