Calvin Luo - Understanding diffusion models: A unified perspective

Title: Understanding diffusion models: A unified perspective Abstract: Diffusion models have shown incredible capabilities as generative models; indeed, they power the current state-of-the-art models on text-conditioned image generation such as Imagen and DALL-E 2. In this work we review, demystify, and unify the understanding of diffusion models across both variational and score-based perspectives. We first derive Variational Diffusion Models (VDM) as a special case of a Markovian Hierarchical Variational Autoencoder, where three key assumptions enable tractable computation and scalable optimization of the ELBO. We then prove that optimizing a VDM boils down to learning a neural network to predict one of three potential objectives: the original source input from any arbitrary noisification of it, the original source noise from any arbitrarily noisified input, or the score function of a noisified input at any arbitrary noise level. We then dive deeper into what it means to learn the score function, and connect the variational perspective of a diffusion model explicitly with the Score-based Generative Modeling perspective through Tweedie's Formula. Lastly, we cover how to learn a conditional distribution using diffusion models via guidance.Paper link: https://arxiv.org/abs/2208.11970 About the Speaker: Calvin Luo, is a PhD Student at Brown University, advised by the Chen Sun. Previously, he was an AI Resident at Google in Mountain View, where he worked on representation learning, model-based reinforcement learning, generalization, and adversarial robustness For previous session recordings please visit - https://sites.google.com/cohere.com/c4ai-community/community-programs/ml-math This session is brought to you by the Cohere For AI Open Science Community - a space where ML researchers, engineers, linguists, social scientists, and lifelong learners connect and collaborate with each other. Thank you to our Community Leads for organizing and hosting this event. If you’re interested in sharing your work, we welcome you to join us! Simply fill out the form at https://forms.gle/ALND9i6KouEEpCnz6 to express your interest in becoming a speaker. Join the Cohere For AI Open Science Community to see a full list of upcoming events: https://tinyurl.com/C4AICommunityApp.