Who's Adam and What's He Optimizing? | Deep Dive into Optimizers for Machine Learning!

Welcome to our deep dive into the world of optimizers! In this video, we'll explore the crucial role that optimizers play in machine learning and deep learning. From Stochastic Gradient Descent to Adam, we cover the most popular algorithms, how they work, and when to use them. 🔍 What You'll Learn: Basics of Optimization - Understand the fundamentals of how optimizers work to minimize loss functions Gradient Descent Explained - Dive deep into the most foundational optimizer and its variants like SGD, Momentum, and Nesterov Accelerated Gradient Advanced Optimizers - Get to grips with Adam, RMSprop, and AdaGrad, learning how they differ and their advantages Intuitive Math - Unveil the equations for each optimizer and learn how it stands out from the others Real World Benchmarks - See real world experiments from papers in domains ranging from computer vision to reinforcement learning to see how these optimizers fare against each other 🔗 Extra Resources: 3Blue1Brown - https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi Artem Kirsanov - https://www.youtube.com/watch?v=SmZmBKc7Lrs 📌 Timestamps: 0:00 - Introduction 1:17 - Review of Gradient Descent 5:37 - SGD w/ Momentum 9:26 - Nesterov Accelerated Gradient 10:55 - Root Mean Squared Propagation 13:59 - Adaptive Gradients (AdaGrad) 14:47 - Adam 18:12 - Benchmarks 22:01 - Final Thoughts Stay tuned and happy learning!