Training learned optimizers: VeLO paper EXPLAINED

Why tune optimizers hyperparameters (Adam) by hand, when one can train a neural network to behave like an optimizer and dynamically find the best update for your neural network’s weights? In this video, we explain the work on VeLO to train an optimizer from data from previous training runs. ► Sponsor: Cohere 👉 https://t1p.de/22srn Check out our daily #MachineLearning Quiz Questions: https://www.youtube.com/c/AICoffeeBreak/community ➡️ AI Coffee Break Merch! 🛍️ https://aicoffeebreak.creator-spring.com/ 📜 VeELO paper: Metz, Luke, James Harrison, C. Daniel Freeman, Amil Merchant, Lucas Beyer, James Bradbury, Naman Agrawal et al. "VeLO: Training Versatile Learned Optimizers by Scaling Up." https://arxiv.org/abs/2211.09760 Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏 Dres. Trost GbR, Siltax, Edvard Grødem, Vignesh Valliappan, Mutual Information, Mike Ton Outline: 00:00 VeLO optimizer without any hyperparameters 01:13 Cohere [Sponsor] 02:27 What are optimizers? 04:37 VeLO idea and training data 06:43 VeLO model and training 10:15 What can VeLO do? 11:52 Limitations of VeLO ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕ Patreon: https://www.patreon.com/AICoffeeBreak Ko-fi: https://ko-fi.com/aicoffeebreak ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 🔗 Links: AICoffeeBreakQuiz: https://www.youtube.com/c/AICoffeeBreak/community Twitter: https://twitter.com/AICoffeeBreak Reddit: https://www.reddit.com/r/AICoffeeBreak/ YouTube: https://www.youtube.com/AICoffeeBreak #AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research​ Video editing: Nils Trost Music 🎵 : Hey There - half.cool