The Misconception that Almost Stopped AI [How Models Learn Part 1]

Take your personal data back with Incogni! Use code WELCHLABS and get 60% off an annual plan: http://incogni.com/welchlabs Loss Landscape Posters! 21:23 https://www.welchlabs.com/resources/loss-landscape-poster-17x19 https://www.welchlabs.com/resources/loss-landscape-poster-digital-download Poster and Book Bundle https://www.welchlabs.com/resources/loss-landscape-bundle-w-imaginary-numbers-book Special Matte Black Edition Poster https://www.welchlabs.com/resources/loss-landscape-poster-17x22-matte-black-special-edition Welch Labs Book https://www.welchlabs.com/resources/imaginary-numbers-book Sections 0:00 - Intro 1:18 - How Incogni gets me more focus time 3:01 - What are we measuring again? 6:18 - How to make our loss go down? 7:32 - Tuning one parameter 9:11 - Tuning two parameters together 11:01 - Gradient descent 13:18 - Visualizing high dimensional surfaces 15:10 - Loss Landscapes 16:55 - Wormholes! 17:55 - Wikitext 18:55 - But where do the wormholes come from? 20:00 - Why local minima are not a problem 21:23 - Posters Special Thanks to Patrons https://www.patreon.com/welchlabs Juan Benet, Ross Hanson, Yan Babitski, AJ Englehardt, Alvin Khaled, Eduardo Barraza, Hitoshi Yamauchi, Jaewon Jung, Mrgoodlight, Shinichi Hayashi, Sid Sarasvati, Dominic Beaumont, Shannon Prater, Ubiquity Ventures, Matias Forti, Brian Henry, Tim Palade, Petar Vecutin, Nicolas baumann, Jason Singh, Robert Riley, vornska, Barry Silverman, Jake Ehrlich, Mitch Jacobs References Li et al: Visualizing the Loss Landscape of Neural Nets. https://arxiv.org/abs/1712.09913 Talking Nets: An Oral History of Neural Networks. (2000). United Kingdom: MIT Press. Hinton quote is on p376. Goodfellow, I., Bengio, Y., Courville, A. (2016). Deep Learning. United Kingdom: MIT Press. Prince, S. J. (2023). Understanding Deep Learning. United Kingdom: MIT Press. Manim Animations: https://github.com/stephencwelch/manim_videos Premium Beat IDs MWROXNAY0SPXCMBS