Goal Misgeneralization: How a Tiny Change Could End Everything

This video explores how YOU, YES YOU, are a case of misalignment with respect to evolution's implicit optimization objective. We also show an example of goal misgeneralization in a simple AI system, and explore how deceptive alignment shares similar features and may arise in future, far more powerful AI systems. ▀▀▀▀▀▀▀▀▀SOURCES & READINGS▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ Quantifying generalization in reinforcement learning: https://openai.com/index/quantifying-generalization-in-reinforcement-learning/ Goal Misgeneralisation: Why Correct Specifications Aren’t Enough For Correct Goals: https://deepmindsafetyresearch.medium.com/goal-misgeneralisation-why-correct-specifications-arent-enough-for-correct-goals-cf96ebc60924 The Simple Math of Evolution: https://www.lesswrong.com/s/MH2b8NfWv22dBtrs8 Rob Miles' explainer about deceptive alignment: https://youtu.be/IeWljQw3UgQ CoinRun original videos: https://sites.google.com/view/goal-misgeneralization/home/coinrun-videos ▀▀▀▀▀▀▀▀▀PATREON, MEMBERSHIP, MERCH▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 🟠 Patreon: https://www.patreon.com/rationalanimations 🔵 Channel membership: https://www.youtube.com/channel/UCgqt1RE0k0MIr0LoyJRy2lg/join 🟢 Merch: https://rational-animations-shop.fourthwall.com 🟤 Ko-fi, for one-time and recurring donations: https://ko-fi.com/rationalanimations ▀▀▀▀▀▀▀▀▀SOCIAL & DISCORD▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ Discord: https://discord.gg/RationalAnimations Reddit: https://www.reddit.com/r/RationalAnimations/ X/Twitter: https://twitter.com/RationalAnimat1 Instagram: https://www.instagram.com/rationalanimations/ ▀▀▀▀▀▀▀▀▀PATRONS & MEMBERS▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ Michael Hewitt ttw Ellis Jones Danilo Stefani - Alessandra Erba Amir Saboury Vivek Menon Forodriac Origamius JanJan Autochthon Tristan Logan J McMillan Matthew Shinkle Alex Amadori SQRT42Pi David Piepgrass Tomas Campos Jana Ingvi Gautsson BlueNotesBlues Terberlo.dog Michael Andregg Vladimir Silyaev Alcher Black RMR Kristin Lindquist Nathan Metzger Glenn Tarigan NMS James Babcock Colin Ricardo Long Hoang Tor Barstad steven michaels Apuis Retsam Stuart Alldritt Chris Painter Juan Benet Christian Loomis Tomarty Edward Yu Chad M Jones Honyopenyoko Neal Strobl bparro Danealor loopuleasa Craig Falls Vincent Weisser Alex Hall Ivan Bachcin joe39504589 Klemen Slavic blasted0glass Scott Alexander noggieB Dawson John Slape Gabriel Ledung Jeroen De Dauw Superslowmojoe Nathan Fish Bleys Goodson Ducky rictic Luke Freeman Jonathan Plasse 7ic7ac Devin King RED Rinthean Thomas Grip Boris Bend J H Richard Stambaugh Teo Val Ken Mc Alcher Black ronvil AWyattLife Lazy Scholar Torstein Haldorsen MichaÅ‚ ZieliÅ„ski ▀▀▀▀▀▀▀CREDITS▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ Directed by: Hannah Levingstone | @hannah_luloo Written by: Markov :3 Producer: :3 Line Producer: Kristy Steffens | https://linktr.ee/kstearb Production Managers: Kristy Steffens | https://linktr.ee/kstearb Jay McMichen | @Jay_TheJester Art Director: Hané Harnett | @Peony_Vibes / @peonyvibes (insta) Quality Assurance Lead: Lara Robinowitz | @CelestialShibe Visual Development: Pierre Broissand | @pierrebrsnd (insta) / artstation.com/brsnd Hané Harnett | @Peony_Vibes / @peonyvibes (insta) Olivia Wang | @whalesharkollie Storyboard Artist: Ira Klages | @/dux (insta) Animation: Michela Biancini Jordan Gilbert | @Twin_Knight / Twin Knight Studios (YT) Zack Gilbert | @Twin_Knight / Twin Knight Studios (YT) Colors Giraldo | @colorsofdoom Keith Kavanagh | @johnnycigarettex Ira Klages | @/dux (insta) Skylar O’Brien | @mutodaes Owen Peurois | @owenpeurois Background Art: Hané Harnett | @Peony_Vibes / @peonyvibes (insta) Zoe Martin-Parkinson | @zoemar_son Olivia Wang | @whalesharkollie Compositing: Renan Kogut | @kogut_r Patrick O' Callaghan | @patrick.h264 Narration: Rob Miles | https://www.youtube.com/c/robertmilesai VO Editor: Tony Dipiazza Original Soundtrack & Sound Design: Epic Mountain | https://www.instagram.com/epicmountainmusic/