TCOptRob Seminar: On the Geometric Foundations of Continuous Control by Nathan Ratliff

TCOptRob Seminar: On the Geometric Foundations of Continuous Control by Nathan Ratliff of NVIDIA (https://www.nvidia.com) 00:00 Intro 01:58 The Talk 49:40 Q&A Title: On the Geometric Foundations of Continuous Control Abstract: In 2022, Bertsekas made a deep observation in Lessons from AlphaZero. He showed that finite-horizon optimization over an actor-critic network is a powerful Newton step on the Bellman equation. In continuous control, this observation is salient because it provides a unifying perspective on optimal control and reinforcement learning, two workhorses which have separately been extremely successful in the field. In this talk, we argue that these observations will shape decision systems into the future, and rather than optimizing over standard physical dynamics, optimal control methods will move toward optimizing more generally over increasingly sophisticated models of behavioral dynamics that capture not only physics but also the statistics (a memory) of past optimal control solutions. We present a class of behavioral dynamics functions called geometric fabrics which build off the geometric properties of classical mechanics to create expressive path priors for statistically efficient behavior synthesis. We show both how to manually design these fabrics to capture relevant engineered behaviors such as obstacle avoidance, end-effector attraction, posturing, etc., and how to learn the fabrics from demonstration, achieving striking sample efficiencies from their geometric inductive bias. Bertsekas’s results are observations on the structure of abstract dynamic programming and apply equally well to discrete systems like AlphaZero and continuous systems like those of robotics. But to leverage them effectively in robotics, we need to better understand how to efficiently encode information into the continuous time differential equations that govern robotic behavior. Geometric fabrics are a step in that direction. Speaker Bio: Nathan Ratliff is a director of systems software at NVIDIA. He holds a PhD in robotics from CMU and his career spans both academia and industry, from research scientist positions at TTI-C, Intel, U. Stuttgart and Max Planck, to core engineering work at Google, Amazon, and his own startup Lula Robotics. His work focuses on developing the enabling theory and technologies to support higher-level programming of collaborative robots. Nathan was a founding member of the robotics research group at NVIDIA where he developed the real-time architectures and motion generation techniques underlying many of NVIDIA’s robotic manipulation systems. Nathan is now part of NVIDIA’s Simulation Technologies group building Isaac Cortex on the Omniverse platform in Isaac Sim. -------------------------------------------------------------- The IEEE RAS Technical Committee on Model Based Optimization for Robotics is focused on building and supporting a community of researchers and practioners focused on the development and application of model-based optimization techniques for the generation and control of dynamic behaviors in robotics and their practical implementation. You can find our more about the TC at: https://www.ieee-ras.org/model-based-optimization-for-robotics. Copyright (c) 2023 TCOptRob. All rights reserved. License: CC BY-NC-SA 4.0