Learning Robot Control: From RL to Differential Simulation - (PhD Defense of Yunlong Song)

This thesis focuses on Learning Robot Control by integrating deep reinforcement learning (RL) and model-based control methods. It aims to develop advanced control methods that bridge the gap between data-driven learning and model-based control. The proposed methods enhance robot agility and robustness in real-world applications. Key contributions are: - Show that RL outperforms Optimal Control in autonomous racing because it directly optimizes a non-differentiable task-level objective. - Propose a policy-search-for-model-predictive-control (MPC) framework, combining RL's ability to optimize high-level task objectives with MPC's precise actuation and constraint handling. - Introduce a differentiable simulation framework to leverage robot dynamics for more stable and - efficient policy training. - Develop a high-performance drone racing system outperforming optimal control methods and professional pilots. - Develop Flightmare, a flexible modular quadrotor simulator for reinforcement learning and vision-based flight. OUTLINE: 00:00 - Introduction 02:37 - Robot Control: An Optimal Control Perspective 03:14 - Robot Control: A Reinforcement Learning Perspective 05:06 - Project 1: Autonomous Drone Racing: Optimal Control vs. Reinforcement Learning 12:05 - Project 2: Flying Through Dynamic Gates: Reinforcement Learning for Optimal Control 16:04 - Project 3: Quadrupedal Locomotion: Differentiable Simulation 20:18 - Conclusions 23:05 - One More Thing