Actuate 2024 | Sergey Levine | Robotic Foundation Models

General-purpose models trained on large and diverse datasets can often outperform more specialized domain-specific systems, and have demonstrated remarkable capabilities in domains from natural language processing to computer vision. But what would such a generalist model look like in robotics? In this presentation, I will discuss steps toward building broadly applicable robotic foundation models, from assembling suitable datasets, to devising viable and general training objectives, to model architectures that can control a wide range of different robotic platforms, as well as the different ways such models can be used..