Description:
Reinforcement learning (RL) has shown promising performance across a variety of complex domains; however, its high sample complexity limits broader application---particularly in real-world, sparse reward settings. In this talk, Matthew Walter, associate professor and director of the Robot Intelligence through Perception Laboratory (RIPL) at the Toyota Technological Institute at Chicago, will describe a body of work that addresses this challenge by drawing on demonstrations and, increasingly, raw action-free video, an abundant and underexplored source of supervision for robot learning.
Walter will begin by presenting a class of imitation learning algorithms capable of directly learning from demonstrations, even when they are suboptimal. Key to these algorithms is their ability to adaptively determine when and how to rely on different demonstrators, and to transition from imitation- to reinforcement-based learning, enabling the learner to outperform the demonstrators. He will then describe work that learns a task-agnostic, progress-based reward function from action-free video demonstrations---without access to action labels or manually specified rewards. Pretrained on large-scale egocentric human videos, this reward model generalizes across tasks and even across embodiments, supporting goal-conditioned policy learning from the large-scale, uncurated video data available on the internet. Finally, he will present very recent work that takes advantage of action-free visual demonstrations to learn a world model jointly with continuous latent action representations. By leveraging adversarial regularization and diffusion-based video generation, this approach learns structured, semantically meaningful action representations that support both imitation learning from observation and goal-directed planning.
This talk is supported by the Data, AI, and Computing Initiative's Physical AI Working Group.
About the Speaker:
