Towards Compositional Generalization in Robot Learning

Danfei Xu, Stanford University


Location: Zoom

Time: 2:55p.m.

Abstract: As robot hardwares become more capable, we will want robots to assist us with wide ranges of long-horizon tasks in open-world environments, such as cooking in a messy kitchen. This requires robots to generalize to new tasks and situations they have never seen before. Despite substantial progress, much of today’s data-driven robot learning systems are limited to optimizing for a single environment and task. On the other hand, long-horizon tasks are composable by nature: short primitives such as grasp-mug and open-drawer constitute long manipulation sequences; a composite goal such as cook-meal can be broken down to simpler subgoals such as preparing individual ingredients. However, there are multitudes of challenges in extracting these structures from the unstructured world, organizing them with coherent task structures, and composing them to solve new tasks. In this talk, I will present some of my Ph.D. works on developing compositional representations and structured learning algorithms to enable robots to generalize across long-horizon manipulation tasks.