Towards Compositional Generalization in Robot Learning

Danfei Xu, Stanford University

11/24/2020

Location: Zoom

Time: 2:55p.m.

Abstract: As robot hardwares become more capable, we will want robots to assist us with wide ranges of long-horizon tasks in open-world environments, such as cooking in a messy kitchen. This requires robots to generalize to new tasks and situations they have never seen before. Despite substantial progress, much of today’s data-driven robot learning systems are limited to optimizing for a single environment and task. On the other hand, long-horizon tasks are composable by nature: short primitives such as grasp-mug and open-drawer constitute long manipulation sequences; a composite goal such as cook-meal can be broken down to simpler subgoals such as preparing individual ingredients. However, there are multitudes of challenges in extracting these structures from the unstructured world, organizing them with coherent task structures, and composing them to solve new tasks. In this talk, I will present some of my Ph.D. works on developing compositional representations and structured learning algorithms to enable robots to generalize across long-horizon manipulation tasks.

An “Additional View” on Human-Robot Interaction and Autonomy in Robot-Assisted Surgery

Alaa Eldin Abdelaal, University of British Columbia

11/17/2020

Location: Zoom

Time: 2:55p.m.

Abstract: Robot-assisted surgery (RAS) has gained momentum over the last few decades with nearly 1,200,000 RAS procedures performed in 2019 alone using the da Vinci Surgical System, the most widely used surgical robotics platform. The current state-of-the-art surgical robotic systems use only a single endoscope to view the surgical field. In this talk, we present a novel design of an additional “pickup” camera that can be integrated into the da Vinci Surgical System. We then explore the benefits of our design for human-robot interaction (HRI) and autonomy in RAS. On the HRI side, we show how this “pickup” camera improves depth perception as well as how its additional view can lead to better surgical training. On the autonomy side, we show how automating the motion of this camera provides better visualization of the surgical scene. Finally, we show how this automation work inspires the design of novel execution models of the automation of surgical subtasks, leading to superhuman performance.

Robot Learning in the Wild

Lerrel Pinto, NYU

11/3/2020

Location: Zoom

Time: 2:55p.m.

Abstract: While robotics has made tremendous progress over the last few decades, most success stories are still limited to carefully engineered and precisely modeled environments. Interestingly, one of the most significant successes in the last decade of AI has been the use of Machine Learning (ML) to generalize and robustly handle diverse situations. So why don’t we just apply current learning algorithms to robots? The biggest reason is a complicated relationship between data and robotics. In other fields of AI such as computer vision, we were able to collect diverse real-world, large-scale data with lots of supervision. These three key ingredients which fueled the success of deep learning in other fields are the key bottlenecks in robotics. We do not have millions of training examples in robots; it is unclear how to supervise robots and most importantly, simulation/lab data is not real-world and diverse. My research has focused on rethinking the relationship between data and robotics to fuel the success of robot learning. Specifically, in this talk, I will discuss three aspects of data that will bring us closer to generalizable robotics: (a) size of data we can collect, (b) amount of supervisory signal we can extract, and (c) diversity of data we can get from robots.

Robots in education – how robots can help learn math, science and engineering

Harshal Chhaya and Ayesha Mayhugh, TI

10/27/2020

Location: Zoom

Time: 2:55p.m.

Abstract: Robots are a fun and engaging way to learn a variety of subjects and concepts – from middle school math to autonomous driving using sensors. In this talk, we will discuss two of TI’s educational robotic products – TI-Rover and TI-RSLK – and how they are being used to teach students across all grades. We will share lessons we learned along the way. We will also share some of the engineering trade-offs we had to make in the design for these robots.

Information Theoretical Regret Bounds for Online Nonlinear Control

Wen Sun, Cornell University

10/21/2020

Location: Zoom

Time: 2:55p.m.

Abstract: This work studies the problem of sequential control in an unknown, nonlinear dynamical system, where we model the underlying system dynamics as an unknown function in a known Reproducing Kernel Hilbert Space. This framework yields a general setting that permits discrete and continuous control inputs as well as non-smooth, non-differentiable dynamics. Our main result, the Lower Confidence-based Continuous Control algorithm, enjoys a near-optimal O(\sqrt{T}) regret bound against the optimal controller in episodic settings, where T is the number of episodes. The bound has no explicit dependence on dimension of the system dynamics, which could be infinite, but instead only depends on information theoretic quantities. We empirically show its application to a number of nonlinear control tasks and demonstrate the benefit of exploration for learning model dynamics.

Joint work with Sham Kakade, Akshay Krishnamurthy, Kendall Lowrey, Motoya Ohnishi. https://arxiv.org/pdf/2006.12466.pdf

Challenges & Opportunities in Maritime Robotics

Matthew Bays, Naval Surface Warfare Center, Panama City Division (NSWC PCD)

10/13/2020

Location: Zoom

Time: 2:55p.m.

Abstract: Interest in unmanned systems has increased considerably within the maritime domain and specifically the U.S. Navy over the last several decades. However, the littoral (shallow water) and undersea environments offer unique challenges resulting in the need for more autonomous, more reliable, and more modular unmanned systems than is often found in other domains. In this talk, we will provide an overview of the particular challenges the U.S. Navy is attempting to solve or mitigate within the littoral environment and solutions currently in development. These challenges include the unique communication constraints of the underwater domain, the difficult maritime sensing environment, and reliability needs of undersea systems.

RoboGami

10/6/2020

Location: Zoom

Time: 2:55p.m.

Abstract: GSGIC (Graduate Students for Gender Inclusion in Computing) + RGSO (Robotics Graduate Student Organization/Robotics Seminar) invite you to join us for an afternoon of origami to build community and build paper decorations for your home/office.

For those who requested materials you should be receiving them in the mail. If you did not previously RSVP feel free to join the event with your own paper anyways!

gradSLAM: Bridging classical and learned methods in SLAM

Krishna Murthy Jatavallabhula, Robotics and Embodied AI Lab (REAL), Mila, Universite de Montreal

9/22/2020

Location: Zoom

Time: 2:55p.m.

Abstract: Modern machine learning has ushered in a new air of excitement in the design of intelligent robots. In particular, gradient-based learning architectures (deep neural networks) have enabled significant strides in robot perception, reasoning, and action. With all of these advancements, one might wonder if “classical” techniques for robot perception and state estimation are relevant in this age. I postulate that a flexible blend of “classical” and “learned” methods is the best foot forward for robot intelligence.

“What is the ideal way to combine “classical” techniques with gradient-based learning architectures?” This is the central question that my research strives to answer. I argue that such a blend should be seamless: we must neither disregard domain-specific inductive biases that influence the design of “classical” robots, nor should we compromise on the representational power that learning-based techniques offer. In particular, I tackle the problem of blending gradient-based learning with visual simultaneous localization and mapping (SLAM), and the new possibilities this opens up. My talk will focus on “gradSLAM”: a fully differentiable dense SLAM system that harnesses the power of computational graphs and automatic differentiation, to enable a new perspective of thinking about deep learning for SLAM.

RGSO Research Recaps

Claire Liang, Michael Suguitan, PhD students, Cornell

9/15/2020

Location: Zoom

Time: 2:55p.m.

Abstracts:

For our first homegrown seminar, some of our grad students will present summaries of their recent work since the Spring and Summer. This is an opportunity for both returning students to catch up with others’ projects and to let new students learn about the breadth of research within robotics at Cornell.

Cornell Tech Lightning Talks

Valts Blukis, Ilan Mandel, David Goedicke, Natalie Friedman, Travers Rhodes, PhD students, Cornell Tech

5/5/2020

Location: Zoom

Time: 2:45p.m.

Abstracts:

Natalie Friedman: Within human-robot interaction, I study how robots should be designed to move and behave in various contexts, based on the perception of social appropriateness.

David Goedicke: Our lab works mostly on resining implicit interaction for devices that to some degree make their own decision. I will show past work on Autonomous Vehicles (as large robots one sits in). How we use Virtual Reality to explore interaction, and my new research direction, which is Acoustically aware robots.

Valts Blukis: We study representation learning approaches for building robots that understand natural language in context of raw visual and sensory observations. I’ll present our recent work on mapping raw images and navigation instructions to physical quadcopter control, using a neural network model trained using simulated and real data. The model reasons about the need to explore the environment and incorporates geometric computation to predict which locations in the environment to visit. Finally, I’ll talk about the challenges when scaling representation learning methods to reason about previously unseen objects and environments.

Travers Rhodes: Variational Auto-Encoders (VAEs) have been known to “ignore” some latent-variable dimensions in their representations. This talk explores known results for what those representations look like for simplified, linear VAEs and presents some directions for future work on more complicated VAEs.