What we talk about when we talk about tooling

Cornell Robotics Grad Students

9/23/2021

Location: 122 Gates Hall

Time: 2:40p.m.

Abstract: Join RGSO for our first homegrown seminar: a discussion on tooling. We have a few of our very own students ready to talk about their workflows and tips-and-tricks for how they get stuff done, whether it’s programming or research reviews. Come ready to listen, learn, and, if you have a cool workflow to communicate to others, share.

Certifiable Outlier-Robust Geometric Perception: Robots that See through the Clutter with Confidence

Heng Yang, Massachusetts Institute of Technology

9/16/2021

Location: 122 Gates Hall

Time: 2:40p.m.

Abstract: Geometric perception is the task of estimating geometric models from sensor measurements and priors. The ubiquitous existence of outliers —measurements that tell no or little information about the models to be estimated— makes it theoretically intractable to perform estimation with guaranteed optimality. Despite this theoretical intractability, safety-critical robotic applications still demand trustworthiness and performance guarantees on perception algorithms. In this talk, I present certifiable outlier-robust geometric perception, a new paradigm to design tractable algorithms that enjoy rigorous performance guarantees, i.e., they commonly return an optimal estimate with a certificate of optimality, but declare failure and provide a measure of suboptimality on worst-case instances. Particularly, I present three algorithms in the certifiable perception toolbox: (i) a pruner that uses graph theory to filter out gross outliers and boost robustness to against over 95% outliers; (ii) an estimator that leverages graduated non-convexity to compute the optimal estimate with high probability of success; and (iii) a certifier that employs sparse semidefinite programming (SDP) relaxation and a novel SDP solver to endow the estimator with an optimality certificate or escape local minima otherwise. I showcase certifiable outlier-robust perception on real robotic applications such as scan matching, satellite pose estimation, and vehicle pose and shape estimation.

Bio: Heng Yang is a Ph.D. candidate in the Department of Mechanical Engineering and the Laboratory for Information & Decision Systems at the Massachusetts Institute of Technology, working with Prof. Luca Carlone. His research interests include large-scale convex optimization, semidefinite relaxation, robust estimation, and machine learning, applied to robotics and trustworthy autonomy. His work includes developing certifiable outlier-robust machine perception algorithms, large-scale semidefinite programming solvers, and self-supervised geometric perception frameworks. Heng Yang is a recipient of the Best Paper Award in Robot Vision at the 2020 IEEE International Conference on Robotics and Automation (ICRA), a Best Paper Award Honorable Mention from the 2020 IEEE Robotics and Automation Letters (RA-L), and a Best Paper Award Finalist at the 2021 Robotics: Science and Systems (RSS) conference. He is a Class of 2021 RSS Pioneer.

 

Formalizing the Structure of Multiagent Domains for Autonomous Robot Navigation in Human Spaces

Christoforos Mavrogiannis, University of Washington

9/9/2021

Location: 122 Gates Hall

Time: 2:40p.m.

Abstract: Pedestrian scenes pose great challenges for robots due to the lack of formal rules regulating traffic, the lack of explicit coordination among agents, and the high dimensionality of the underlying space of outcomes. However, humans navigate with ease and comfort through a variety of complex multiagent environments, such as busy train stations, crowded malls or academic buildings. Human effectiveness in such domains can be largely attributed to cooperation, which introduces structure to multiagent behavior. In this talk, I will discuss how we can formalize this structure through the use of representations from low-dimensional topology. I will describe how these representations can be used to build prediction and planning algorithms for socially compliant robot navigation in pedestrian domains and show how their machinery may transfer to additional challenging environments such as uncontrolled street intersections.

Bio: Christoforos (Chris) Mavrogiannis is a postdoctoral research associate in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, working with Prof. Siddhartha Srinivasa. His interests lie at the intersection of motion planning, multiagent systems, and human-robot interaction. He is particularly interested in the design and evaluation of algorithms for multiagent domains in human environments. To this end, he employs tools from motion planning and machine learning, and often seeks insights from (algebraic) topology and social sciences. Chris has been a best-paper award finalist at the ACM/IEEE International Conference on Human-Robot Interaction (HRI), and selected as a Pioneer at the HRI and RSS conferences. He has also led open-source initiatives (Openbionics, MuSHR), for which he has been a finalist for the Hackaday Prize and a winner of the Robotdalen International Innovation Award. Chris holds M.S. and Ph.D. degrees from Cornell University, and a Diploma in mechanical engineering from the National Technical University of Athens.

Welcome to the Fall 2021 Robotics Seminar!

Tapomayukh Bhattacharjee and Claire Liang

9/2/2021

Location: 122 Gates Hall

Time: 2:40p.m.

Hey everyone! Welcome back for the semester. Robotics seminar is starting a new era and is (officially) a class again. The first seminar will cover the logistics of what to expect from this semester’s seminar/class as well as serve as an introduction to Cornell Robotics as a community. We will be announcing some new resources available (such as the new Robot Library) and taking feedback for what everyone would like to see in the future. The Robotics Graduate Student Organization will also cover some of what is to come for graduate students. If you’re new to the Cornell Robotics community, be sure to come for this week’s seminar!

P.S. Unfortunately, since Cornell is at a yellow COVID level, we will not have snacks for the foreseeable future.

Building Caregiving Robots

Tapomayukh Bhattacharjee, Cornell University

5/20/2021

Location: Zoom

Time: 2:40p.m.

Abstract:How do we build robots that can assist people with mobility limitations with activities of daily living? To successfully perform these activities, a robot needs to be able to physically interact with humans and objects in unstructured human environments. In the first part of my talk, I will show how a robot can use multimodal sensing to infer properties of these physical interactions using data-driven methods and physics-based models. In the second part of the talk, I will show how a robot can leverage these properties to feed people with mobility limitations. Successful robot-assisted feeding depends on reliable bite acquisition of hard-to-model deformable food items and easy bite transfer. Using insights from human studies, I will showcase algorithms and technologies that leverage multiple sensing modalities to perceive varied food item properties and determine successful strategies for bite acquisition and transfer. Using feedback from all the stakeholders, I will show how we built an autonomous robot-assisted feeding system that uses these algorithms and technologies and deployed it in the real world that fed real users with mobility limitations. I will conclude the talk with some ideas for future work in my new lab at Cornell.

A dirty laundry list for paper writing

Kirstin Petersen, Cornell University

5/6/2021

Location: Zoom

Time: 2:40p.m.

Abstract: This is meant as a highly interactive discussion on how to write and review research papers. The “to dos” and, perhaps more importantly “not to dos”.

Learning through Interaction in Cooperative Multi-Agent Systems

Kalesha Bullard, Facebook AI Research

4/29/2021

Location: Zoom

Time: 2:40p.m.

Abstract: Effective communication is an important skill for enabling information exchange and cooperation in multi-agent systems, in which agents coexist in shared environments with humans and/or other artificial agents.  Indeed, human domain experts can be a highly informative source of instructive guidance and feedback (supervision).  My prior work explores this type of interaction in depth, as a mechanism for enabling learning for artificial agents.  However, dependence upon human partners for acquiring or adapting skills has important limitations.  Human time and cognitive load is typically constrained (particularly in realistic settings) and data collection from humans, though potentially qualitatively rich, can be slow and costly to acquire. Yet, the ability to learn through interaction with other agents represents another powerful mechanism for enabling interactive learning.  Though other artificial agents may also be novices, agents can co-learn through providing each other evaluative feedback (reinforcement), given the learning task has been sufficiently structured and allows for generalization to novel settings.

This talk presents research that investigates methods for enabling agents to learn general communication skills through interactions with other agents.  In particular, the talk will focus on my ongoing work within Multi-Agent Reinforcement Learning, investigating emergent communication protocols, inspired by communication in more realistic settings.  We present a novel problem setting and a general approach that allows for zero-shot coordination (ZSC), i.e., discovering protocols that can generalize to independently trained agents.  We also explore and analyze specific difficulties associated with finding globally optimal ZSC protocols, as complexity of the communication task increases or the modality for communication changes (e.g. from symbolic communication to implicit communication through physical movement, by an embodied artificial agent).  Overall, this work opens up exciting avenues for learning general communication protocols in complex domains.

Physically Interactive Intelligence — A path towards autonomous embodied agents

Roberto Martin-Martin, Stanford University

4/22/2021

Location: Zoom

Time: 2:40p.m.

Abstract: What is the role of physical interaction in embodied intelligence? In robotics, physical interaction is often reduced to a minimum because it is considered difficult to plan, control and execute, has unpredictable effects and may be dangerous for the robot and anything or anyone around it. To compensate, we impose extremely high requirements on computation: perception, planning and control. However, when observing humans, we see that our autonomy to perform tasks in a versatile and robust manner come from rich, continuous and resourceful interactions with the environment, what I call Physically Interactive Intelligence.

In my research, I develop new learning algorithms to enable embodied AI agents to exploit interactions to gain autonomy, and I test them in realistic integrated robotic systems. I propose to promote physical interaction to foundational component of novel robotic solutions. I will present new methods to learn to control and exploit physical interactions even for tasks where they are not traditionally used such as perception and navigation. These lines of work support my overall research hypothesis: autonomous behavior and grounded understanding in embodied AI agents are achieved through the resourceful use of physical interaction with the environment, i.e. through physically interactive intelligence.

From Semantics to Localization in LiDAR Maps for Autonomous Vehicles

Abhinav Valada, University of Freiburg

4/15/2021

Location: Zoom

Time: 2:40p.m.

Abstract: LiDAR-based scene interpretation and localization play a critical role in enabling autonomous vehicles to safely navigate in the environment. The last decade has witnessed unprecedented progress in these tasks by exploiting learning techniques to improve the performance and robustness. Despite these advances, the unordered spare, and irregular structure of point clouds pose several unique challenges that lead to suboptimal performance while employing standard convolutional neural networks (CNNs). In this talk, I will discuss three efforts targeted at addressing some of these challenges. First, I will present our state-of-the-art approach to LiDAR panoptic segmentation that employs a 2D CNN while explicitly leveraging the unique 3D information provided by point clouds at multiple stages in the network. I will then present our recent work that incorporates a differentiable unbalanced optimal transport algorithm to detect loop closures in LiDAR point clouds and outperforms both existing learning-based as well as hardcrafted methods. Next, to alleviate the need for expensive LiDAR sensors on every robot, I will present the first approach for monocular camera localization in LiDAR maps that effectively generalizes to new environments without any retraining and independent of the camera parameters. Finally, I will conclude the talk with a discussion on opportunities for further scaling up the learning of these tasks.

Tracking Beyond Recognition

Aljosa Osep, Technical University in Munich

4/8/2021

Location: Zoom

Time: 2:40p.m.

Abstract: Spatio-temporal interpretation of raw sensory data is vital for intelligent agents to understand how to interact with the environment and perceive how trajectories of moving agents evolve in the 4D continuum, i.e., 3D space and time. To this end, I will first talk about our recent efforts in the semantic and temporal understanding of raw sensory data. I will first present our work on multi-object and segmentation. Then, I will discuss how to generalize these ideas towards holistic temporal scene understanding, jointly tackling object instance segmentation, tracking, and semantic understanding of monocular video sequences and LiDAR streams. Finally, I will move on to the challenging problem of scaling object instance segmentation and tracking models to the open world, in which future mobile agents will need to continuously learn without explicit human supervision. In such scenarios, intelligent agents encounter and need to react to unknown dynamic objects that were not observed during the model training.