RSS 2021 Papers

Recent work on decision-making and planning for autonomous driving has made use of game theoretic methods to model interaction between agents. We demonstrate that methods based on the Stackelberg game formulation of this problem are susceptible to an issue that we refer to as Conflict. Our results show that when Conflict occurs; it can cause sub-optimal and potentially dangerous behaviour. In response; we develop a theoretical framework for analysing the extent to which such methods are impacted by Conflict; and apply this framework to several existing approaches modelling interaction between agents. Moreover; we propose Augmented Altruism; a novel approach to modelling interaction between players in a Stackelberg game; and show that it is less prone to Conflict than previous techniques. Finally; we investigate the behavioural assumptions that underpin our approach by performing experiments with human participants. The results show that our model approximates human decision-making more accurately than existing game-theoretic models of interactive driving.

We propose and study a class of multi-object rearrangement problems in which a robotic manipulator; capable of carrying an item and making item swaps; is tasked to sort items stored in lattices in a time-optimal manner. We systematically analyze the intrinsic optimality structure; which is fairly rich and intriguing; under different levels of item distinguishability (fully labeled; where each item has a unique label; or partially labeled; where multiple items may be of the same type) and different lattice dimensions. Focusing on the most practical setting of one and two dimensions; we develop efficient (low polynomial time) algorithms that optimally perform rearrangements on 1D lattices under both fully- and partially-labeled settings. On the other hand; we prove that rearrangement on 2D and higher dimensional lattices becomes computationally intractable to optimally solve. Despite their NP-hardness; we are able to again develop efficient algorithms for 2D fully- and partially-labeled settings that are asymptotically optimal; in expectation; assuming that the initial configuration is randomly selected. Simulation studies confirm the effectiveness of our algorithms in comparison to greedy best-first approaches. Source code of Python implementation:

We present a novel method for estimation of 3D human poses from a multi-camera setup; employing distributed smart edge sensors coupled with a backend through a semantic feedback loop. 2D joint detection for each camera view is performed locally on a dedicated embedded inference processor. Only the semantic skeleton representation is transmitted over the network and raw images remain on the sensor board. 3D poses are recovered from 2D joints on a central backend; based on triangulation and a body model which incorporates prior knowledge of the human skeleton. A feedback channel from backend to individual sensors is implemented on a semantic level. The allocentric 3D pose is backprojected into the sensor views where it is fused with 2D joint detections. The local semantic model on each sensor can thus be improved by incorporating global context information. The whole pipeline is capable of real-time operation. We evaluate our method on three public datasets; where we achieve state-of-the-art results and show the benefits of our feedback architecture; as well as in our own setup for multi-person experiments. Using the feedback signal improves the 2D joint detections and in turn the estimated 3D poses.

This paper presents a radar odometry method that combines probabilistic trajectory estimation and deep learned features without needing groundtruth pose information. The feature network is trained unsupervised; using only the on-board radar data. With its theoretical foundation based on a data likelihood objective; our method leverages a deep network for processing rich radar data; and a non-differentiable classic estimator for probabilistic inference. We provide extensive experimental results on both the publicly available Oxford Radar RobotCar Dataset and an additional 100 km of driving collected in an urban setting. Our sliding-window implementation of radar odometry outperforms most hand-crafted methods and approaches the current state of the art without requiring a groundtruth trajectory for training. We also demonstrate the effectiveness of radar odometry under adverse weather conditions. Code for this project can be found at:

We consider the problem of learning motion policies for acceleration-based robotics systems with a structured policy class. We leverage a multi-task control framework called RMPflow which has been successfully applied in many robotics problems. Using RMPflow as a structured policy class in learning has several benefits; such as sufficient expressiveness; the flexibility to inject different levels of prior knowledge as well as the ability to transfer policies between robots. However; implementing a system for end-to-end learning of RMPflow policies faces several computational challenges. In this work; we re-examine the RMPflow algorithm and propose a more practical alternative; called RMP2; that uses modern automatic differentiation tools (such as TensorFlow and PyTorch) to compute RMPflow policies. Our new design retains the strengths of RMPflow while bringing in advantages from automatic differentiation; including 1) simple programming interfaces to designing complex transformations; 2) support of general directed acyclic graph (DAG) transformation structures; 3) end-to-end differentiability for policy learning; 4) improved computational efficiency. Because of these features; RMP2 can be treated as a structured policy class for efficient robot learning that is suitable for encoding domain knowledge. Our experiments show that using the structured policy class given by RMP2 can improve policy performance and safety in reinforcement learning tasks for goal reaching in cluttered space. The video for our experimental results can be found at

Successful real-world deployment of legged robots would require them to

Control of robots with kinematic constraints like loop-closure constraints or interactions with the environment requires solving the underlying constrained dynamics equations of motion. Several approaches have been proposed so far in the literature to solve these constrained optimization problems; for instance by either taking advantage in part of the sparsity of the kinematic tree or by considering an explicit formulation of the constraints in the problem resolution. Yet; not all the constraints allow an explicit formulation and in general; approaches of the state of the art suffer from singularity issues; especially in the context of redundant or singular constraints. In this paper; we propose a unified approach to solve forward dynamics equations involving constraints in an efficient; generic and robust manner. To this aim; we first (i) propose a proximal formulation of the constrained dynamics which converges to an optimal solution in the least-square sense even in the presence of singularities.Based on this proximal formulation; we introduce (ii) a sparse Cholesky factorization of the underlying Karush–Kuhn–Tucker matrix related to the constrained dynamics; which exploits at best the sparsity of the kinematic structure of the robot. We also show (iii) that it is possible to extract from this factorization the Cholesky decomposition associated to the so-called Operational Space Inertia Matrix; inherent to task-based control frameworks or physic simulations. These new formulation and factorization;implemented within the Pinocchio library; are benchmark on various robotic platforms; ranging from classic robotic arms or quadrupeds to humanoid robots with closed kinematic chains; and show how they significantly outperform alternative solutions of the state of the art by a factor 2 or more.

Ensuring human safety without unnecessarily impacting task efficiency during human-robot interactive manipulation tasks is a critical challenge. In this work; we formally define human physical safety as collision avoidance or safe impact in the event of a collision. We developed a motion planner that theoretically guarantees safety; with a high probability; under the uncertainty in human dynamic models. Our two-pronged definition of safety is able to unlock the planner's potential in finding efficient plans even when collision avoidance is nearly impossible. The improved efficiency is empirically demonstrated in both a simulated goal-reaching domain and a real-world robot-assisted dressing domain. We provide a unified view of two approaches to safe human-robot interaction: human-aware motion planners that use predictive human models and reactive controllers that compliantly handle collisions.

In this paper; our aim is to highlight Tactile Perceptual Aliasing as a problem when using deep neural networks and other discriminative models. Perceptual aliasing will arise wherever a physical variable extracted from tactile data is subject to ambiguity between stimuli that are physically distinct. Here we address this problem using a probabilistic discriminative model implemented as a 5-component mixture density network comprised of a deep neural network that predicts the parameters of a Gaussian mixture model. We show that discriminative regression models such as deep neural networks and Gaussian process regression perform poorly on aliased data; with accurate predictions only when the sources of aliasing are removed. In contrast; the mixture density network identifies aliased data with improved prediction accuracy. The uncertain predictions of the model form patterns that are consistent with the various sources of perceptual ambiguity. In our view; perceptual aliasing will become an unavoidable issue for robot touch as the field progresses to training robots that act in uncertain and unstructured environments; such as with deep reinforcement learning.

Many robotic applications involve interactions between multiple agents where an agent's decisions affect the behavior of other agents. Such behaviors can be captured by the equilibria of differential games which provide an expressive framework for modeling the agents' mutual influence. However; finding the equilibria of differential games is in general challenging as it involves solving a set of coupled optimal control problems. In this work; we propose to leverage the special structure of multi-agent interactions to generate interactive trajectories by simply solving a single optimal control problem; namely; the optimal control problem associated with minimizing the potential function of the differential game. Our key insight is that for a certain class of multi-agent interactions; the underlying differential game is indeed a potential differential game for which equilibria can be found by solving a single optimal control problem. We introduce such an optimal control problem and build on single-agent trajectory optimization methods to develop a computationally tractable and scalable algorithm for planning multi-agent interactive trajectories. We will demonstrate the performance of our algorithm in simulation and show that our algorithm outperforms the state-of-the-art game solvers. To further show the real-time capabilities of our algorithm; we will demonstrate the application of our proposed algorithm in a set of experiments involving interactive trajectories for two quadcopters.

The ability to transfer a policy from one environment to another is a promising avenue for efficient robot learning in realistic settings where task supervision is not available. This can allow us to take advantage of environments well suited for training; such as simulators or laboratories; to learn a policy for a real robot in a home or office. To succeed; such policy transfer must overcome both the visual domain gap (e.g. different illumination or background) and the dynamics domain gap (e.g. different robot calibration or modelling error) between source and target environments. However; prior policy transfer approaches either cannot handle a large domain gap or can only address one type of domain gap at a time. In this paper; we propose a novel policy transfer method with iterative "environment grounding"; IDAPT; that alternates between (1) directly minimizing both visual and dynamics domain gaps by grounding the source environment in the target environment domains; and (2) training a policy on the grounded source environment. This iterative training progressively aligns the domains between the two environments and adapts the policy to the target environment. Once trained; the policy can be directly executed on the target environment. The empirical results on locomotion and robotic manipulation tasks demonstrate that our approach can effectively transfer a policy across visual and dynamics domain gaps with minimal supervision and interaction with the target environment. Videos and code are available at

Robots frequently need to perceive object attributes; such as "red;" "heavy;" and "empty;" using multimodal exploratory actions; such as "look;" "lift;" and "shake." Robot attribute learning algorithms aim to learn an observation model for each perceivable attribute given an exploratory action. Once the attribute models are learned; they can be used to identify attributes of new objects; answering questions; such as "Is this object red and empty?" Attribute learning and identification are being treated as two separate problems in the literature. In this paper; we first define a new problem called online robot attribute learning (On-RAL); where the robot works on attribute learning and attribute identification simultaneously. Then we develop an algorithm called information-theoretic reward shaping (ITRS) that actively addresses the trade-off between exploration and exploitation in On-RAL problems. ITRS was compared with competitive robot attribute learning baselines; and experimental results demonstrate ITRS' superiority in learning efficiency and identification accuracy.

We present a novel approach to motion planning for autonomous ground vehicles by formulating motion primitives as probabilistic distributions of trajectories (aka probabilistic motion primitives - ProMP) and performing stochastic optimisation on them for finding an optimal path. We show that compared to the traditional approach of using discrete motion primitives or direct stochastic optimisation of the whole path; incorporating ProMPs enables higher quality of paths by enabling constraint conditioning; combination and blending of probability distributions. We present two motion planners utilizing this approach: feasibility-based trajectory sampling (PROMPT-S) and stochastic gradient-based trajectory optimisation (PROMPT-O). We show simulation results of our approach outperforming state-of-the-art optimisation as well as discrete motion primitives-based planners. We additionally illustrate the versatility of our approach by showing PROMPT's ability to handle significantly skewed motion primitives; e.g; as induced by steering failure in AGVs as well as the composition of motion primitives to perform complex maneuvers. Finally; we demonstrate the practicality of these planners by implementing them on a real self-driving vehicle navigating on structured and unstructured off-road terrains.

We consider a category-level perception problem; where one is given 3D sensor data picturing an object of a given category (e.g.; a car); and has to reconstruct the pose and shape of the object despite intra-class variability (i.e.; different car models have different shapes). We consider an active shape model; where—for an object category— we are given a library of potential CAD models describing objects in that category; and we adopt a standard formulation where pose and shape estimation are formulated as a non-convex optimization. Our first contribution is to provide the first certifiably optimal solver for pose and shape estimation. In particular; we show that rotation estimation can be decoupled from the estimation of the object translation and shape; and we demonstrate that (i) the optimal object rotation can be computed via a tight (small-size) semidefinite relaxation; and (ii) the translation and shape parameters can be computed in closed-form given the rotation. Our second contribution is to add an outlier rejection layer to our solver; hence making it robust to a large number of misdetections. Towards this goal; we wrap our optimal solver in a robust estimation scheme based on graduated non-convexity. To further enhance robustness to outliers; we also develop the first graph-theoretic formulation to prune outliers in category-level perception; which removes outliers via convex hull and maximum clique computations; the resulting approach is robust to 70 − 90% outliers. Our third contribution is an extensive experimental evaluation. Besides providing an ablation study on a simulated dataset and on the PASCAL3D+ dataset; we combine our solver with a deep-learned keypoint detector; and show that the resulting approach improves over the state of the art in vehicle pose estimation in the ApolloScape driving datasets.

When studying robots collaborating with humans; much of the focus has been on robot policies that coordinate fluently with human teammates in collaborative tasks. However; less emphasis has been placed on the effect of the environment on coordination behaviors. To thoroughly explore environments that result in diverse behaviors; we propose a framework for procedural generation of environments that are (1) stylistically similar to human-authored environments; (2) guaranteed to be solvable by the human-robot team; and (3) diverse with respect to coordination measures. We analyze the procedurally generated environments in the Overcooked benchmark domain via simulation and an online user study. Results show that the environments result in qualitatively different emerging behaviors and statistically significant differences in collaborative fluency metrics; even when the robot runs the same planning algorithm.

High capacity end-to-end approaches for human motion (behavior) prediction have the ability to represent subtle nuances in human behavior; but struggle with robustness to out of distribution inputs and tail events. Planning-based prediction; on the other hand; can reliably output decent-but-not-great predictions: it is much more stable in the face of distribution shift (as we verify in this work); but it has high inductive bias; missing important aspects that drive human decisions; and ignoring cognitive biases that make human behavior suboptimal. In this work; we analyze one family of approaches that strive to get the best of both worlds: use the end-to-end predictor on common cases; but do not rely on it for tail events / out-of-distribution inputs — switch to the planning-based predictor there. We contribute an analysis of different approaches for detecting when to make this switch; using an autonomous driving domain. We find that promising approaches based on ensembling or generative modeling of the training distribution might not be reliable; but that there very simple methods which can perform surprisingly well — including training a classifier to pick up on tell-tale issues in predicted trajectories.

For tabletop rearrangement problems with overhand grasps; storage space outside the tabletop workspace; or buffers; can temporarily hold objects which greatly facilitates the resolution of a given rearrangement task. This brings forth the natural question of how many running buffers are required so that certain classes of tabletop rearrangement problems are feasible. In this work; we examine the problem for both the labeled (where each object has a specific goal pose) and the unlabeled (where goal poses of objects are interchangeable) settings. On the structural side; we observe that finding the minimum number of running buffers (MRB) can be carried out on a dependency graph abstracted from a problem instance; and show that computing MRB on dependency graphs is NP-hard. We then prove that under both labeled and unlabeled settings; even for uniform cylindrical objects; the number of required running buffers may grow unbounded as the number of objects to be rearranged increases; we further show that the bound for the unlabeled case is tight. On the algorithmic side; we develop highly effective algorithms for finding MRB for both labeled and unlabeled tabletop rearrangement problems; scalable to over a hundred objects under very high object density. Employing these algorithms; empirical evaluations show that random labeled and unlabeled instances; which more closely mimics real-world setups; have much smaller MRB.

Quadrotors are extremely agile; so much in fact; that classic first-principle-models come to their limits. Aerodynamic effects; while insignificant at low speeds; become the dominant model defect during high speeds or agile maneuvers. Accurate modeling is needed to design robust high-performance control systems and enable flying close to the platform's physical limits. We propose a hybrid approach fusing first principles and learning to model quadrotors and their aerodynamic effects with unprecedented accuracy. First principles fail to capture such aerodynamic effects; rendering traditional approaches inaccurate when used for simulation or controller tuning. Data-driven approaches try to capture aerodynamic effects with blackbox modeling; such as neural networks; however; they struggle to robustly generalize to arbitrary flight conditions. Our hybrid approach unifies and outperforms both first-principles blade-element momentum theory and learned residual dynamics. It is evaluated in one of the world's largest motion-capture systems; using autonomous-quadrotor-flight data at speeds up to 65 km/h. The resulting model captures the aerodynamic thrust; torques; and parasitic effects with astonishing accuracy; outperforming existing models with 50% reduced prediction errors; and shows strong generalization capabilities beyond the training set.

Robots will be expected to manipulate a wide variety of objects in complex and arbitrary ways as they become more widely used in human environments. As such; the rearrangement of objects has been noted to be an important benchmark for AI capabilities in recent years. We propose NeRP (Neural Rearrangement Planning); a deep learning based approach for multi-step neural object rearrangement planning which works with never-before-seen objects; that is trained on simulation data; and generalizes to the real world. We compare NeRP to several naive and model-based baselines; demonstrating that our approach is measurably better and can efficiently arrange unseen objects in fewer steps and with less planning time. Finally; we demonstrate it on several challenging rearrangement problems in the real world.