RSS 2021 Papers

Highly constrained manipulation tasks continue to be challenging for autonomous robots as they require high levels of precision; typically less than 1mm; which is often incompatible with what can be achieved by traditional perception systems. This paper demonstrates that the combination of state-of-the-art object tracking with passively adaptive mechanical hardware can be leveraged to complete precision manipulation tasks with tight; industrially-relevant tolerances (0.25mm). The proposed control method closes the loop through vision by tracking the relative 6D pose of objects in the relevant workspace. It adjusts the control reference of both the compliant manipulator and the hand to complete object insertion tasks via within-hand manipulation. Contrary to previous efforts for insertion; our method does not require expensive force sensors; precision manipulators; or time-consuming; online learning; which is data hungry. Instead; this effort leverages mechanical compliance and utilizes an object-agnostic manipulation model of the hand learned offline; off-the-shelf motion planning; and an RGBD-based object tracker trained solely with synthetic data. These features allow the proposed system to easily generalize and transfer to new tasks and environments. This paper describes in detail the system components and showcases its efficacy with extensive experiments involving tight tolerance peg-in-hole insertion tasks of various geometries as well as open-world constrained placement tasks.

Place recognition is the task of recognizing the current scene from a database of known places. The currently dominant algorithmic paradigm is to use (deep learning based) holistic feature vectors to describe each place and use fast vector query methods to find matchings. We propose a novel type of image descriptor; Vector Semantic Representations (VSR); that encodes the spatial semantic layout from a semantic segmentation together with appearance properties in a; for example; 4;096 dimensional vector for place recognition. We leverage operations from the established class of Vector Symbolic Architectures to combine symbolic (e.g. class label) and numeric (e.g. feature map response) information in a common vector representation. We evaluate the proposed semantic descriptor on 13 standard mobile robotic place recognition datasets and compare to six descriptors from the literature. VSR is on par with the best compared descriptor (NetVLAD) in terms of mean average precision and superior in terms of recall and worst-case average precision. This makes the approach particularly interesting for candidate selection. For a more detailed investigation; we discuss and evaluate recall integrity as additional criterion. Further; we demonstrate that the semantic descriptor is particularly well suited for combination with existing appearance descriptors indicating that semantics provide complementary information for image matching.

In this paper; we provide a generalized framework for Variational Inference-Stochastic Optimal Control by using the non-extensive Tsallis divergence. By incorporating the deformed exponential function into the optimality likelihood function; a novel Tsallis Variational Inference-Model Predictive Control algorithm is derived; which includes prior works such as Variational Inference-Model Predictive Control; Model Predictive Path Integral Control; Cross Entropy Method; and Stein Variational Inference Model Predictive Control as special cases. The proposed algorithm allows for effective control of the cost/reward transform and is characterized by superior performance in terms of mean and variance reduction of the associated cost. The aforementioned features are supported by a theoretical and numerical analysis on the level of risk sensitivity of the proposed algorithm as well as simulation experiments on 5 different robotic systems with 3 different policy parameterizations.

Robot manipulation for untangling 1D deformable structures such as ropes; cables; and wires is challenging due to their infinite dimensional configuration space; complex dynamics; and tendency to self-occlude. Analytical controllers often fail in the presence of dense configurations; due to the difficulty of grasping between adjacent cable segments. We present two algorithms that enhance robust cable untangling; LOKI and SPiDERMan; which operate alongside HULK; a high-level planner from prior work. LOKI uses a learned model of manipulation features to refine a coarse grasp keypoint prediction to a precise; optimized location and orientation; while SPiDERMan uses a learned model to sense task progress and apply recovery actions. We evaluate these algorithms in physical cable untangling experiments with 336 knots and over 1500 actions on real cables using the da Vinci surgical robot. We find that the combination of HULK; LOKI; and SPiDERMan is able to untangle dense overhand; figure-eight; double-overhand; square; bowline; granny; stevedore; and triple-overhand knots. The composition of these methods successfully untangles a cable from a dense initial configuration in 68.3% of 60 physical experiments and achieves 50% higher success rates than baselines from prior work. Supplementary material; code; and videos can be found at

In this work; we analyze and present an algorithm to find shortest-paths for generic rigid bodies. We derived the necessary conditions for optimality using Lagrange multipliers; and compared it to the conditions derived from Pontraygin's Maximum Principle. We derived the equations of the necessary conditions using geometric Jacobian; drawing inspiration from the similarity between the rigid-body systems and the arm-like systems. In the previous work, the analysis focused on finding shortest-paths to reach goals in positions only. This work extends the analysis to find the shortest-path to reach a goal with complete configuration in 3D. We show that the algorithm is resolution complete even when the orientations are included. To overcome the complexity of 3D orientations; we describe the system using three points in the robot frame; and show that this parameter system is redundant but can derive the same necessary conditions as those derived using the minimum parameters (configuration). We used a 3D Dubins system to demonstrate the correctness of the analysis and the algorithm.

Medical steerable needles can move along 3D curvilinear trajectories to avoid anatomical obstacles and reach clinically significant targets inside the human body. Automating steerable needle procedures can enable physicians and patients to harness the full potential of steerable needles by maximally leveraging their steerability to safely and accurately reach targets for medical procedures such as biopsies and localized therapy delivery for cancer. For the automation of medical procedures to be clinically accepted; it is critical from a patient care; safety; and regulatory perspective to certify the correctness and effectiveness of the motion planning algorithms involved in procedure automation. In this paper; we take an important step toward creating a certifiable motion planner for steerable needles. We introduce the first motion planner for steerable needles that offers a guarantee; under clinically appropriate assumptions; that it will; in finite time; compute an exact; obstacle-avoiding motion plan to a specified target; or notify the user that no such plan exists. We present an efficient; resolution-complete motion planner for steerable needles based on a novel adaptation of multi-resolution planning. Compared to state-of-the-art steerable needle motion planners (none of which provide any completeness guarantees); we demonstrate that our new resolution-complete motion planner computes plans faster and with a higher success rate.

We present a method for autonomous exploration in complex three-dimensional (3D) environments. Our method demonstrates exploration faster than the current state-of-the-art using a hierarchical framework — one level maintains data densely and computes a detailed path within a local planning horizon; while another level maintains data sparsely and computes a coarse path at the global scale. Such a framework shares the insight that detailed processing is most effective close to the robot; and gains computational speed by trading-off computation of details far away from the robot. The method optimizes an overall exploration path with respect to the length of the path. In addition; the path in the local area is kinodynamically feasible for the vehicle to follow at a high speed. In experiments; our systems autonomously explore indoor and outdoor environments at a high degree of complexity; with ground and aerial robots. The method produces 80% more exploration efficiency; defined as the average explored volume per second through a run; and consumes less than 50% of computation compared to the state-of-the-art.

Grasp detection in clutter requires the robot to reason about the 3D scene from incomplete and noisy perception. In this work; we draw insight that 3D reconstruction and grasp learning are two intimately connected tasks; both of which require a fine-grained understanding of local geometry details. We thus propose to utilize the synergies between grasp affordance and 3D reconstruction through multi-task learning of a shared representation. Our model takes advantage of deep implicit functions; a continuous and memory-efficient representation; to enable differentiable training of both tasks. We train the model on self-supervised grasp trials data in simulation. Evaluation is conducted on a clutter removal task; where the robot clears cluttered objects by grasping them one at a time. The experimental results in simulation and on the real robot have demonstrated that the use of implicit neural representations and joint learning of grasp affordance and 3D reconstruction have led to state-of-the-art grasping results. Our method outperforms baselines by over 10% in terms of grasp success rate.

We present in-hand manipulation skills on a dexterous; compliant; anthropomorphic hand. Even though these skills were derived in a simplistic manner; they exhibit surprising robustness to variations in shape; size; weight; and placement of the manipulated object. They are also very insensitive to variation of execution speeds; ranging from highly dynamic to quasi-static. The robustness of the skills leads to compositional properties that enable extended and robust manipulation programs. To explain the surprising robustness of the in-hand manipulation skills; we performed a detailed; empirical analysis of the skills' performance. From this analysis; we identify three principles for skill design: 1) Exploiting the hardware's innate ability to drive hard-to-model contact dynamics. 2) Taking actions to constrain these interactions; funneling the system into a narrow set of possibilities. 3) Composing such action sequences into complex manipulation programs. We believe that these principles constitute an important foundation for robust robotic in-hand manipulation; and possibly for manipulation in general.

Performing a number of motion patterns — referred to as skills — (e.g.; wave; spiral; sweeping motions) during teleoperation is an integral part of many industrial processes such as spraying; welding; and wiping (cleaning; polishing). Maintaining these motions whilst simultaneously avoiding obstacles and traversing complex terrain requires expert operators. In this work; we propose a novel skill-based shared control framework for incorporating the notion of skill assistance to aid novice operators to sustain these motion patterns whilst adhering to environmental constraints. Our shared control method uses streaming joystick data to estimate the model parameters that provide a description of the operator's intention. We introduce a novel parametrization for state and control that combines skill and underlying trajectory models; leveraging a special type of curve known as Clothoids. This new parameterization allows for efficient computation of skill-based short term horizon plans; enabling the use of a Model Predictive Control (MPC) loop. We perform experiments on a hardware mock-up; validating the effectiveness of our method to recognize a switch of intended skill; and showing an improved quality of output motion; even under dynamically changing obstacles. See our accompanying video here:

Complex mission specifications can be often specified through temporal logics; such as Linear Temporal Logic and its syntactically co-safe fragment; scLTL. Finding trajectories that satisfy such specifications becomes hard if the robot is to fulfil the mission in an initially unknown environment; where neither locations of regions or objects of interest in the environment nor the obstacle space are known a priori. We propose an algorithm that; while exploring the environment; learns important semantic dependencies in the form of a semantic abstraction; and uses it to bias the growth of an Rapidly-exploring random graph towards faster mission completion. Our approach leads to finding trajectories that are much shorter than those found by the sequential approach; which first explores and then plans. Simulations comparing our solution to the sequential approach; carried out in 100 randomized office-like environments; show more than 50% reduction in the trajectory length.

Robots deployed at orders of magnitude different size scales; and that retain the same desired behavior at any of those scales; would greatly expand the environments in which the robots could operate. However it is currently not known whether such robots exist; and; if they do; how to design them. Since self similar structures in nature often exhibit self similar behavior at different scales; we hypothesize that there may exist robot designs that have the same property. Here we demonstrate that this is indeed the case for some; but not all; modular soft robots: there are robot designs that exhibit a desired behavior at a small size scale; and if copies of that robot are attached together to realize the same design at higher scales; those larger robots exhibit similar behavior. We show how to find such designs in simulation using an evolutionary algorithm. Further; when fractal attachment is not assumed and attachment geometries must thus be evolved along with the design of the base robot unit; scale invariant behavior is not achieved; demonstrating that structural self similarity; when combined with appropriate designs; is a useful path to realizing scale invariant robot behavior. We validate our findings by demonstrating successful transferal of self similar structure and behavior to pneumatically-controlled soft robots. Finally; we show that biobots can spontaneously exhibit self similar attachment geometries; thereby suggesting that self similar behavior via self similar structure may be realizable across a wide range of robot platforms in future.

We address the problem of planning robot motions in constrained configuration spaces where the constraints change throughout the motion. The problem is formulated as a fixed sequence of intersecting manifolds; which the robot needs to traverse in order to solve the task. We specify a class of sequential motion planning problems that fulfill a particular property of the change in the free configuration space when transitioning between manifolds. For this problem class; the algorithm Planning on Sequenced Manifolds (PSM*) is developed which searches for optimal intersection points between manifolds by using RRT* in an inner loop with a novel steering strategy. We provide a theoretical analysis regarding PSM*s probabilistic completeness and asymptotic optimality. Further; we evaluate its planning performance on multi-robot object transportation tasks.

Reach-avoid optimal control problems; in which the system must reach certain goal conditions while staying clear of unacceptable failure modes; are central to safety and liveness assurance for autonomous robotic systems; but their exact solutions are intractable for complex dynamics and environments. Recent successes in the use of reinforcement learning methods to approximately solve optimal control problems with performance objectives make their application to certification problems attractive; however; the Lagrange-type objective (cumulative costs or rewards over time) used in reinforcement learning is not suitable to encode temporal logic requirements. Recent work has shown promise in extending the reinforcement learning machinery to safety-type problems; whose objective is not a sum but a minimum over time. In this work; we generalize the reinforcement learning formulation to handle all optimal control problems in the reach-avoid category. We derive a time-discounted reach-avoid Bellman backup with contraction mapping properties and prove that the resulting reach-avoid Q-learning algorithm converges under analogous conditions to the traditional Lagrange-type problem; yielding an arbitrarily tight conservative approximation to the reach-avoid set. We further demonstrate the use of this formulation with deep reinforcement learning methods; by treating their approximate solutions as untrusted oracles in a supervisory control framework. We evaluate our proposed framework on a range of nonlinear systems; validating the results against analytic and numerical solutions; and through Monte Carlo simulation in previously intractable problems. Our results open the door to a range of learning-based methods for safe-and-live autonomous behavior; with applications across robotics and automation.

Reinforcement learning is a promising approach to solving hard robotics tasks. An important challenge is ensuring safety—e.g.; that a walking robot does not fall over or an autonomous car does not crash into an obstacle. We build on an approach that composes the learned policy with a backup policy—it uses the learned policy on the interior of the region where the backup policy is guaranteed to be safe; and switches to the backup policy on the boundary of this region. The key challenge is checking when the backup policy is guaranteed to be safe. Our algorithm; statistical model predictive shielding (SMPS); uses sampling-based verification and linear systems analysis to perform this check. We prove that SMPS ensures safety with high probability; and empirically evaluate its performance on several benchmarks.

We study planning problems where a controllable agent operates under partial observability and interacts with an uncontrollable opponent; also referred to as the adversary. The agent has two distinct objectives: To maximize an expected value and to adhere to a safety specification. Multi-objective partially observable stochastic games (POSGs) formally model such problems. Yet; even for a single objective; the task of computing suitable policies for POSGs is theoretically hard and computationally intractable in practice. Using a factored state-space representation; we define a decoupling scheme for the POSG state space that—under certain assumptions on the observability and the reward structure—separates the state components relevant for the reward from those relevant for safety. This decoupling affects the possibility to compute provably safe and reward-optimal policies in a tractable two-stage approach. In particular; on the fully observable components related to safety; we exactly compute the set of policies that captures all possible safe choices against the opponent. We restrict the agent's behavior to these safe policies and project the POSG to a partially observable Markov decision process (POMDP). Any reward-maximal policy for the POMDP is then guaranteed to be safe and reward-maximal for the POSG. We showcase our approach's feasibility using high-fidelity simulations of two case studies that concern UAV path planning and autonomous driving. Moreover; to demonstrate the practical applicability; we design a physical experiment involving a robot decision making problem under energy constraints that is motivated by a paired helicopter with NASA's Perseverance Mars rover.

Autonomous vehicles interacting with other traffic participants heavily rely on the perception and prediction of other agents' behaviors to plan safe trajectories. However; as occlusions limit the vehicle's perception ability; reasoning about potential hazards beyond the field-of-view is one of the most challenging issues in developing autonomous driving systems. This paper introduces a novel analytical approach that poses the problem of safe trajectory planning under occlusions as a hybrid zero-sum dynamic game between the autonomous vehicle (evader); and an initially hidden traffic participant (pursuer). Due to occlusions; the pursuer's state is initially unknown to the evader and may later be discovered by the vehicle's sensors. The analysis yields optimal strategies for both players as well as the set of initial conditions from which the autonomous vehicle is guaranteed to avoid collisions. We leverage this theoretical result to develop a novel trajectory planning framework that provides worst-case safety guarantees while minimizing conservativeness by accounting for the autonomous vehicle's ability to actively avoid other road users as soon as they are detected in future observations. Our framework is agnostic to the driving environment and suitable for various motion planners. We demonstrate our algorithm on challenging urban and highway driving scenarios using the open-source CARLA simulator.

Although ground robotic autonomy has gained widespread usage in structured and controlled environments; autonomy in unknown and off-road terrain remains a difficult problem. Extreme; off-road; and unstructured environments such as undeveloped wilderness; caves; and rubble pose unique and challenging problems for autonomous navigation. To tackle these problems we propose an approach for assessing traversability and planning a safe; feasible; and fast trajectory in real-time. Our approach; which we name STEP (Stochastic Traversability Evaluation and Planning); relies on: 1) rapid uncertainty-aware mapping and traversability evaluation; 2) tail risk assessment using the Conditional Value-at-Risk (CVaR); and 3) efficient risk and constraint-aware kinodynamic motion planning using sequential quadratic programming-based (SQP) model predictive control (MPC). We analyze our method in simulation and validate its efficacy on wheeled and legged robotic platforms exploring extreme terrains including an abandoned subway and an underground lava tube.

When transferring a control policy from simulation to a physical system; this policy needs to be robust to variations in the dynamics to perform well. Commonly; the optimal policy overfits to the approximate model and the corresponding state-distribution. Therefore; the policy fails when transferred to the physical system. In this paper; we are presenting robust value iteration. This approach uses dynamic programming to compute the optimal value function on the compact state domain and incorporates adversarial perturbations of the system dynamics. The adversarial perturbations cause the resulting optimal policy to be robust to changes in the dynamics. Utilizing the continuous time perspective of reinforcement learning; we derive the optimal perturbations for the states; actions; observations and model parameters in closed-form. The resulting algorithm does not require discretization of states or actions. Therefore; the optimal adversarial perturbations can be efficiently incorporated in the min-max value function update. We apply the resulting algorithm to the physical Furuta Pendulum and cartpole. By changing the masses of the systems we evaluate the quantitative and qualitative performance across different model parameters. We show that robust value iteration is more robust compared to deep reinforcement learning algorithm and the non-robust version of the algorithm.