ICRA 2020 Papers

Motivated by the success of encoding multi-scale contextual information for image analysis, we propose our PointAtrousGraph (PAG) - a deep permutation-invariant hierarchical encoder-decoder for efficiently exploiting multi-scale edge features in point clouds. Our PAG is constructed by several novel modules, such as Point Atrous Convolution (PAC), Edgepreserved Pooling (EP) and Edge-preserved Unpooling (EU). Similar with atrous convolution, our PAC can effectively enlarge receptive fields of filters and thus densely learn multi-scale point features. Following the idea of non-overlapping maxpooling operations, we propose our EP to preserve critical edge features during subsampling. Correspondingly, our EU modules gradually recover spatial information for edge features. In addition, we introduce chained skip subsampling/upsampling modules that directly propagate edge features to the final stage. Particularly, our proposed auxiliary loss functions can further improve our performance. Experimental results show that our PAG outperform previous state-of-the-art methods on various 3D semantic perception applications.

We consider the task of autonomously unloading boxes from trucks using an industrial manipulator robot. There are multiple challenges that arise: (1) real-time motion planning for a complex robotic system carrying two articulated mechanisms, an arm and a scooper, (2) decision-making in terms of what action to execute next given imperfect information about boxes such as their masses, (3) accounting for the sequential nature of the problem where current actions affect future state of the boxes, and (4) real-time execution that interleaves high-level decision-making with lower level motion planning. In this work, we propose a planning, learning, and reasoning framework to tackle these challenges, and describe its components including motion planning, belief space planning for offline learning, online decision-making based on offline learning, and an execution module to combine decision-making with motion planning. We analyze the performance of the framework on real-world scenarios. In particular, motion planning and execution modules are evaluated in simulation and on a real robot, while offline learning and online decision-making are evaluated in simulated real-world scenarios.

Use of physics-based simulation as a planning model enables a planner to reason and generate plans that involve non-trivial interactions with the world. For example, grasping a milk container out of a cluttered refrigerator may involve moving a robot manipulator in between other objects, pushing away the ones that are moveable and avoiding interactions with certain fragile containers. A physics-based simulator allows a planner to reason about the effects of interactions with these objects and to generate a plan that grasps the milk container successfully. The use of physics-based simulation for planning however is underutilized. One of the reasons for it being that physics-based simulations are typically way too slow for being used within a planning loop that typically requires tens of thousands of actions to be evaluated within a matter of a second or two. In this work, we develop a planning algorithm that tries to address this challenge. In particular, it builds on the observation that only a small number of actions actually need to be simulated using physics, and the remaining set of actions, such as moving an arm around obstacles, can be evaluated using a much simpler internal planning model, e.g., a simple collision-checking model. Motivated by this, we develop an algorithm called Planning with Selective Physics-based Simulation that automatically discovers what should be simulated with physics and what can utilize an internal planning model for pick-and-place tasks.

We propose a method to generate actuation plans for a reduced order, dynamic model of bipedal running. This method explicitly enforces robustness to ground uncertainty. The plan generated is not a fixed body trajectory that is aggressively stabilized: instead, the plan interacts with the passive dynamics of the reduced order model to create emergent robustness. The goal is to create plans for legged robots that will be robust to imperfect perception of the environment, and to work with dynamics that are too complex to optimize in real-time. Working within this dynamic model of legged locomotion, we optimize a set of disturbance cases together with the nominal case, all with linked inputs. The input linking is nontrivial due to the hybrid dynamics of the running model but our solution is effective and has analytical gradients. The optimization procedure proposed is significantly slower than a standard trajectory optimization, but results in robust gaits that reject disturbances extremely effectively without any replanning required.

In this paper, we address efficiently and robustly collecting objects stored in different trays using a mobile manipulator. A resolution complete method, based on precomputed reachability database, is proposed to explore collision-free inverse kinematics (IK) solutions and then a resolution complete set of feasible base positions can be determined. This method approximates a set of representative IK solutions that are especially helpful when solving IK and checking collision are treated separately. For real world applications, we take into account the base positioning uncertainty and plan a sequence of base positions that reduce the number of necessary base movements for collecting the target objects, the base sequence is robust in that the mobile manipulator is able to complete the part-supply task even there is certain deviation from the planned base positions. Our experiments demonstrate both the efficiency compared to regular base sequence and the feasibility in real world applications.

Maintaining high levels of productivity for planetary rover missions is very difficult due to limited communication and heavy reliance on ground control. There is a need for autonomy that enables more adaptive and efficient actions based on real-time information. This paper presents an autonomous mapping and exploration approach for planetary rovers. We first describe a machine learning model that actively combines remote and rover measurements for mapping. We focus on spectroscopic data because they are commonly used to investigate surface composition. We then incorporate notions from information theory and non-myopic path planning to improve exploration productivity. Finally, we demonstrate the feasibility and successful performance of our approach via spectroscopic investigations of Cuprite, Nevada; a well-studied region of mineralogical and geological interest. We first perform a detailed analysis in simulations, and then validate those results with an actual rover in the field in Nevada.

To date, many control strategies for legged robots have been proposed for stable locomotion over rough and unstructured terrains. However, these approaches require sensing information throughout locomotion, which may be noisy or unavailable at times. An alternative solution to rough terrain locomotion is a legged robot design that can passively adapt to the variations in the terrain without requiring knowledge of them. This paper presents one such solution in the design of a walking robot that employs pin array mechanisms to passively adapt to rough terrains. The pins are passively dropped over the terrain to conform to its variations and then locked to provide a statically stable stance. Locomotion is achieved with parallel four-bar linkages that swing forward the platforms in an alternating manner. Experimental evaluation of the robot demonstrates that the pin arrays enable legged locomotion over rough terrains under open-loop control.

This paper introduces the technique of tilt-and-pivot manipulation, a new method for picking thin, rigid objects lying on a flat surface through robotic dexterous in-hand manipulation. During the manipulation process, the gripper is controlled to reorient about the contact with the object such that its finger can get in the space between the object and the supporting surface, which is formed by tilting up the object, with no relative sliding motion at the contact. As a result, a pinch grasp can be obtained on the faces of the thin object with ease. We discuss issues regarding the kinematics and planning of tilt-and-pivot, effector shape design, and the overall practicality of the manipulation technique, which is general enough to be applicable to any rigid convex polygonal objects. We also present a set of experiments in a range of bin picking scenarios.

Light Detection and Ranging (LIDAR) sensors play an important role in the perception stack of autonomous robots, supplying mapping and localization pipelines with depth measurements of the environment. While their accuracy outperforms other types of depth sensors, such as stereo or time-of-flight cameras, the accurate modeling of LIDAR sensors requires laborious manual calibration that typically does not take into account the interaction of laser light with different surface types, incidence angles and other phenomena that significantly influence measurements. In this work, we introduce a physically plausible model of a 2D continuous-wave LIDAR that accounts for the surface-light interactions and simulates the measurement process in the Hokuyo URG-04LX LIDAR. Through automatic differentiation, we employ gradient-based optimization to estimate model parameters from real sensor measurements.

We present a vision-based navigation system that uses a visual memory to navigate. Such memory corresponds to a topological map of key images created from moving a virtual camera over a model of the real scene. The advantage of our approach is that it provides a useful insight into the navigability of a visual path without relying on a traditional learning stage. During the navigation stage, the robot is controlled by sequentially comparing the images stored in the memory with the images acquired by the onboard camera.The evaluation is conducted on a robotic arm equipped with a camera and the model of the environment corresponds to a top view image of an urban scene.

Recent studies in radar-based navigation present promising navigation performance using scanning radars. These scanning radar-based odometry methods are mostly feature-based; they detect and match salient features within a radar image. Differing from existing feature-based methods, this paper reports on a method using direct radar odometry, PhaRaO, which infers relative motion from a pair of radar scans via phase correlation. Specifically, we apply the Fourier Mellin transform (FMT) for Cartesian and log-polar radar images to sequentially estimate rotation and translation. In doing so, we decouple rotation and translation estimations in a coarse-to-fine manner to achieve real-time performance. The proposed method is evaluated using large-scale radar data obtained from various environments. The inferred trajectory yields a 2.34% (translation) and 2.93° (rotation) Relative Error (RE) over a 4km path length on average for the odometry estimation.

In this paper we introduce a novel technique that aims to dynamically control a modular bio-inspired soft-robotic arm in order to perform cyclic rhythmic patterns. Oscillatory signals are produced at the actuator's level by a central pattern generator (CPG), resulting in the generation of a periodic motion by the robot's end-effector. The proposed controller is based on a model-free neurodynamic scheme and is assigned with the task of training a policy that computes the parameters of the CPG model which generates a trajectory with desired features. The proposed methodology is first evaluated with a simulation model, which successfully reproduces the trained targets. Then experiments are also conducted using the real robot. Both procedures validate the efficiency of the learning architecture to successfully complete these tasks.

In this paper, we present a Perception-constrained Nonlinear Model Predictive Control (NMPC) framework for the real-time control of multi-rotor aerial vehicles. Our formulation considers both constraints from a perceptive sensor and realistic actuator limitations that are the rotor minimum and maximum speeds and accelerations. The formulation is meant to be generic and considers a large range of multi-rotor platforms (such as underactuated quadrotors or tilted-propellers hexarotors) since it does not rely on differential flatness for the dynamical equations, and a broad range of sensors, such as cameras, lidars, etc.... The perceptive constraints are expressed to maintain visibility of a feature point in the sensor's field of view, while performing a reference maneuver. We demonstrate both in simulation and real experiments that our framework is able to exploit the full capabilities of the multi-rotor, to achieve the motion under the aforementioned constraints, and control in real-time the platform at a motor-torque level, avoiding the use of an intermediate unconstrained trajectory tracker.

The increasing popularity of quadrotors has given rise to a class of predominantly vision-driven vehicles. This paper addresses the problem of perception-aware time optimal path parametrization for quadrotors. Although many different choices of perceptual modalities are available, the low weight and power budgets of quadrotor systems makes a camera ideal for on-board navigation and estimation algorithms. However, this does come with a set of challenges. The limited field of view of the camera can restrict the visibility of salient regions in the environment, which dictates the necessity to consider perception and planning jointly. The main contribution of this paper is an efficient time optimal path parametrization algorithm for quadrotors with limited field of view constraints. We show in a simulation study that a state-of-the-art controller can track planned trajectories, and we validate the proposed algorithm on a quadrotor platform in experiments.

Telepresence tele-action robots enable human workers to reliably perform difficult tasks in remote, cluttered, and human environments. However, the effort to control coordinated manipulation and active perception motions may exhaust and intimidate novice workers. We hypothesize that such cognitive efforts would be effectively reduced if the teleoperators are provided with autonomous camera selection and control aligned with the natural perception-action coupling of the human motor system. Thus, we conducted a user study to investigate the coordination of active perception control and manipulation motions performed with visual feedback from various wearable and standalone cameras in a telepresence scenario. Our study discovered rich information about telepresence camera selection to inform telepresence system configuration and possible teleoperation assistance design for reduced cognitive effort in robot teleoperation.

Robotic bronchoscopic intervention requires detailed 3D airway maps for both localisation and enhanced visualisation, especially at peripheral airways. Patient-specific airway maps can be generated from preoperative chest CT scans. Due to pathological abnormalities and anatomical variations, automatically delineating the airway tree with distal branches is a challenging task. In the paper, we propose a cascaded 2D+3D model that has been tailored for airway segmentation from pathological CT scans. A novel 2D neural network is developed to generate the initial predictions where the peripheral airways are refined by a 3D adversarial training model. A sampling strategy based on a sequence of morphological operations is employed for the concatenation of the 2D and 3D models. The method has been validated on 20 pathological CT scans with results demonstrating improved segmentation accuracy and consistency, especially in peripheral airways.

This paper introduces a novel approach for model predictive control of ballbots for path-following tasks. Ballbots are dynamically unstable mobile robots which are designed to balance on a single ball. The model presented in this paper is a simplied version of a full quaternion-based model of ballbots' underactuated dynamics which is suited for online implementation. Furthermore, the approach is extended to handle nearby obstacles directly in the MPC formulation. The presented controller is validated through simulation on a high fidelity model as well as through real-world experiments on a physical ballbot system.

State of the art methods for robotic path planning in dynamic environments, such as crowds or traffic, rely on hand crafted motion models for agents. These models often do not reflect interactions of agents in real world scenarios. To overcome this limitation, this paper proposes an integrated path planning framework using generative Recurrent Neural Networks within a Monte Carlo Tree Search (MCTS). This approach uses a learnt model of social response to predict crowd dynamics during planning across the action space. This extends our recent work using generative RNNs to learn the relationship between planned robotic actions and the likely response of a crowd. We show that the proposed framework can considerably improve motion prediction accuracy during interactions, allowing more effective path planning. The performance of our method is compared in simulation with existing methods for collision avoidance in a crowd of pedestrians, demonstrating the ability to control future states of nearby individuals. We also conduct preliminary real world tests to validate the effectiveness of our method.

The Extra Robotic Legs (XRL) system is a robotic augmentation worn by a human operator consisting of two articulated robot legs that walk with the operator and help bear a heavy backpack payload. It is desirable for the Human-XRL quadruped system to walk with the rear legs lead the front by 25% of the gait period, minimizing the energy lost from foot impacts while maximizing balance stability. Unlike quadrupedal robots, the XRL cannot command the human's limbs to coordinate quadrupedal locomotion. Using a pair of Rimless Wheel models, it is shown that the systems coupled with a spring and damper converge to the desired 25% phase difference. A Poincaré return map was generated using numerical simulation to examine the convergence properties to different coupler design parameters, and initial conditions. The Dynamically Coupled Double Rimless Wheel system was physically realized with a spring and dashpot chosen from the theoretical results, and initial experiments indicate that the desired synchronization properties may be achieved within several steps using this set of passive components alone.