ICRA 2021 Papers

Safety is an important topic in autonomous driving since any collision may cause serious injury to people and damage to property. Hamilton-Jacobi (HJ) Reachability is a formal method that verifies safety in multi-agent interaction and provides a safety controller for collision avoidance. However, due to the worst-case assumption on the car’s future behaviours, reachability might result in too much conservatism such that the normal operation of the vehicle is badly hindered. In this paper, we leverage the power of trajectory prediction and propose a prediction-based reachability framework to compute safety controllers. Instead of always assuming the worst case, we cluster the car’s behaviors into multiple driving modes, e.g. left turn or right turn. Under each mode, a reachability-based safety controller is designed based on a less conservative action set. For online implementation, we first utilize the trajectory prediction and our proposed mode classifier to predict the possible modes, and then deploy the corresponding safety controller. Through simulations in a T-intersection and an 8-way roundabout, we demonstrate that our prediction-based reachability method largely avoids collision between two interacting cars and reduces the conservatism that the safety controller brings to the car’s original operation.

Accurate post-impact velocity predictions are essential in developing impact-aware manipulation strategies for robots, where contacts are intentionally established at non-zero speed mimicking human manipulation abilities in dynamic grasping and pushing of objects. Starting from the recorded dynamic response of a 7DOF torque-controlled robot that intentionally impacts a rigid surface, we investigate the possibility and accuracy of predicting the post-impact robot velocity from the pre-impact velocity and impact configuration. The velocity prediction is obtained by means of an impact map, derived using the framework of nonsmooth mechanics, that makes use of the known rigid-body robot model and the assumption of a frictionless inelastic impact.The main contribution is proposing a methodology that allows for a meaningful quantitative comparison between the recorded post-impact data, that exhibits a damped oscillatory response after the impact, and the post-impact velocity prediction derived via the readily available rigid-body robot model, that presents no oscillations and that is the one typically obtained via mainstream robot simulator software. The results of this new approach are promising in terms of prediction accuracy and thus relevant for the growing field of impact-aware robot control. The recorded impact data (18 experiments) is made publicly available, together with the numerical routines employed to generate the quantitative comparison, to further stimulate interest/research in this field.

We investigate a novel deep-learning-based approach to estimate uncertainty in stereo disparity prediction networks. Current state-of-the-art methods often formulate disparity prediction as a regression problem with a single scalar output in each pixel. This can be problematic in practical applications as in many cases there might not exist a single well defined disparity, for example in cases of occlusions or at depth-boundaries. While current neural-network-based disparity estimation approaches obtain good performance on benchmarks, the disparity prediction is treated as a black box at inference time. In this paper we show that by formulating the learning problem as a regression with a distribution target, we obtain a robust estimate of the uncertainty in each pixel, while maintaining the performance of the original method. The proposed method is evaluated both on a large-scale standard benchmark, as well on our own data. We also show that the uncertainty estimate significantly improves by maximizing the uncertainty in those pixels that have no well defined disparity during learning.

In industrial assembly tasks, the in-hand pose of grasped objects needs to be known with high precision for subsequent manipulation tasks such as insertion. This problem (in-hand-pose estimation) has traditionally been addressed using visual recognition or tactile sensing. On the one hand, while visual recognition can provide efficient pose estimates, it tends to suffer from low precision due to noise, occlusions and calibration errors. On the other hand, tactile fingertip sensors can provide precise complementary information, but their low durability significantly limits their use in real-world applications. To get the best of both worlds, we propose an efficient method for in-hand pose estimation using off-the-shelf cameras and robot wrist force sensors, which requires no precise camera calibration. The key idea is to utilize visual and contact information adaptively to maximally reduce the uncertainty about the in-hand object pose in a Bayesian state estimation framework. As most of the uncertainty can be resolved from visual observations, our approach reduces the number of physical environment interactions while keeping a high pose estimation accuracy. Our experimental evaluation demonstrates that our approach can estimate object poses with sub-mm precision with an off-the-shelf camera and force-torque sensor.

This paper deals with a problem of precise jumping for legged robots: what are the control inputs required to perform a jump that results in a desired landing point? We propose a novel precise jump planning method, formulated as a bilevel optimization problem. The presented formulation exploits certain insights into the jump dynamics, leading to a low-dimensional optimization problem, and allowing fast computation. During the Flight phase of a jump there are no external forces other than gravity acting on the robot, so its centroidal angular momentum (CAM) is conserved, and its center of mass (COM) follows a ballistic trajectory. This trajectory depends solely on COM position and velocity at Liftoff. We define a bilevel optimization problem consisting of a nonlinear upper-level problem, and a lower-level quadratic programming (QP) problem. The upper-level problem selects COM position and velocity at Liftoff that result in a desired landing point, while minimizing CAM at Liftoff. The lower-level problem selects ground reaction forces during Push-Off that achieve desired COM position and velocity at Liftoff. The results are presented on a simulated one-legged robot, but the proposed approach can be extended to bipeds and multi-legged robots.

Motivated by the limited flight time of batterypowered multi-rotor UAVs, in this paper we address the problem of generating energy-optimal trajectories for a planar quadrotor. More specifically, by considering an accurate electrical model for the brushless DC motors and rest-to-rest maneuvers between two predefined boundary states, we explicitly compute the minimum-energy curves by adopting a free and a fixed end-time optimal control formulation. The numerical solution of these optimal control problems hinges upon a simple yet effective indirect projected gradient method. Simulation experiments illustrate the theory in a variety of realistic flight scenarios.

Compliance in robot actuation provides a solution to perform safe physical human-robot interaction. Conventional compliant actuators (variable stiffness actuators, series elastic actuators) used more than two motors or closed-loop controller to modulate both stiffness and equilibrium position independently. These actuators are complex, lack of energy efficiency, and have limited stiffness range. In conjunction with an active, positive stiffness modulation, implementing a passive negative stiffness element enabled a compact design of the compliant actuator. This paper suggests a power transmission design of fast and energy-efficient stiffness modulation based on this new compliant actuator concept. First, the double slider-crank mechanism made fast stiffness modulation and high energy-efficiency. Second, positioning the leaf spring’s bending location to the center also enabled the fast stiffness modulation speed and wide range stiffness modulation. Third, optimized elliptical cam with compression spring generated negative stiffness in output. We provide theoretical modeling of each mechanical drivetrains and characterization of positive stiffness modulation (range and speed) and negative stiffness with corresponding power consumption experimentally.

In this paper, we present the positioning control problem for underactuated unmanned surface vehicles (USVs) in the presence of unknown external disturbances, such as the wind, waves, and currents. The three control objectives of positioning control for underactuated USVs are, firstly, to retain a predefined constant distance to a look-ahead point, secondly, to regulate the vehicle heading to point at the look-ahead point and, thirdly, to update the position of the look-ahead point to rotate the USVs. Particularly, a new guidance rule is proposed to update the look-ahead point based on the positioning error such that the predefined desired position is maintained and the vehicle heading is counter to the direction of resultant environmental force at the same time. To illustrate the stability of the positioning control system, the error dynamics driven by the component of resultant environmental force is derived and analyzed. In addition to the simulations, experiments were carried out offshore in the face of wind and waves to verify the validity and effectiveness of the proposed method.

In this work, we propose an approach to manipulate objects by position-controlled robot hands: in-hand caging manipulation. In this method, an object is manipulated based on caging without force sensing or force control. An object is caged by a robot hand throughout manipulation, and we can locate the object around a goal by deformation of the cage without sensing the object configuration. In this paper, we considered 2-D polygonal objects as the targets of in- hand caging manipulation. We also considered some position- controlled hands as the devices to manipulate the objects. A motion planning algorithm for the hands was proposed and applied to planar in-hand caging manipulation. By moving the hands according to the result of the motion planning, it was possible to manipulate the objects without sensing. Our proposed method can be applied to a device such as a part feeder that aligns variously shaped parts in the same pose.

We propose novel solvers for estimating the egomotion of a calibrated camera mounted to a moving vehicle from a single affine correspondence via recovering special homographies. For the first, second and third classes of solvers, the sought plane is expected to be perpendicular to one of the camera axes. For the fourth class, the plane is orthogonal to the ground with unknown normal, e.g., it is a building facade. All methods are solved via a linear system with a small coefficient matrix, thus, being extremely efficient. Both the minimal and over-determined cases can be solved by the proposed solvers. They are tested on synthetic data and on publicly available real-world datasets. The novel methods are more accurate or comparable to the traditional algorithms and are faster when included in state-of-the-art robust estimators. The source code is publicly available[1].

A polyhedral friction cone is a set of reaction wrenches that an object can experience whilst in contact with its environment. This polyhedron is a powerful tool to control an object’s motion and interaction with the environment. It can be derived analytically, upon knowledge of object and environment geometries, contact point locations and friction coefficients. We propose to estimate the polyhedral friction cone so that a priori knowledge of these quantities is no longer required. Additionally, we introduce a solution to transform the estimated friction cone to avoid re-estimation while the object moves. We present an analysis of the estimated polyhedral friction cone and demonstrate its application for manipulating an object in simulation and with a real robot.

Transferring reinforcement learning policies trained in physics simulation to the real hardware remains a challenge, known as the "sim-to-real" gap. Domain randomization is a simple yet effective technique to address dynamics discrepancies across source and target domains, but its success generally depends on heuristics and trial-and-error. In this work we investigate the impact of randomized parameter selection on policy transferability across different types of domain discrepancies. Contrary to common practice in which kinematic parameters are carefully measured while dynamic parameters are randomized, we found that virtually randomizing kinematic parameters (e.g., link lengths) during training in simulation generally outperforms dynamic randomization. Based on this finding, we introduce a new domain adaptation algorithm that utilizes simulated kinematic parameters variation. Our algorithm, Multi-Policy Bayesian Optimization, trains an ensemble of universal policies conditioned on virtual kinematic parameters and efficiently adapts to the target environment using a limited number of target domain rollouts. We showcase our findings on a simulated quadruped robot in five different target environments covering different aspects of domain discrepancies.

Accurately localizing in and mapping an environment are essential building blocks of most autonomous systems. In this paper, we present a novel approach for LiDAR odometry and mapping, focusing on improving the mapping quality and at the same time estimating the pose of the vehicle. Our approach performs frame-to-mesh ICP, but in contrast to other SLAM approaches, we represent the map as a triangle mesh computed via Poisson surface reconstruction. We perform the surface reconstruction in a sliding window fashion over a sequence of past scans. In this way, we obtain accurate local maps that are well suited for registration and can also be combined into a global map. This enables us to build a 3D map showing more geometric details than common mapping approaches relying on a truncated signed distance function or surfels. Our experimental evaluation shows quantitatively and qualitatively that our maps offer higher geometric accuracies than these other map representations. We also show that our maps are compact and can be used for LiDAR-based odometry estimation with a novel ray-casting-based data association.

We propose a practical approach for detecting the event that a human wearing an IMU-equipped bracelet points at a moving robot; the approach uses a learned classifier to verify if the robot motion (as measured by its odometry) matches the wrist motion, and does not require that the relative pose of the operator and robot is known in advance. To train the model and validate the system, we collect datasets containing hundreds of real-world pointing events. Extensive experiments quantify the performance of the classifiers and relevant metrics of the resulting detectors; the approach is implemented in a real-world demonstrator that allows users to land quadrotors by pointing at them.

We introduce a new solution to point set registration, a fundamental geometric problem occurring in many computer vision and robotics applications. We consider the specific case in which the point sets are segmented into semantically annotated parts. Such information may for example come from object detection or instance-level semantic segmentation in a registered RGB image. Existing methods incorporate the additional information to restrict or re-weight the point-pair associations occurring throughout the registration process. We introduce a novel hierarchical association framework for a simultaneous inference of semantic region association likelihoods. The formulation is elegantly solved using cascaded expectation-maximization. We conclude by demonstrating a substantial improvement over existing alternatives on open RGBD datasets.

Traditional convolution for capturing local structures and relationships remains a key technical limit in 3D semantic segmentation, which neglects the certain influence of the adjacent points on the central point in the disordered local point clouds. In this paper, we propose a novel joint-edge graph convolution neural network (JEGCN), which can extract the dynamic features of each local area and transfer the edge information between the vertex pairs to the adjacent vertices. In the proposed graph convolution module, the adjacent vertices are selected with high classification confidence which can guide the central vertex, and then reweight these vertices. Considering the lack of texture features in 3D point clouds, we incorporate 2D image features to adjacent feature propagation to effectively extract the local and global features of point clouds. The experimental results based on ScanNet and S3DIS datasets demonstrate the effectiveness of the proposed method.

In this paper, we present PocoNet: Point cloud Online COmpression NETwork to address the task of SLAM-oriented compression. The aim of this task is to select a compact subset of points with high priority to maintain localization accuracy. The key insight is that points with high priority have similar geometric features in SLAM scenarios. Hence, we tackle this task as point cloud segmentation to capture complex geometric information. We calculate observation counts by matching between maps and point clouds and divide them into different priority levels. Trained by labels annotated with such observation counts, the proposed network could evaluate the point-wise priority. Experiments are conducted by integrating our compression module into an existing SLAM system to evaluate compression ratios and localization performances. Experimental results on two different datasets verify the feasibility and generalization of our approach.

Unmanned Aerial Vehicles (UAVs) have introduced benefits in many areas of the energy sector. Today, power line sensor deployment is manually executed on passive power lines using helicopters, introducing great risks, costs and difficulties for the power distribution companies and the human operators.In this paper, we present a novel modular mechanical system utilizing pneumatic as the actuation source to deploy a sensor unit to a power line using a UAV, with aid of an autonomous alignment algorithm. The results show that the UAV can facilitate the sensor unit, and is capable of deploying it to the power line in an autonomous manner. The works leave opportunity for expanding to further applications in the future.

This paper extends our previous work on a pneumatic bending module and presents two more modules for rotational and translational motions. In these modules, antagonistic chambers enveloped by rigid shells are adopted to realize bidirectional actuation, and they are characterized by safe actuation, enhanced torque/force output, independent stiffness tuning, and real-time position control. Due to their mechanical modularity, they can be conveniently assembled into robotic systems with multiple degrees of freedom (DoFs) according to different requirements. A complete workflow is presented including the module design, fabrication, theoretical modelling, controller design, and experimental validation. A reconfigurable robotic arm with high dexterity is also assembled using these modules, demonstrating the effectiveness of the proposed modules to develop robotic systems for safe, forceful, and precise tasks.