ICRA 2021 Papers

Planning in environments with other agents whose future actions are uncertain often requires compromise between safety and performance. Here our goal is to design efficient planning algorithms with guaranteed bounds on the probability of safety violation, which nonetheless achieve non-conservative performance. To quantify a system’s risk, we define a natural criterion called interval risk bounds (IRBs), which provide a parametric upper bound on the probability of safety violation over a given time interval or task. We present a novel receding horizon algorithm, and prove that it can satisfy a desired IRB. Our algorithm maintains a dynamic risk budget which constrains the allowable risk at each iteration, and guarantees recursive feasibility by requiring a safe set to be reachable by a contingency plan within the budget. We empirically demonstrate that our algorithm is both safer and less conservative than strong baselines in two simulated autonomous driving experiments in scenarios involving collision avoidance with other vehicles, and additionally demonstrate our algorithm running on an autonomous class 8 truck.

Multi-stage forceful manipulation tasks, such as twisting a nut on a bolt, require reasoning over interlocking constraints over discrete and continuous choices. The robot must choose a sequence of discrete actions, or strategy, such as whether to pick up an object, and the continuous parameters of each of those actions, such as how to grasp that object. In forceful manipulation tasks, the force requirements substantially impact the choices of both strategy and parameters. To enable planning and executing forceful manipulation, we augment an existing task and motion planner with controllers that exert wrenches and constraints that explicitly consider torque and frictional limits. In two domains, opening a childproof bottle and twisting a nut, we demonstrate how the system considers a combinatorial number of strategies and how choosing actions that are robust to parameter variations impacts the choice of strategy. https://mcube.mit.edu/forceful-manipulation/

Designing a robot or structure that can fold into a target shape is a process that involves challenges originated from multiple sources. For example, the designer of self-folding robots must consider foldability from geometric and kinematic aspects to avoid self-collisions and undesired deformations. Recent works have shown success in estimating foldability of a design using robot motion planners. However, many foldable structures are actuated using physically coupled reactions, e.g., folding originated from thermal, chemical, or electromagnetic loads. Therefore, a reliable folding process must consider additional constraints that result from these critical and coupled phenomena. This work investigates the idea of efficiently incorporating computationally intensive physics simulations within the folding motion planner to affect the physical folding results. We will use the manufacturing process of laser-forming origami as an example to demonstrate the benefits of considering the physical properties of the foldable structure beyond its kinematics. We show that designs produced by the proposed method can be fabricated more efficiently.

Instance segmentation of planar regions in indoor scenes benefits visual SLAM and other applications such as augmented reality (AR) where scene understanding is required. Existing methods built upon two-stage frameworks show satisfactory accuracy but are limited by low frame rates. In this work, we propose a real-time deep neural architecture that estimates piece-wise planar regions from a single RGB image. Our model employs a variant of a fast single-stage CNN architecture to segment plane instances. Considering the particularity of the target detected, we propose Fast Feature Non-maximum Suppression (FF-NMS) to reduce the suppression errors resulted from overlapping bounding boxes of planes. We also utilize a Residual Feature Augmentation module in the Feature Pyramid Network (FPN) . Our method achieves significantly higher frame-rates and comparable segmentation accuracy against two-stage methods. We automatically label over 70,000 images as ground truth from the Stanford 2D-3D-Semantics dataset. Moreover, we incorporate our method with a state-of-the-art planar SLAM and validate its benefits.

The segmentation of a point cloud into planar primitives is a popular approach to first-line scene interpretation and is particularly useful in mobile robotics for the extraction of drivable or walkable surfaces and for tabletop segmentation for manipulation purposes. Unfortunately, the planar segmentation task becomes particularly challenging when the point clouds are obtained from an inherently noisy, robot-mounted sensor that is often in motion, therefor requiring real time processing capabilities. We present a real time-capable plane segmentation technique based on a region growing algorithm that exploits the organized structure of point clouds obtained from RGB-D sensors. In order to counteract the sensor noise, we invest into careful selection of seeds that start the region growing and avoid the computation of surface normals whenever possible. We implemented our algorithm in C++ and thoroughly tested it in both simulated and real-world environments where we are able to compare our approach against existing state-of-the-art methods implemented in the Point Cloud Library. The experiments presented here suggest that our approach is accurate and fast, even in the presence of considerable sensor noise.

We propose a novel method for planning shortest length piecewise-linear motions through complex environments punctured with static, moving, or even morphing obstacles. Using a moment optimization approach, we formulate a hierarchy of semidefinite programs that yield increasingly refined lower bounds converging monotonically to the optimal path length. Our global moment optimization approach natively handles continuous time constraints without any need for time discretization. For computational tractability, we derive an iterative motion planner which compares favorably with sampling-based and nonlinear optimization baselines.

Ego-motion estimation with 3D perception using visual odometry (VO) is known to be robust and economical among the existing odometry techniques. However, existing VO solutions are typically both computation intensive and memory intensive, which dramatically inhibits their deployment in IoT platforms such as robotic vehicles and handheld devices mostly equipped with resource-constrained MCU-level processors. To enable real-time and high-quality VO on these scenarios with thrifty resource budgets, we investigate state-of-the-art edge-based VO (EBVO) and propose an optimization framework called PicoVO that can greatly reduce the amount of computation as well as the memory footprint from the perspectives of both algorithm and implementation. First of all, we revisit the key processing stages of EBVO and propose an EBVO-oriented lightweight edge detector in the pre-processing stage, a sparse-to-dense processing scheme in the tracking stage, and a lightweight key-frame management in the post-processing stage. In addition to the algorithmic optimization, we further develop a dedicated quantization scheme particularly for the 3D feature calculation and Levenberg-Marquardt (LM) solver that are critical to the memory footprint and computation requirements of PicoVO. Evaluation on realistic RGB-D benchmark datasets is conducted on NUCLEO-F767ZI equipped with a 216MHz Cortex-M7 MCU and 512KB RAM. It reveals that PicoVO achieves 33fps@320x240 with high tracking precision comparable to state-of-the-art VOs on PC.

Road Network Persistent Surveillance Problem (RPSP) involves path planning for an unmanned ground vehicle (UGV) with detection ability to timely detect the events randomly occurred. The road network is formed by edges and weighted viewpoints, where the UGV must move along the edges. The existing method based on cognitive architecture is inadequate in terms of real-time and effective decision making. To improve the computation efficiency and accuracy of solving RPSP, a new algorithm called Pheromone-Diffusion-based Conscientious Reactive Persistent Surveillance (PD-CRPS) is proposed in this paper. Considering the detection ability of UGV, the monitoring weight, the surveillance effect, and the topology of the road network, a model of pheromone release and diffusion is established to estimate the global uncertainty through the local information around the UGV. The local optimum avoidance technology based on pheromone is designed, and the reactive architecture is used to design the payoff function of decision making. The worst-case computational time complexity of the PD-CRPS is far less than the existing cognitive architecture method. Simulation results show that the PD-CRPS can not only efficiently plan the path of UGV in the road networks with different topologies and observation obstacles, but also improve the calculation accuracy.

Assembly systems are commonly seen in production practice, where multiple components are joined in a manufacturing process to make a final product. In this paper, a decomposition/aggregation-based method is presented to evaluate the performance metrics of assembly systems with machines following the exponential reliability model (either synchronous or asynchronous). In particular, we consider the assembly system with multiple merge operations, each connected to a single external component line. The idea of the proposed method is to decompose the assembly system into a set of virtual serial lines based on the overlapping decomposition technique, evaluate of the starvation and blockage of the merge operations, and recursively update of the virtual machines’ parameters in the thus-obtained serial lines. Then, the performance metrics of the original assembly system can be approximated based on the corresponding machines and buffers in the virtual serial lines. Numerical experiments are carried out to justify the convergence and computational efficiency of the method, as well as to evaluate the approximation accuracy of the proposed algorithm.

In this paper, we propose an end-to-end self-driving network featuring a sparse attention module that learns to automatically attend to important regions of the input. The attention module specifically targets motion planning, whereas prior literature only applied attention in perception tasks. Learning an attention mask directly targeted for motion planning significantly improves the planner safety by performing more focused computation. Furthermore, visualizing the attention improves interpretability of end-to-end self-driving.

A technological revolution is occurring in the field of robotics with the data-driven deep learning technology. However, building datasets for each local robot is laborious. Meanwhile, data islands between local robots make data unable to be utilized collaboratively. To address this issue, the work presents Peer-Assisted Robotic Learning (PARL) in robotics, which is inspired by the peer-assisted learning in cognitive psychology and pedagogy. PARL implements data collaboration with the framework of cloud robotic systems. Both data and models are shared by robots to the cloud after semantic computing and training locally. The cloud converges the data and performs augmentation, integration, and transferring. Finally, fine tune this larger shared dataset in the cloud to local robots. Furthermore, we propose the DAT Network (Data Augmentation and Transferring Network) to implement the data processing in PARL. DAT Network can realize the augmentation of data from multi-local robots. We conduct experiments on a simplified self-driving task for robots (cars). DAT Network has a significant improvement in the augmentation in self-driving scenarios. Along with this, the self-driving experimental results also demonstrate that PARL is capable of improving learning effects with data collaboration of local robots.

We present a path planning framework for marine robots subject to uncertain ocean currents that exploits data from ensemble forecasting, which is a technique for current prediction used in oceanography. Ensemble forecasts represent a distribution of predicted currents as a set of flow fields that are considered to be equally likely. We show that the typical approach of computing the vector-wise mean and variance over this set can yield meaningless results, and propose an alternative approach that considers each flow field in the ensemble simultaneously. Our framework finds a sequence of vehicle controls that minimises the root-mean-square error distance (RMSE) over the full set of ensemble-induced trajectories. The key to achieving computational efficiency in this approach is our use of Monte Carlo tree search (MCTS) with a specialised heuristic that improves convergence rate while preserving asymptotic optimality and the anytime property. We demonstrate our results using real ensemble forecasts provided by the Australian Bureau of Meteorology, and provide comparisons with the deterministic mean-based approach where we observe RMSE reductions of 92% and 43% in two example scenarios. Further, we argue that the framework can be used in a plan-as-you-go manner where ensemble forecasts change over time. These results help to introduce ensemble forecasts as a viable source of data to improve path planning in marine robotics.

In recent years, the inspection of extreme environments using mobile robots has gained traction, as robots are able to mitigate the risk placed on humans and at times achieve what humans are unable to. In some scenarios, the robot is required to operate in cluttered environments with highly restricted access through 150 mm diameter ports. The MIRRAX robot has been designed to meet these challenges with the capability of reconfiguring itself to both access environments and navigate through tightly spaced obstacles. The joints used for reconfiguration of the robot introduce additional challenges for path planning due to the significant changes that can occur between adjacent poses. This paper presents a global path planner for MIRRAX. A Voronoi diagram is first used to generate a sparse graph to represent the topology of the environment, which allows for fast, coarse path planning. The coarse path is then refined via a heuristic pose fitting routine to ensure that the path is both collision-free and reduce unnecessary joint angle changes. The planner has been evaluated in simulation, demonstrating the feasibility of generating collision-free paths through narrow pathways for a reconfigurable robot.

We present a method for path optimization for ground vehicles in off-road environments at high speeds. This path optimization considers the kinematic constraints of the vehicle. By thinking in the actuator space we can represent such constraints as limits in the space rather than derived properties of the path. In this paper we present an actuator space approach to path optimization for off-road ground vehicles. This is done by representing the path as a list of steering angles over the path length. This transforms the set of kinematic constraints into constraints on the steering angle. We then put this path into a gradient descent solver. This produces paths that are kinematically feasible and optimized in accordance with our cost function. Finally, we tested the system both in simulation and on an off-road vehicle at speeds of 5 m/s.

We formulate a scenario in which an autonomous defender is tasked with intercepting an intruder that tries to reach a circular target region. This is a variant of the target defense problem proposed by Isaacs as a pursuit-evasion game. Unlike the original target guarding problem and its various extensions, we consider the effect of partial information by imposing sensing limitation to the robots. We analyze the game by decomposing it into three game phases: deployment, asymmetric information, and engagement phase. Focusing on a particular parameter regime, we propose a simple defender strategy together with the lower bound on the probability that it wins the game. The defender strategy in each phase is constructed so that the subsequent phase starts in a desired initial configuration. The proposed problem is rich in terms of the parameter regimes that it contains, and thus is expected to be a useful platform in exploring effective control policies.

Most industrial parts are parametric and their special properties are not fully explored yet. This paper proposes a new 6DoF pose estimation network for parametric shapes in stacked scenarios (ParametricNet). It treats a parametric shape, instead of a part object, as a category. The keypoints of individual instances are learned with point- wise regression and Hough voting scheme, from which specific parameter values are calculated. Then, the template keypoints are obtained based on the computed parameter values and the parametric shape templates. Finally, the 6DoF pose is estimated by least-square fitting between the individual instance’s and the template’s keypoints & centroid. On the public Siléane dataset, the average of APs of ParametricNet is 96%, compared with 82% for the state-of-the-art method. In addition, a new parametric dataset with four shape templates is constructed, in which the evaluated learning and generalization abilities of ParametricNet outperform the state-of-the-art methods. In particular, for the less symmetric shape, the mAP is improved by over 20%, which is an obvious improvement. Real-world experiments show that our method can grasp parametric shapes with unknown parameter values in stacked scenarios.

After years of development, various swarms of robots have been proposed for many complicated tasks, such as forming patterns, cooperative locomotion, and adapting to different environments. However, controlling microrobotic swarms is still a challenging task owing to the lacking of integrated devices on the small-scale agents, and actuation of multiple microrobotic swarms to different targets under the same global input will be even more difficult. In this work, we present a swarm of nickel nanorods and its diverse locomotion velocity compared with Fe3O4 nanoparticle swarms is implemented for actuating the two swarms to different targets under the same customized oscillating magnetic field. The effects of the magnetic anisotropy of agents on the macroscopic swarm behaviour are analysed theoretically. To prove the strategy, the speeds of the two swarms were characterized through experiments, and demonstrations were conducted to show the capability of driving the two swarms to different locations in the same environment. Furthermore, parallel locomotion of the two swarms towards opposite directions was also achieved on a tilted substrate. This work has proved the feasibility of simultaneously actuating two swarms to diverse targets and promoted fundamental understandings of microrobotic swarms.

Lidar odometry (LO) is a key technology in numerous reliable and accurate localization and mapping systems of autonomous driving. The state-of-the-art LO methods generally leverage geometric information to perform point cloud registration. Furthermore, obtaining the point cloud semantic information describing the environment more abundantly will facilitate the registration. We present a novel semantic lidar odometry method based on self-designed parameterized seman-tic features (PSFs) to achieve low-drift ego-motion estimation for autonomous vehicle in real time. We first use a convolutional neural network-based algorithm to obtain point-wise semantics from the input laser point cloud, and then use semantic labels to separate road, building, traffic sign and pole-like point cloud and fit them separately to obtain corresponding PSFs. A fast PSF-based matching enables us to refine geometric features (GeFs) registration, thereby reducing the impact of blurred submap surface on the accuracy of GeFs matching. Besides, we design an efficient instance-level method to accurately recognize and remove the dynamic objects while retaining static ones in the semantic point cloud, which are beneficial to further improve the accuracy of LO. We evaluate our method, namely PSF-LO, on the public dataset KITTI Odometry Benchmark and rank #1 among semantic lidar methods with an average translational error of 0.82% in the test dataset.