ICRA 2022 论文列表

Automatic disassembly planning for complex industrial products like vehicles checks the expandability of components already at early stages of design. For a fast computation of collision-free disassembly paths, sampling-based rigid body motion planning is used in the literature. However, in real-world scenarios there are circumstances that prevent the finding of plausible collision-free disassembly paths with these conventional motion planners. The most difficult problem is that many components have deformable fastening elements that are modeled in a relaxed state and often as a part of the rigid object. The fastening elements cause unavoidable collisions of the component with its environment along the actual disassembly path. In this paper, we present Iterative Mesh Modification Planning (IMMP). Given the information about fastening elements in advance, our method applies a controlled iterative process of geometric deformations and planning attempts to the component to be disassembled. With this process, we are able to disassemble the component from its installed position with a conventional rigid body motion planner taking fastening elements and also overpressure into account. We demonstrate the effectiveness of our method on real-world planning scenarios from the automotive industry.

This paper investigates the usefulness of reasoning about the uncertain presence of obstacles during path planning, which typically stems from the usage of probabilistic occupancy grid maps for representing the environment when mapping via a noisy sensor like a stereo camera. The traditional planning paradigm prescribes using a hard threshold on the occupancy probability to declare that a cell is an obstacle, and to plan a single path accordingly while treating unknown space as free. We compare this approach against a new uncertainty-aware planner, which plans two different path hypotheses and then merges their initial trajectory segments into a single one ending in a “next-best view” pose. After this informative view is taken, the planner commits to one of the hypotheses, or to a completely new one if a collision is imminent. Simulations were conducted comparing the proposed and traditional planner. Results show the existence of planning scenarios -like when the environment contains a dead-end, or when the goal is placed close to an obstacle- in which reasoning about uncertainty can significantly decrease the robot's traveled distance and increase the chances of reaching the goal. The new planner was also validated on a real Clearpath Jackal robot equipped with a ZED 2 stereo camera.

Circular economy and agile manufacturing require a safe and efficient industrial robot system working in close human proximity. Although, close proximity local sensing enables safe collaboration with small cobots. However, they cannot ensure safety at high velocities with a heavy-duty industrial robot. Stereo-camera and 3D LiDAR-based touch-less global sensing methods exist, but they do not address the safety standards. This work proposes a novel method for estimating the safety parameters in speed and separation monitoring mode for 3D vision sensors. Accurate estimation of these parameters ensures compact sensing zones. Thus, enabling efficient human-robot collaboration for permanent operator presence. The method requires less effort in setup. The developed software with a graphical user interface enables workers from wide technical expertise to perform safety measurements at a precision of ±15ms in reaction time estimation. The experiments are repeatable and capture the statistical data for low error in estimation. The estimated parameters for two exemplary 3D sensors enable a human to work in close proximity of 20 cm while enabling safety from a collision.

It is well established that a stiff structure along with an optimal mass distribution are key features to perform dynamic movements, and parallel designs provide these characteristics to a robot. This work presents the new upper-body design of the humanoid robot RH5 named RH5 Manus with series-parallel hybrid design. The new design choices allow us to perform dynamic motions including tasks that involve a payload of 4 kg in each hand and fast boxing motions. The parallel kinematics combined with an overall serial chain of the robot provides us with high force production along with a larger range of motion and low peripheral inertia. The robot is equipped with backdrivable actuators with current sensing, force-torque sensors, stereo camera, laser scanners, high-resolution encoders etc that provide interaction with operators and environment. We generate several diverse dynamic motions using trajectory optimization, and successfully execute them on the robot with accurate trajectory and velocity tracking, while respecting joint rotation, velocity, and torque limits.

In vision-based reinforcement learning (RL) tasks, it is prevalent to assign auxiliary tasks with a surrogate self-supervised loss so as to obtain more semantic representations and improve sample efficiency. However, abundant information in self-supervised auxiliary tasks has been disregarded, since the representation learning part and the decision-making part are separated. To sufficiently utilize information in auxiliary tasks, we present a simple yet effective idea to employ self-supervised loss as an intrinsic reward, called Intrinsically Motivated Self-Supervised learning in Reinforcement learning (IM-SSR). We formally show that the self-supervised loss can be decomposed as exploration for novel states and robustness improvement from nuisance elimination. IM-SSR can be effortlessly plugged into any reinforcement learning with self-supervised auxiliary objectives with nearly no additional cost. Combined with IM-SSR, the previous underlying algorithms achieve salient improvements on both sample efficiency and generalization in various vision-based robotics tasks from the DeepMind Control Suite, especially when the reward signal is sparse.

We present a novel interval-based visual-inertial LiDAR SLAM (i-VIL SLAM) method that solely assumes sensor errors to be bounded and propagates the error from the input sources to the estimated map and trajectory using interval analysis. The method allows us to restrict the solution set of the robot poses and the position of the landmarks to the set that is consistent with the measurements. If the error limits are not violated, it is guaranteed that the estimated set contains the true solution. The accumulation of the uncertainty is stabilized by anchoring poses derived from GNSS/INS data. Furthermore, for the first time we compare confidence ellipses determined by a classical SLAM graph optimization approach with the interval estimates of the robot poses provided by our method. In this work, we experimentally show that the marginal co-variances computed by the classical SLAM graph optimization are too overconfident and underestimate the uncertainty of the poses. While the 99.9 %-ellipsoids derived from the marginal covariances of the poses only enclose less than 64 % of the ground truth in the worst case, our method provides interval bounds for the pose parameters that enclose the ground truth for more than 96 % of all frames.

In this study, working with the task of object retrieval in clutter, we have developed a robot learning framework in which Monte Carlo Tree Search (MCTS) is first applied to enable a Deep Neural Network (DNN) to learn the intricate interactions between a robot arm and a complex scene containing many objects, allowing the DNN to partially clone the behavior of MCTS. In turn, the trained DNN is integrated into MCTS to help guide its search effort. We call this approach learning-guided Monte Carlo tree search for Object REtrieval (MORE), which delivers significant computational efficiency gains and added solution optimality. MORE is a self-supervised robotics framework/pipeline capable of working in the real world that successfully embodies the System 2 → System 1 learning philosophy proposed by Kahneman, where learned knowledge, used properly, can help greatly speed up a time-consuming decision process over time. Videos and supplementary material can be found at https://github.com/arc-l/more.

Interactive robotic grasping using natural language is one of the most fundamental tasks in human-robot interaction. However, language can be a source of ambiguity, particularly when there are ambiguous visual or linguistic contents. This paper investigates the use of object attributes in disambiguation and develops an interactive grasping system capable of effectively resolving ambiguities via dialogues. Our approach first predicts target scores and attribute scores through vision-and-language grounding. To handle ambiguous objects and commands, we propose an attribute-guided formulation of the partially observable Markov decision process (Attr-POMDP) for disambiguation. The Attr-POMDP utilizes target and attribute scores as the observation model to calculate the expected return of an attribute-based (e.g., “what is the color of the target, red or green?”) or a pointing-based (e.g., “do you mean this one?”) question. Our disambiguation module runs in real time on a real robot, and the interactive grasping system achieves a 91.43 % selection accuracy in the real-robot experiments, outperforming several baselines by large margins. Supplementary material is available at https://sites.google.com/umn.eduJattr-disam.

Learning from demonstration (LfD) provides a fast, intuitive and efficient framework to program robot skills, which has gained growing interest both in research and industrial applications. Most complex manipulation tasks are long-term and involve a set of skill primitives. Thus it is crucial to have a reliable coordination scheme that selects the correct sequence of skill primitive and the correct parameters for each skill, under various scenarios. Instead of relying on a precise simulator, this work proposes a human-in-the-loop coordination framework for LfD skills that: builds parameterized skill models from kinesthetic demonstrations; constructs a geometric task network (GTN) on-the-fly from human instructions; learns a hierarchical control policy incrementally during execution. This framework can reduce significantly the manual design efforts, while improving the adaptability to new scenes. We show on a 7-DoF robotic manipulator that the proposed approach can teach complex industrial tasks such as bin sorting and assembly in less than 30 minutes.

Accurate and robust initialization is crucial in visual-inertial system, which significantly affects the localization accuracy. Most of the existing feature-based initialization methods rely on point features to estimate initial parameters. However, the performance of these methods often decreases in real scene, as point features are unstable and may be discontinuously observed especially in low textured environments. By contrast, line features, providing richer geometrical information than points, are also very common in man-made buildings. Thereby, in this paper, we propose a novel visual-inertial initialization method integrating both point and line features. Specifically, a closed-form method of line features is presented for initialization, which is combined with point-based method to build an integrated linear system. Parameters including initial velocity, gravity, point depth and line's endpoints depth can be jointly solved out. Furthermore, to refine these parameters, a global optimization method is proposed, which consists of two novel nonlinear least squares problems for respective points and lines. Both gravity magnitude and gyroscope bias are considered in refinement. Extensive experimental results on both simulated and public datasets show that integrating point and line features in initialization stage can achieve higher accuracy and better robustness compared with pure point-based methods.

The challenge of mapping indoor environments is addressed. Typical heuristic algorithms for solving the motion planning problem are frontier-based methods, that are especially effective when the environment is completely unknown. However, in cases where prior statistical data on the environment's architectonic features is available, such algorithms can be far from optimal. Furthermore, their calculation time may increase substantially as more areas are exposed. In this paper we propose two means by which to overcome these shortcomings. One is the use of deep reinforcement learning to train the motion planner. The second is the inclusion of a pre-trained generative deep neural network, acting as a map predictor. Each one helps to improve the decision making through use of the learned structural statistics of the environment, and both, being realized as neural networks, ensure a constant calculation time. We show that combining the two methods can shorten the duration of the mapping process by up to 4 times, compared to frontier-based motion planning.

We propose a motion generation model that can achieve robust behavior against environmental changes based on language instructions at a low cost. Conventional robots that communicate with humans use a restricted environment and language to build up a mapping between language and motion, and thus need to prepare a huge training set in order to achieve versatility. Our method trains pairs of language, visual, and motor information of the robot, and generates motions in real-time based on the “attention” of the language instructions. Specifically, the robot generates motions while focusing on the indicated objects by the human when multiple objects are in the field of view. In addition, since position recognition and motion generation of the indicated object are performed in real-time, robust motion generation is possible in response to changes in the object position and lighting conditions. We clarified that features related to the object name and its location are self-organized in the latent (PB: Parametric Bias) space by end-to-end learning of robot motion and sentences. These observations may indicate the importance of integrated learning of robot motion and sentences since such feature representations cannot be obtained by learning motions alone.

Insulator visual aiming is difficult for washing drone due to the complex washing environment, strong dis-turbance, lack of debugging environment, and other factors. Conventional visual servo control methods often fail to consider these complex factors adequately and fall short in reliable insulator visual aiming. To address these problems, we propose a novel multi-feature fusion-based drone visual servo control method for accurate insulator visual aiming. A multi-feature fusion neural network (MFFNet) is proposed to map the dif-ferent input modalities into an embedding space spanned by the learned deep features. Suitable control commands are generated by the simple combination of learned deep features. These deep features represent the intrinsic structural properties of the insulator and the motion pattern of the drones. Particularly, our method is trained purely in simulation and transferred to a real drone directly. Moreover, accurate visual aiming is guaranteed even in strong disturbance environments. Simulation and experimental results verify the high accurate insulator aiming, anti-disturbance, and sim-to-real transfer capabilities of the proposed method. Video: https://youtu.be/Ptlajzvp46A.

Achieving instinctive multi-grasp control of prosthetic hands typically still requires a large number of sensors, such as electromyography (EMG) electrodes mounted on a residual limb, that can be costly and time consuming to position, with their signals difficult to classify. Deep-learning-based EMG classifiers however have shown promising results over traditional methods, yet due to high computational requirements, limited work has been done with in-prosthetic training. By targeting specific muscles non-invasively, separating grasping action into hold and release states, and implementing data augmentation, we show in this paper that accurate results for embedded, instinctive, multi-grasp control can be achieved with only 2 low-cost sensors, a simple neural network, and minimal amount of training data. The presented controller, which is based on only 2 surface EMG (sEMG) channels, is implemented in an enhanced version of the OLYMPIC prosthetic hand. Results demonstrate that the controller is capable of identifying all 7 specified grasps and gestures with 93% accuracy, and is successful in achieving several real-life tasks in a real world setting.

Even if UAVs undoubtedly had a profound effect on the visual inspection capabilities of transmission lines, rolling robots, especially for bundled configurations, will still play an extensive role in the maintenance of these strategic assets. As such, LineRanger is among the most efficient and capable wheeled platform, that can travel at an average speed of 8 km/h. In this paper, LineRanger mechanical design insights are shared, unfolding how strictly mechanical solutions deals with ease of obstacle crossing, ease of installation procedure without requiring any linemen to access the high voltage environment area, and ease of deployment of dedicated, custom-made sensors onto the line component of interest. This novel robotic platform is now being deployed onto Hydro-Quebec power grid network, performing high value applications.

We address the problem of devising the means for a robot to rapidly and safely learn insertion skills with just a few human interventions and without hand-crafted rewards or demonstrations. Our InsertionNet version 2.0 provides an improved technique to robustly cope with a wide range of use-cases featuring different shapes, colors, initial poses, etc. In particular, we present a regression-based method based on multimodal input from stereo perception and force, augmented with contrastive learning for the efficient learning of valuable features. In addition, we introduce a one-shot learning technique for insertion, which relies on a relation network scheme to better exploit the collected data and to support multi-step insertion tasks. Our method improves on the results obtained with the original InsertionNet, achieving an almost perfect score (above 97.5% on 200 trials) in 16 real-life insertion tasks while minimizing the execution time and contact during insertion. We further demonstrate our method's ability to tackle a real-life 3-step insertion task and perfectly solve an unseen insertion task without learning.

Although there has been tremendous progress in autonomous driving, navigating environments and predicting the behavior of other drivers in the presence of occlusions remains challenging. Cities have started investing in infrastructure sensors that could provide information about occluded spaces. We propose a framework that integrates infrastructure-to-vehicle communication in autonomous vehicle decision making, improving operational safety and mobility in challenging environments. By framing the problem as a partially observable Markov decision process in which querying an infrastructure sensor is a data-gathering action, we reduce the computational complexity associated with sensor processing while maintaining equivalent performance compared to an omniscient actor and demonstrate the value of infrastructure communication through a series of experiments.

Informative planning seeks a sequence of actions that guide the robot to collect the most informative data to build a large-scale environmental model or learn a dynamical system. Existing work in informative planning mainly focuses on proposing new planners and applying them to various robotic applications such as environmental monitoring, autonomous exploration, and system identification. The informative planners optimize an objective given by a probabilistic model, e.g., Gaussian process regression (GPR). In practice, the ubiquitous sensing outliers can easily affect the model, resulting in a misleading objective. A straightforward solution is to filter out the outliers in the sensing data stream using an off-the-shelf outlier detector. However, informative samples are also scarce by definition so they might be falsely filtered out. In this paper, we propose a method to enable the robot to re-visit the locations where outliers were sampled besides optimizing the informative planning objective. The robot can collect more samples in the vicinity of outliers and update the outlier detector to reduce the number of false alarms. We achieve this by designing a new objective for the Pareto Monte Carlo tree search (MCTS). We demonstrate that the proposed framework performs better than applying an outlier detector naively.

We present a planning framework for min-imising the deterministic worst-case error in sparse Gaus-sian process (GP) regression. We first derive a univer-sal worst-case error bound for sparse GP regression with bounded noise using interpolation theory on reproducing kernel Hilbert spaces (RKHSs). By exploiting the conditional inde-pendence (CI) assumption central to sparse GP regression, we show that the worst-case error minimisation can be achieved by solving a posterior entropy minimisation problem. In turn, the posterior entropy minimisation problem is solved using a Gaussian belief space planning algorithm. We corroborate the proposed worst-case error bound in a simple 1D example, and test the planning framework in simulation for a 2D vehicle in a complex flow field. Our results demonstrate that the proposed posterior entropy minimisation approach is effective in minimising deterministic error, and outperforms the conventional measurement entropy maximisation formulation when the inducing points are fixed.