IROS 2022 论文列表

Sensor-based environmental perception is a crucial part of the autonomous driving system. In order to get an excellent perception of the surrounding environment, an intelligent system would configure multiple LiDARs (3D Light Detection and Ranging) to cover the distant and near space of the car. The precision of perception relies on the quality of sensor calibration. This research aims at developing an accurate, automatic, and robust calibration strategy for multiple LiDAR systems in the general road scene. We thus propose CROON (automatic multi-LiDAR Calibration and Refinement methOd in rOad sceNe), a two-stage method including rough and refinement calibration. The first stage can calibrate the sensor from an arbitrary initial pose, and the second stage is able to precisely calibrate the sensor iteratively. Specifically, CROON utilize the nature characteristics of road scene so that it is independent and easy to apply in large-scale conditions. Experimental results on real-world and simulated data sets demonstrate the reliability and accuracy of our method. All the related data sets and codes are open-sourced on the Github website https://github.com/OpenCalib/LiDAR2LiDAR.

In typical data-based grasping methods, a grasp based on parallel-jaw grippers is parameterized by the center of the gripper, the rotation angle, and the gripper opening width so as to predict the quality and pose of grasps at every pixel. In contrast, a grasp is represented using only two contact points for contact-points-based grasp representation, which allows for fusion with tactile sensors more naturally. In this work, we propose a method using contact-points-based grasp representation to get a robust grasp using only one contact points quality map generated by a neural network, which significantly reduces the complexity of the network with fewer parameters. We provide a synthetic dataset including depth image and contact points quality map generated by thousands of 3D models. We also provide the method for data generation, which can be used for contact-points-based multi-fingers grasp. Experiments show that contact points quality network can plan an available grasp in 0.15 seconds. The grasping success rate for unknown household objects is 94%. Our method is also available for deformable objects with a success rate of 95%. The dataset and reference code can be found on the project website: https://sites.google.com/view/cpqnet.

Learning representations that generalize across tasks and domains is challenging yet necessary for autonomous systems. Although task-driven approaches are appealing, de-signing models specific to each application can be difficult in the face of limited data, especially when dealing with highly variable multimodal input spaces arising from different tasks in different environments. We introduce the first general-purpose pretraining pipeline, COntrastive Multimodal Pretraining for AutonomouS Systems (COMPASS), to overcome the limitations of task-specific models and existing pretraining approaches. COMPASS constructs a multimodal graph by considering the essential information for autonomous systems and the proper-ties of different modalities. Through this graph, multimodal signals are connected and mapped into two factorized spatio-temporal latent spaces: a “motion pattern space” and a “current state space.” By learning from multimodal correspondences in each latent space, COMPASS creates state representations that models necessary information such as temporal dynamics, geometry, and semantics. We pretrain COMPASS on a large-scale multimodal simulation dataset TartanAir [1] and evaluate it on drone navigation, vehicle racing, and visual odometry tasks. The experiments indicate that COMPASS can tackle all three scenarios and can also generalize to unseen environments and real-world data.11Our code implementation can be found at https://github.com/microsoft/COMPASS

We describe a novel approach to indoor place recognition from RGB point clouds based on aggregating low-level colour and geometry features with high-level implicit semantic features. It uses a 2-stage deep learning framework, in which the first stage is trained for the auxiliary task of semantic segmentation and the second stage uses features from layers in the first stage to generate discriminate descriptors for place recognition. The auxiliary task encourages the features to be semantically meaningful, hence aggregating the geometry and colour in the RGB point cloud data with implicit semantic information. We use an indoor place recognition dataset derived from the ScanNet dataset for training and evaluation, with a test set comprising 3,608 point clouds generated from 100 different rooms. Comparison with a traditional feature-based method and four state-of-the-art deep learning methods demonstrate that our approach significantly outperforms all five methods, achieving, for example, a top-3 average recall rate of 75% compared with 41% for the closest rival method. Our code is available at: https://github.com/YuhangMing/Semantic-Indoor-Place-Recognition

We present a decentralized path-planning algorithm for navigating multiple differential-drive robots in dense environments. In contrast to prior decentralized methods, we propose a novel congestion metric-based replanning that couples local and global planning techniques to efficiently navigate in scenarios with multiple corridors. To handle dense scenes with narrow passages, our approach computes the initial path for each agent to its assigned goal using a lattice planner. Based on neighbors' information, each agent performs online replanning using a congestion metric that tends to reduce the collisions and improves the navigation performance. Furthermore, we use the Voronoi cells of each agent to plan the local motion as well as a corridor selection strategy to limit the congestion in narrow passages. We evaluate the performance of our approach in complex scenes with tens of agents and narrow passages. We show that our Coupled Global-Local approach and Replanning (CGLR) improves the performance and efficiency over prior decentralized methods. In addition, our approach results in a higher success rate in terms of collision-free navigation to the goals, showing improvement in the range of 3-70% over prior decentralized solutions in certain scenarios.

The dynamic factors in the environment will lead to the decline of camera localization accuracy due to the violation of the static environment assumption of SLAM algorithm. Recently, some related works generally use the combination of semantic constraints and geometric constraints to deal with dynamic objects, but problems can still be raised, such as poor real-time performance, easy to treat people as rigid bodies, and poor performance in low dynamic scenes. In this paper, a dynamic scene-oriented visual SLAM algorithm based on object detection and coarse-to-fine static probability named CFP-SLAM is proposed. The algorithm combines semantic constraints and geometric constraints to calculate the static probability of objects, keypoints and map points, and takes them as weights to participate in camera pose estimation. Extensive evaluations show that our approach can achieve almost the best results in high dynamic and low dynamic scenarios compared to the state-of-the-art dynamic SLAM methods, and shows quite high real-time ability.

Reliable and stable 6D pose estimation of un-cooperative space objects plays an essential role in on-orbit servicing and debris removal missions. Considering that the pose estimator is sensitive to background interference, this paper proposes a counterfactual analysis framework named CA-SpaceNet to complete robust 6D pose estimation of the space-borne targets under complicated background. Specifically, conventional methods are adopted to extract the features of the whole image in the factual case. In the counterfactual case, a non-existent image without the target but only the background is imagined. Side effect caused by background interference is reduced by counterfactual analysis, which leads to unbiased prediction in final results. In addition, we also carry out low-bit-width quantization for CA-SpaceNet and deploy part of the framework to a Processing-In-Memory (PIM) accelerator on FPGA. Qualitative and quantitative results demonstrate the effectiveness and efficiency of our proposed method. To our best knowledge, this paper applies causal inference and network quantization to the 6D pose estimation of space-borne targets for the first time. The code is available at https://github.com/Shunli-Wang/CA-SpaceNet.

Quadrotors are agile platforms. With human experts, they can perform extremely high-speed flights in cluttered environments. However, fully autonomous flight at high speed remains a significant challenge. In this work, we propose a motion planning algorithm based on the corridor-constrained minimum control effort trajectory optimization (MINCO) framework. Specifically, we use a series of overlapping spheres to represent the free space of the environment and propose two novel designs that enable the algorithm to plan high-speed quadrotor trajectories in real-time. One is a sampling-based corridor generation method that generates spheres with large overlapped areas (hence overall corridor size) between two neighboring spheres. The second is a Receding Horizon Corridors (RHC) strategy, where part of the previously generated corridor is reused in each replan. Together, these two designs enlarge the corridor spaces in accordance with the quadrotor's current state and hence allow the quadrotor to maneuver at high speeds. We benchmark our algorithm against other state-of-the-art planning methods to show its superiority in simulation. Comprehensive ablation studies are also conducted to show the necessity of the two designs. The proposed method is finally evaluated on an autonomous LiDAR-navigated quadrotor UAV in woods environments, achieving flight speeds over 13.7m/s without any prior map of the environment or external localization facility.

This paper is about extremely robust and lightweight localisation using LiDAR point clouds based on instance segmentation and graph matching. We model 3D point clouds as fully-connected graphs of semantically identified components where each vertex corresponds to an object instance and encodes its shape. Optimal vertex association across graphs allows for full 6-Degree-of-Freedom (DoF) pose estimation and place recognition by measuring similarity. This representation is very concise, condensing the size of maps by a factor of 25 against the state-of-the-art, requiring only 3 kB to represent a 1.4 MB laser scan. We verify the efficacy of our system on the SemanticKITTI dataset, where we achieve a new state-of-the-art in place recognition, with an average of 88.4 % recall at 100 % precision where the next closest competitor follows with 64.9 %. We also show accurate metric pose estimation performance - estimating 6-DoF pose with median errors of 10cm and 0.33 deg.

As humans and robots start to collaborate in close proximity, robots are tasked to perceive, comprehend, and anticipate human partners' actions, which demands a predictive model to describe how humans collaborate with each other in joint actions. Previous studies either simplify the collaborative task as an optimal control problem between two agents or do not consider the learning process of humans during repeated interaction. This idyllic representation is thus not able to model human rationality and the learning process. In this paper, a bounded-rational and game-theoretical human cooperative model is developed to describe the cooperative behaviors of the human dyad. An experiment of a joint object pushing collaborative task was conducted with 30 human subjects using haptic interfaces in a virtual environment. The proposed model uses inverse optimal control (IOC) to model the reward parameters in the collaborative task. The collected data verified the accuracy of the predicted human trajectory generated from the bounded rational model excels the one with a fully rational model. We further provide insight from the conducted experiments about the effects of leadership on the performance of human collaboration.

Cultivation and weeding are two of the primary tasks performed by farmers today. A recent challenge for weeding is the desire to reduce herbicide and pesticide treatments while maintaining crop quality and quantity. In this paper we introduce BonnBot-I a precise weed management platform which can also performs field monitoring. Driven by crop monitoring approaches which can accurately locate and classify plants (weed and crop) we further improve their performance by fusing the platform available GNSS and wheel odometry. This improves tracking accuracy of our crop monitoring approach from a normalized average error of 8.3% to 3.5%, evaluated on a new publicly available corn dataset. We also present a novel arrangement of weeding tools mounted on linear actuators evaluated in simulated environments. We replicate weed distributions from a real field, using the results from our monitoring approach, and show the validity of our work-space division techniques which require significantly less movement (a 50% reduction) to achieve similar results. Overall, BonnBot-I is a significant step forward in precise weed management with a novel method of selectively spraying and controlling weeds in an arable field.

This work proposes a novel haptic data reduction scheme for time-delayed teleoperation by coding information as blocks. State-of-the-art (SOTA) haptic data reduction approaches are mainly sampled-based schemes. They encode haptic signals sample by sample in order to minimize the introduced coding delay. In contrast, our proposed block-based coding approach transmits a sample block as a single unit (haptic packet). Although it introduces additional algorithmic delays that are proportional to the block length, block coding has benefits since the packet rate is easy to control, the coding approach can be lossless, and the intra-block information can be employed to improve the force feedback quality. We further develop an energy adjustment approach that uses the information in a block to mitigate force oscillations caused by the Time Domain Passivity Approach. Simulation experiments and subjective tests demonstrate that our method reduces network load and significantly increases force feedback quality compared with the SOTA sample-based coding schemes, particularly for mid- to high-latency networks and low packet rates.

Passive exoskeletons are unpowered wearable robotic devices aimed at providing biomechanical assistance. They can be applied in industries such as manufacturing, construction and logistics to reduce repetitive stress injuries among workers. Their design process typically considers a static user, with muscle outputs computed later during dynamic tasks to evaluate performance. Attempting to reduce human muscle effort at this stage requires manual redesign. Instead, we propose a parameter optimization approach that minimizes muscle effort rates during realistic dynamic tasks in the design stage itself. We extract human kinematics in assembly tasks from an industry-oriented motion capture dataset, and compute the induced joint torques. Using a passive exoskeleton for shoulder joint gravity compensation from the literature as a baseline, we optimize its design parameters through a multi-objective Pareto Local Search, minimizing the muscle effort rates during these tasks. As the estimation of muscle outputs through biomechanical simulation techniques is computation-ally expensive, we train ensemble regression models for each muscle of interest during the task motions. These models serve as surrogates for the objective function in the design optimization procedure, speeding up search in the parameter space. The resulting exoskeleton with optimized design param-eters reduces estimated muscle effort rates by an average of 5.73% and peak of 35.1 % compared to default parameters, and an average of 14.5% and peak of 32.2% compared to not wearing an exoskeleton in overhead assembly tasks. A larger peak reduction compared to default parameters may be due to hindrance in motion caused by device. This approach may be adapted to other exoskeletons and applications, improving biomechanical assistance by design.

Natural eyeball motions in humanoid robots can contribute to friendly communication, thus improving the human-robot interaction. In this paper, we develop antagonist-agonist artificial muscles for humanoid eyeball motions, by using dielectric elastomer actuators (DEAs). Inspired by human eyeballs, the artificial muscles consist of two pairs of DEA: one pair for the horizontal motion, and the other for the vertical motion. The fabrication time of actuators can be significantly decreased due to their simple structure. The antagonist-agonist actuator outperforms the dielectric elastomer minimum energy structure in terms of actuation displacement and response time. We conduct experiments in a lifesize human face model. The experiments demonstrate the capability of antagonist-agonist artificial muscles to mimic eyeball motions in the horizontal, vertical, and diagonal directions. Future work includes modeling and control of artificial muscles for optimal performance of various humanoid eyeball motions.

Humans have an incredible sense of self-preservation that is both instilled, and also learned through experience. One system which contributes to this is the pain and reflex system which both minimizes damage through involuntary reflex actions and also serves as a means of 'negative reinforcement’ to allow learning of poor actions or decision. Equipping robots with a reflex system and parallel learning architecture could help to prolong their useful life and allow for continued learning of safe actions. Focusing on a specific mock-up scenario of cubes on a 'stove’ like setup, we investigate the hardware and learning approaches for a robotic manipulator to learn the presence of 'hot’ objects and its contextual relationship to the environment. By creating a reflex arc using analog electronics that bypasses the 'brain’ of the system we show an increase in the speed of release by at least two-fold. In parallel we have a learning procedure which combines visual information of the scene with this 'pain signal’ to learn and predict when an object may be hot, utilizing an object detection neural network. Finally, we are able to extract the learned contextual information of the environment by introducing a method inspired by 'thought experiments' to generate heatmaps that indicate the probability of the environment being hot.

Climbing robots have the potential to revolutionize the maintenance and inspection operations of many types of vertical structures. In nature, parrots exhibit a remarkable capacity for manipulation during climbing behaviors, for which robotics can benefit from studying. In this paper we present a novel tripedal robot that is inspired by the morphology of these impressive birds, which use their legs and beak in a tripedal fashion when climbing. We propose several foot placement, trajectory generation, and control methods for this system along with performance evaluation in simulation. A video of select simulations and live bird data is included in the supplementary material, and can also be found at https://youtu.be/vRVGralyQgQ.

We present a holistic grasping controller, combining free-space position control and in-contact force-control for reliable grasping given uncertain object pose estimates. Employing tactile fingertip sensors, undesired object displacement during grasping is minimized by pausing the finger closing motion for individual joints on first contact until force-closure is established. While holding an object, the controller is compliant with external forces to avoid high internal object forces and prevent object damage. Gravity as an external force is explicitly considered and compensated for, thus preventing gravity-induced object drift. We evaluate the controller in two experiments on the TIAGo robot and its parallel-jaw gripper proving the effectiveness of the approach for robust grasping and minimizing object displacement. In a series of ablation studies, we demonstrate the utility of the individual controller components.

Unsupervised domain adaptation (UDA) aims to learn domain-invariant representations between the labeled source domain and the unlabeled target domain. Existing self- training-based UDA methods use ground truth and pseudo- labels to supervise source data and target data respectively. However, strong supervision in the source domain and pseudo- label noise in the target domain lead to some problems, such as biased predictions and over-fitting. To tackle these issues, we propose a novel Bilateral Knowledge Distillation (BKD) framework for UDA in semantic segmentation, which adopts different knowledge distillation strategies depending on the domain. Specifically, we first introduce a Source-Flow Distillation (SD) to smooth the labels of source images, which weakens the supervision in the source domain. Meanwhile, a Target-Flow Distillation (TD) is designed to extract the inter- class knowledge in the probability map output from the teacher model, which alleviates the influence of pseudo-label noise in the target domain. Considering the class imbalance in semantic segmentation, we further propose an Image-Wise Hard Pixel Mining (HPM) to address this issue without estimating class frequency in the unlabeled target domain. The effectiveness of our framework against existing state-of-the-art methods is demonstrated by extensive experiments on two benchmarks: GTA5-to-Cityscapes and SYNTHIA-to-Cityscapes.

Professional human drivers usually have more than one driving strategy to handle incoming traffic situations. These different strategies activate different performance characteristics of the vehicle, enabling the driver to minimize the risk in a variety of situations by optimizing the strategy selection. In the same spirit, we define a novel concept of strategy-wise performance metric and creatively combine this performance metric with reachability analysis to evaluate candidate control strategies. Such a performance evaluation produces solid guarantees on which strategies will not qualify for the given traffic scenario. Then we automate the strategy selection process by weighing and minimizing the overall risk of each strategy candidate.