IROS 2021 论文列表

Dense image alignment works by minimizing the photometric error of two images since it is assumed that the illumination changes between images close in time remain the same—this is what is called the brightness constancy assumption. However, this assumption does not hold with long-term maps since illumination changes continually from day to day (morning, afternoon, evening) and is dependent on certain external conditions like weather or even seasons. In this work, we present an image registration algorithm based on the Normalized Information Distance (NID) that is shown to be robust to extreme illumination changes comparing to the traditional direct methods. The pose is estimated by minimizing the NID function with the help of the nonlinear least square optimization library G2O. We share our source code 1 (CPU and GPU version) for the benefit of the community, which can be a strong basis for future tracking and mapping system based on NID.

Learning robust policies for robotic systems operating in presence of uncertainty is a challenging task. For safe navigation, in addition to the natural stochasticity of the environment and vehicle dynamics, the perception uncertainty associated with dynamic entities, e.g. pedestrians, must be accounted for during motion planning. To this end, we construct an algorithm with built-in robustness to uncertainty by directly minimizing an upper confidence bound on the expected cost of trajectories instead of employing a standard approach based on minimizing the expected cost itself. Perception uncertainty is incorporated into the policy search framework by predicting each pedestrian’s intent belief and propagating their state distribution in time using closed-loop goal-directed dynamics. We train the policy in simulation and show that it could be transferred to an agile ground vehicle for successful autonomous robot navigation in presence of pedestrians with perception uncertainty. We further show the superior performance of this policy over a policy that does not consider pedestrian intent and perception uncertainty.

We propose a multi-camera simultaneous localization and mapping (SLAM) system using the Manhattan constraint to support automated valet parking. The proposed method uses multiple cameras to expand the system field of view, to improve the robustness of the SLAM system in textureless regions, where point features from different cameras are jointly optimized by a uniform cost function. To improve global map scale consistency, we utilize wheel odometer in the system initialization and the multi-camera cost function. In addition, we introduce the Manhattan world assumption, an abstraction of a man-made environment, into the proposed algorithm, to improve its estimation processes and make it suitable for the multi-camera SLAM system. The Manhattan world assumption is used to estimate the camera rotation by line features in the image and provide a global orientation constraint that increases the mapping accuracy. The proposed algorithm demonstrates stability in low-texture regions and achieves superior accuracy in experiments conducted in multistory parking lots, compared with other algorithms including monocular and multi-camera versions. Regarding efficiency, the proposed algorithm processes twice the number of measurements with 50% additional computation time while maintaining SLAM stability under a textureless environment.

Many autonomous driving applications nowadays come along with a prebuilt vector map for routing and planning purposes. In order to localize on this map, traditional LiDAR localization methods usually require a separate localization layer to function. On one hand, the separate layer occupies large storage and is not convenient to update. On the other hand, the potential of the vector map itself has not been fully exploited by existing methods. In this paper, we present a LiDAR localization system that leverages the vector map directly as the localization layer. A semantic extraction module is developed to match the heterogeneous data between LiDAR measurements and the 3D vector elements. A local map maintenance module is introduced to keep the system function robustly when there are not enough vector matches. The system adopts an optimization-based framework and infers 6-DOF poses. Experiments show that the proposed system is able to achieve centimeter accuracy robustly in both highway and urban environments, without a separate localization layer.

This paper proposes a robust initialization method for a multi-camera visual SLAM system where cameras have only a limited common field of views and inaccurate extrinsic calibration. The limited common field of views leads to only a few common features that can be matched between cameras. Inaccurate extrinsic poses, caused by vibrations or misplacement of cameras after offline calibration, make it even harder for triangulating the seed 3D points to initialize the SLAM system successfully. Instead of taking the extrinsic parameters as constants for feature matching and 3D point triangulation as most multi-camera systems did, we propose to take the inaccurate extrinsic poses as soft constraints to accommodate the calibration errors. Our initialization method consists of two stages by matching across different cameras and between two key frames. Both stages involve optimizing the cost functions that contain the extrinsic pose priors from inaccurate calibration parameters. By incorporating those soft pose constraints, we may avoid false feature matching and triangulation caused by inaccurate extrinsic parameters, while keeping solution space limited when only a few feature correspondences exist. The results in real-world tests show that such a simple solution can improve the success rate of SLAM initialization notably, even when the pose priors from the offline calibration differ significantly from the real ones.

Unknown properties of a robot’s environment are one of the sources of uncertainty in autonomous navigation. This uncertainty has to be accounted for when modelling robot dynamics. For ground vehicles in particular, terrain structure is one of the main environmental factors that can strongly influence the dynamics. Therefore, to ensure the ability of a robot to safely and efficiently navigate new environments, robust motion planning and control systems are needed. This paper investigates a data-driven approach to planning and control based on construction of robust motion primitives (MPs) and corresponding feedback rules that ensure a bounded error along the planned trajectory. The approach is tested in an exploration scenario in which a robot systematically inspects an area consisting of several terrain types with the aim of recognizing changes in dynamical properties, learning new dynamics models when such changes are detected and recording that information for future use. The advantage of incorporating the collected data into motion planning in multi-terrain environments is illustrated via simulation.

In this paper, a hierarchical and robust framework for learning bipedal locomotion is presented and successfully implemented on the 3D biped robot Digit built by Agility Robotics. We propose a cascade-structure controller that combines the learning process with intuitive feedback regulations. This design allows the framework to realize robust and stable walking with a reduced-dimensional state and action spaces of the policy, significantly simplifying the design and increasing the sampling efficiency of the learning method. The inclusion of feedback regulation into the framework improves the robustness of the learned walking gait and ensures the success of the sim-to-real transfer of the proposed controller with minimal tuning. We specifically present a learning pipeline that considers hardware-feasible initial poses of the robot within the learning process to ensure the initial state of the learning is replicated as close as possible to the initial state of the robot in hardware experiments. Finally, we demonstrate the feasibility of our method by successfully transferring the learned policy in simulation to the Digit robot hardware, realizing sustained walking gaits under external force disturbances and challenging terrains not incurred during the training process. To the best of our knowledge, this is the first time a learning-based policy is transferred successfully to the Digit robot in hardware experiments.

This paper proposes a novel dictionary learning approach to detect event anomalities using skeletal information extracted from RGBD video. The event action is represented as several latent action atoms and composed of latent spatial and temporal attributes. We aim to construct a network able to learn from few examples and also rules defined by the user. The skeleton frames are clustered by an initial K-means method. Each skeleton frame is assigned with a varying weight parameter and fed into our Gradual Online Dictionary Learning (GODL) algorithm. During the training process, outlier frames will be gradually filtered by reducing the weight that is inversely proportional to a cost. To strictly distinguish the event action from similar actions and robustly acquire its action units, we build a latent unit temporal structure for each sub-action.We validate the method at the example of fall event detection on NTU RGB+D dataset, because it provides a benchmark available for comparison. We present the experimental validation of the achieved accuracy, recall, and precision. Our approach achieves the best performance in precision and accuracy of human fall event detection, compared with other existing dictionary learning methods. Our method remains the highest accuracy and the lowest variance, with increasing noise ratio.

Imitation learning (IL) frameworks in robotics typically assume that a domain expert's demonstration always contains a correct way of doing the task. Despite its theoretical convenience, this assumption has limited practical values for an IL-powered robot in real world. There are many reasons for an expert in the real world to provide demonstrations that may contain incorrect or potentially unsafe way of doing a task. In order for IL-powered robots to work in the real world, IL frameworks need to detect such adversarial demonstrations and not learn from them. This paper proposes an IL framework that can autonomously detect and remove adversarial demonstrations, if they exist in the demonstration set, as it directly learns a task policy from the expert. The proposed framework that we term Robust Maximum Entropy behavior cloning (R-MaxEnt) learns a stochastic model that maps states to actions. In doing so, R-MaxEnt solves a minmax problem that leverages the entropy of the model to assign weights to different demonstrations while assigning poor weights to adversarial samples. Our empirical results show that R-MaxEnt outperforms the existing IL approaches in both real and simulated robotics tasks.

Reasoning about potential occlusions is essential for robots to efficiently predict whether an object exists in an environment. Though existing work shows that a robot with active perception can achieve various tasks, it is still unclear if occlusion reasoning can be achieved. To answer this question, we introduce the task of robotic object existence prediction: when being asked about an object, a robot needs to move as few steps as possible around a table with randomly placed objects to predict whether the queried object exists. To address this problem, we propose a novel recurrent neural network model that can be jointly trained with supervised and reinforcement learning methods using a curriculum training strategy. Experimental results show that 1) both active perception and occlusion reasoning are necessary to successfully achieve the task; 2) the proposed model demonstrates a good occlusion reasoning ability by achieving a similar prediction accuracy to an exhaustive exploration baseline while requiring only about 10% of the baseline’s number of movement steps on average; and 3) the model generalizes to novel object combinations with a moderate loss of accuracy.

The problem of robotic lime picking is challenging; lime plants have dense foliage which makes it difficult for a robotic arm to grasp a lime without coming in contact with leaves. Existing approaches either do not consider leaves, or treat them as obstacles and completely avoid them, often resulting in undesirable or infeasible plans. We focus on reaching a lime in the presence of dense foliage by considering the leaves of a plant as permeable obstacles with a collision cost. We then adapt the rapidly exploring random tree star (RRT*) algorithm for the problem of fruit harvesting by incorporating the cost of collision with leaves into the path cost. To reduce the time required for finding low-cost paths to goal, we bias the growth of the tree using an artificial potential field (APF). We compare our proposed method with prior work in a 2-D environment and a 6-DOF robot simulation. Our experiments and a real-world demonstration on a robotic lime picking task demonstrate the applicability of our approach.

Bladed tools such as jigsaws are common tools for wood workers on job-sites and in workshops, but do not currently have sufficient autonomous hardware or path planning algorithms to enable automation. Here we present a system of an autonomous robot and a path planning algorithm for automating jigsaw operations. The robot can drill holes, insert the jigsaw, and cut plywood. Our algorithm converts complex shapes into paths for the jigsaw, drill holes, and traversal movements for the robot. The algorithm decomposes input shapes into cuttable sections and determines possible locations for drilling entry holes for inserting the blade. We cast the drill hole problem as a set coverage problem with a trade-off between number of holes and cutting distance. We characterize the algorithm on a series of shapes and determined the algorithm found valid solutions. We executed an example on the robot to demonstrate the end-to-end system.

For the visually impaired people, some outdoor activities like running or soccer are difficult, due to not being able to clearly see the environment. Recently, multiple researchers have contributed to help the visually impaired people run outdoors using robotic systems with different types of feedback, such as auditory feedback and haptic feedback. They discovered that using robotic systems can be an effective way to guide visually impaired people while exercising outdoors. In this paper, we propose a method to guide the visually impaired people to do sports outside using a robotic system with haptic feedback, and we evaluate the feasibility of the proposed system through experiments with blindfolded users running outdoors. In the running guidance task, the position of the runner is determined from the visual feed of a drone, and haptic feedback produced on the users’ left lower leg is used to convey to the runner the directions in which to move to remain on a specific path. Additionally, we compared the performance of users under different haptic feedback modalities in the running task. The three compared modalities are: producing vibration only during the swing phase, only during stance phase or producing vibration continuously. The experimental results for a running task showed that our system enabled users to remain inside a specified track 93% of the total running time, while the ratio decreased to 79%, 77%, and 61% when receiving vibrations during only swing phase, during only stance phase, and without using any feedback respectively. We also observed users felt safer while running blindfolded by using the proposed method.

In this paper, we propose a robotic ultrasound imaging method that scans the breast in two separate phases to acquire high-quality ultrasound images. Our proposed system controls five Degrees of Freedom (DoFs) of the robot that hold an ultrasound probe to perform precise scanning. This system finds the desired trajectory based on geometrical analysis of the target inside the breast in a pre-scan phase and uses this information to control the probe in a post-scan phase. The proposed method updates the desired values of rotational and translational movement of the probe in the post-scan by calculating the center of mass of segmented target in each acquired frame and the average of image confidence map. The proposed method has been tested experimentally on a plastisol phantom. Given a specific trajectory, the position and orientation of the probe have been controlled at each point of the trajectory. The experiments’ result shows us that our proposed visual servoing algorithm successfully controls the probe to look at target tissue and is fast enough for use in a robotic control loop.

Rotating a grasped object about all three spatial axes is challenging, because kinematically redundant robot hands require complex control schemes for within-hand rotations, and simple parallel grippers require inefficient whole arm motions. We present a novel 3-finger robot hand design inspired by a spherical parallel mechanism that achieves these rotations with just 3 actuators. The hand is designed such that at every hand-object configuration, the object pose moves along a sphere with a fixed center, which is determined by the intersection of the fingers’ revolute axes and is independent of the object shape, pose, and the initial grasp configuration. We optimize the hand based on 3-RRS spherical manipulator to maximize both its rotational workspace size and manipulation motion quality. From these parameters, we implement and experimentally evaluate the hand design through grasping tests, manipulation characterization, and real-world task scenarios, which show that the hand is able to grasp a variety of object geometries and accomplish precise single and multi-DOF rotations about a fixed point. We believe this design can remarkably improve robustness and simplify control for dexterous within-hand rotations, which finds utility in augmenting the capabilities of low-DOF robot arms without an active wrist.

Long-term deployment of a fleet of mobile robots requires reliable and secure two-way communication channels between individual robots and remote human operators for supervision and tasking. Existing open-source solutions to this problem degrade in performance in challenging real-world situations such as intermittent and low-bandwidth connectivity, do not provide security control options, and can be computationally expensive on hardware-constrained mobile robot platforms. In this paper, we present Robofleet, a lightweight open-source system which provides inter-robot communication, remote monitoring, and remote tasking for a heterogenous fleet of ROS-enabled service-mobile robots that is designed with the practical goals of resilience to network variance and security control in mind.Robofleet supports multi-user, multi-robot communication via a central server. This architecture deduplicates network traffic between robots, significantly reducing overall network load when compared with native ROS communication. This server also functions as a single entrypoint into the system, enabling security control and user authentication. Individual robots run the lightweight Robofleet client, which is responsible for exchanging messages with the Robofleet server. It automatically adapts to adverse network conditions through backpressure monitoring as well as topic-level priority control, ensuring that safety-critical messages are successfully transmitted. Finally, the system includes a web-based visualization tool that can be run on any internet-connected, browser-enabled device to monitor and control the fleet.We compare Robofleet to existing methods of robotic communication, and demonstrate that it provides superior resilience to network variance while maintaining performance that exceeds that of widely-used systems.

We consider the problem of generating a fixed path for a mobile observer in a polygonal environment that can maintain a line-of-sight with an unpredictable target. In contrast to purely off-line or on-line techniques, we propose a hierarchical tracking strategy in which an off-line path generation technique based on a RRT is coupled with an online feedback-control technique to generate trajectories for the mobile observer.

We propose a novel autonomous driving frame-work that leverages graph-based features of roads, such as road positions and connections. The proposed method is divided into two parts: a low-level controller which follows the trajectory calculated by a graph-based path planner, and a high-level controller which determines the speed of the vehicle to follow the traffic flow. The high-level controller uses a road graphical neural network (Road-GNN), which encodes a road graph into latent features to perceive the surrounding environment. We use a 3D driving simulator to test the performance of Road-GNN, which is implemented based on the satellite image data of 30 roundabout intersections. To show that the proposed method can be generalized to various road environments, the proposed method is tested using roundabouts which are different from the training set. In the experiment, the proposed method successfully trains the agent and drives an ego-vehicle through various roundabout environments. The results show that the graph-based method is effective for autonomous driving.

The use of local detectors and descriptors in typical computer vision pipelines works well until variations in viewpoint and appearance change become extreme. Past research in this area has typically focused on one of two approaches to this challenge: the use of projections into spaces more suitable for feature matching under extreme viewpoint changes, and attempting to learn features that are inherently more robust to viewpoint change. In this paper, we present a novel framework that combines the learning of invariant descriptors through data augmentation and orthographic viewpoint projection. We propose rotation-robust local descriptors, learnt through training data augmentation based on rotation homographies, and a correspondence ensemble technique that combines vanilla feature correspondences with those obtained through rotation-robust features. Using a range of benchmark datasets as well as contributing a new bespoke dataset for this research domain, we evaluate the effectiveness of the proposed approach on key tasks including pose estimation and visual place recognition. Our system outperforms a range of baseline and state-of-the-art techniques, including enabling higher levels of place recognition precision across opposing place viewpoints, and achieves practically useful performance levels even under extreme viewpoint changes. We reduce pose estimation error by 86.72% relative to state of the art.