ICRA 2021 Papers

Mobile robots deployed in human-populated environments must be able to safely and comfortably navigate in close proximity to people. Head orientation and gaze are both mechanisms which help people to interpret where other people intend to walk, which in turn enables them to coordinate their movement. Head orientation has previously been leveraged to develop classifiers which are able to predict the goal of a person’s walking motion. Gaze is believed to generally precede head orientation, with a person quickly moving their eyes to a target and then following it with a turn of their head. This study leverages state-of-the-art virtual reality technology to place participants into a simulated environment in which their gaze and motion can be observed. The results of this study indicate that position, velocity, head orientation, and gaze can all be used as predictive features of the goal of a person’s walking motion. The results also indicate that gaze both precedes head orientation and can be used to predict the goal of a person’s walking motion at a higher level of accuracy earlier in their walking trajectory. These findings can be leveraged in the design of social navigation systems for mobile robots.

A hydraulic drive system can generate a large force and is impact resistant. Thus, robots that utilize hydraulic drive systems have been developed for use in disaster areas. In a hydraulic drive system, it is necessary to design the motion of the robot within the capacity of the pump unit, but it is generally difficult to predict the required flow rate and pressure. Especially for hydraulically driven legged robots, the periodic significant changes in the required flow rate and pressure can make it difficult to select a pump and design the leg operation within pump capacity. Therefore, most operations are not designed in full consideration of the characteristics of the pump. By designing the operation of the robot considering the characteristics of the pump, improvements in energy efficiency and working speed can be expected. In this study, we theoretically derive the relationship between the leg motion of the hydraulic leg robot and the required pressure and flow rate. Based on the results, a method for designing the operation considering the characteristics of the pump is proposed, and its validity is experimentally confirmed.

The ability to safely navigate through complex and cluttered environments is required for a wide range of robotics applications. This paper introduces a framework to compute safe global paths in maps represented as collections of 3D Signed Distance Function (SDF) submaps. Such maps are able to maintain global consistency in spite of odometry drift. However, computationally efficient global path planning in this context remains a challenging problem. We present a planning approach based on pre-computed local graphs, computed in each submap, that are linked to form a global path at planning time. To ensure globally safe paths, planning algorithms make frequent queries to the submap collection, which grows over time as the agent collects observational data. We present an efficient algorithm for performing these queries, through the use of a spatial hash table. We analyze the performance of our proposal extensively in simulation and real-world environments, and compare our approach to state- of-the-art planning approaches designed for monolithic maps, extended to submap-based maps. We show the efficacy of our method at adapting to global map deformations, while significantly reducing the planning time to an average of ~1.2 seconds, a reduction by 90 % compared to classical monolithic approaches.

This paper presents the voxelized generalized iterative closest point (VGICP) algorithm for fast and accurate three-dimensional point cloud registration. The proposed approach extends the generalized iterative closest point (GICP) approach with voxelization to avoid costly nearest neighbor search while retaining its accuracy. In contrast to the normal distributions transform (NDT), which calculates voxel distributions from point positions, we estimate voxel distributions by aggregating the distribution of each point in the voxel. The voxelization approach allows us to efficiently process the optimization in parallel, and the proposed algorithm can run at 30 Hz on a CPU and 120 Hz on a GPU. Through evaluations in simulated and real environments, we confirmed that the accuracy of the proposed algorithm is comparable to GICP, but is substantially faster than existing methods. This will enable the development of real-time 3D LIDAR applications that require extremely fast evaluations of the relative poses between LIDAR frames.

Volumetric objectives for exploration and perception tasks seek to capture a sense of value (or reward) for hypothetical observations at one or more camera views for robots operating in unknown environments. For example, a volumetric objective may reward robots proportionally to the expected volume of unknown space to be observed. We identify connections between existing information-theoretic and coverage objectives in terms of expected coverage, particularly that mutual information without noise is a special case of expected coverage. Likewise, we provide the first comparison, of which we are aware, between information-based approximations and coverage objectives for exploration, and we find, perhaps surprisingly, that coverage objectives can significantly outperform information-based objectives in practice. Additionally, the analysis for information and coverage objectives demonstrates that Randomized Sequential Partitions—a method for efficient distributed sensor planning—applies for both classes of objectives, and we provide simulation results in a variety of environments for as many as 32 robots.

Three-dimensional reconstruction in confined spaces is important for the manufacturing of aircraft wings, inspection of narrow pipes, examination of turbine blades, etc. It is also challenging because confined spaces tend to lack a positioning infrastructure, and conventional sensors often cannot detect objects in close range. Therefore, such tasks require a sensor that is compact, operates in short-range, and able to localize itself. In this paper, we introduce a miniature and low-cost 3D scanning system including an active laser-stripe triangulation hardware, integrated inertial sensors, and a Simultaneous Localization and Mapping (SLAM) software tailored for the sensor. The proposed system is capable of reconstructing photo-realistic 3D point cloud in real-time in spite of its compact monocular configuration. To achieve this capability, we propose an approach to capture both color and geometry using alternating shutter-speed on a single camera. A novel SLAM method is proposed to accurately localize the sensor by fusing laser, camera, and inertial measurements. Evaluation of localization accuracy and comparison on reconstruction performance against a significantly larger commercial off-the-shelf sensor demonstrate the proposed system’s advantages in real-world applications.

We propose a novel system to track human lower-body motion as part of a larger movement assessment system for clinical evaluation. Our system combines multiple wearable Inertial Measurement Unit (IMU) sensors and a single external RGB-D camera. We use a factor graph with a Sliding Window Filter (SWF) formulation that merges 2-D joint data extracted from the RGB images via a Deep Neural Network, raw depth information, raw IMU gyroscope readings, and estimated foot contacts extracted from IMU gyroscope and accelerometer data. For the system, we use an articulated model of human body motion based on differential manifolds. We compare the results of our system against a gold-standard motion capture system and a vision-only alternative. Our proposed system qualitatively presents smoother 3D joint trajectories when compared to noisy depth data, allowing for more realistic gait estimations. At the same time, with respect to the vision-only baseline, it improves the median of the joint trajectories by around 2cm, while considerably reducing outliers by up to 0.6m.

In this paper, we propose a framework for tracking the deformation of soft objects using a RGB-D camera by utilizing the physically-based model of the considered object. A coarse, 3D template of the object being tracked is the only prior information required by the proposed method. The proposed approach does not rely on the accurate knowledge of the material properties of the object being tracked. In this paper, we integrate computer vision based tracking methodology with physical model based deformation representation without requiring expensive numerical optimization for minimizing nonlinear error terms. The proposed approach enables deformation tracking by joint minimization of a geometric error and a direct photometric intensity error while utilizing co-rotational Finite Element Method (FEM) as the underlying deformation model. The proposed method has been validated both on synthetic data (with groundtruth) and real data.

Cable-driven parallel robots (CDPRs) are a type of parallel robots, where cables are used instead of rigid links. This leads to many advantages, such as large workspace, low mass in motion and simple reconfiguration. The drawbacks are accuracy issues and complex cable management. Indeed, it is usual that cables become slack. That can be caused by, for example, cable mass, uncertainties in the system, and a higher number of cables than the number of degrees of freedom of the moving-platform. This reduces CDPR stiffness and degree of actuation. While visual servoing provides good accuracy and is robust to different perturbations in the system and to modeling errors, it does not deal with cable slackness. Thus, a CDPR with visual servoing can become underactuated due to cable slack. We propose in this paper to enrich visual servoing with a tension correction algorithm. Experimental results show reduction of slackness and thus avoiding slackness-related trajectory perturbations and loss of stability.

Highly accurate and robust localization ability is of great importance for autonomous vehicles (AVs) in urban scenarios. Traditional vision-based methods suffer from lost due to illumination, weather, viewing and appearance changes. In this paper we propose a novel visual semantic localization algorithm based on HD map and semantic features which are compact in representation. Semantic features are widely appeared on urban roads, and are robust to illumination, weather, viewing and appearance changes. The repeated structures, missed detections and false detections make data association (DA) highly ambiguous. To this end, a robust DA method considering local structural consistency, global pattern consistency and temporal consistency is performed. Further, we introduce a sliding window factor graph optimization framework to fuse association and odometry measurements without the requirements of high-precision absolute height information for map features.We evaluate the proposed localization framework on both simulated and real urban road. The experiments show that the proposed approach is able to achieve highly accurate localization with a mean longitudinal error of 0.43m, a mean lateral error of 0.12m and a mean yaw angle error of 0.11°.

Visual Place Recognition (VPR) is a crucial component for long-term mobile robot autonomy. In this paper, we exploit a coarse-to-fine paradigm to recognize places. In particular, we first select candidate frames for each query image, and then check the spatial geometric relationship between the query and its candidate frames to determine the final place match. In the coarse match stage, we employ the deep learning network to extract global features that encode semantic information of images, then by comparing the similarity between features to obtain a candidate list of the query place. In the fine match stage, we propose an effective and efficient feature matching algorithm for real-time geometrical verification of candidate places, termed as local affine preserving matching (LAP). Extensive experimental results demonstrate that our LAP can significantly promote the VPR performance, and the proposed overall VPR method can achieve much better performance over the current state-of-the-art approaches.

In order to engage in complex social interaction, humans learn at a young age to infer what others see and cannot see from a different point-of-view, and learn to predict others’ plans and behaviors. These abilities have been mostly lacking in robots, sometimes making them appear awkward and socially inept. Here we propose an end-to-end long-term visual prediction framework for robots to begin to acquire both these critical cognitive skills, known as Visual Perspective Taking (VPT) and Theory of Behavior (TOB). We demonstrate our approach in the context of visual hide-and-seek – a game that represents a cognitive milestone in human development. Unlike traditional visual predictive model that generates new frames from immediate past frames, our agent can directly predict to multiple future timestamps (25 s), extrapolating by 175% beyond the training horizon. We suggest that visual behavior modeling and perspective taking skills will play a critical role in the ability of physical robots to fully integrate into real-world multi-agent activities.

We propose a vision-based architecture search algorithm for robot manipulation learning, which discovers interactions between low dimension action inputs and high dimensional visual inputs. Our approach automatically designs architectures while training on the task – discovering novel ways of combining and attending image feature representations with actions as well as features from previous layers. The obtained new architectures demonstrate better task success rates, in some cases with a large margin, compared to a recent high performing baseline. Our real robot experiments also confirm that it improves grasping performance by 6%. This is the first approach to demonstrate a successful neural architecture search and attention connectivity search for a real-robot task.

Due to the head swinging and the body winding, the self-localization and path following for snake-like robots based on vision are very challenging. In this paper, a novel pantilt compensation method and curve parameter compensation path following controller are proposed to solve these problems, which can achieve high-precision path following. More specifically, to realize real-time positioning of the snakelike robot, a camera-mounted pan-tilt is equipped on the head of the snake-like robot, and the Apritag detection is used after the angle between the Apriltag plane and the camera plane being compensated. Then, due to the modeling approximation, the snake-like robot with a simplified controller usually deviates from the desired path, so a curve parameter compensation path following controller is proposed to eliminate the deviation and improve the following accuracy. Finally, the experiments show that with proposed methods, the snake-like robot can converge to the desired path stably and accurately.

Interest in soft continuum arms has increased as their inherent material elasticity enables safe and adaptive interactions with the environment. However to achieve full autonomy in these arms, accurate three-dimensional shape sensing is needed. Vision-based solutions have been found to be effective in estimating the shape of soft continuum arms. In this paper, a vision-based shape estimator that utilizes a geometric strain based representation for the soft continuum arm’s shape, is proposed. This representation reduces the dimension of the curved shape to a finite set of strain basis functions, thereby allowing for efficient optimization for the shape that best fits the observed image. Experimental results demonstrate the effectiveness of the proposed approach in estimating the end effector with accuracy less than the soft arm’s radius. Multiple basis functions are also analyzed and compared for the specific soft continuum arm in use.

Increasing the robustness of grasping actions and the recovery from failure is key to improving a robot’s autonomy. Endowing robots with the ability to robustly grasp and manipulate unknown difficult objects such as stones is required for sample collection in unknown environments. In this paper, we present a complete system for robust grasping of stones, which integrates stone segmentation based on depth information, the generation of grasp hypotheses and pushing actions as well as their execution. In particular, our system has been designed to solve these tasks on robots with limited computing resources. We evaluate the performance in real robot experiments in the context of stone sample collection. The results show that such a challenging task is achievable under computing resource constraints.

Obstacle avoidance is a fundamental and challenging problem for autonomous navigation of mobile robots. In this paper, we consider the problem of obstacle avoidance in simple 3D environments where the robot has to solely rely on a single monocular camera. In particular, we are interested in solving this problem without relying on localization, mapping, or planning techniques. Most of the existing work consider obstacle avoidance as two separate problems, namely obstacle detection, and control. Inspired by the recent advantages of deep reinforcement learning in Atari games and understanding highly complex situations in Go, we tackle the obstacle avoidance problem as a data-driven end-to-end deep learning approach. Our approach takes raw images as input and generates control commands as output. We show that discrete action spaces are outperforming continuous control commands in terms of expected average reward in maze-like environments. Furthermore, we show how to accelerate the learning and increase the robustness of the policy by incorporating predicted depth maps by a generative adversarial network.

Humans in contrast to robots are excellent in performing fine manipulation tasks owing to their remarkable dexterity and sensorimotor organization. Enabling robots to acquire such capabilities, necessitates a framework that not only replicates the human behaviour but also integrates the multi-sensory information for autonomous object interaction. To address such limitations, this research proposes to augment the previously developed kernelized synergies framework with visual perception to automatically adapt to the unknown objects. The kernelized synergies, inspired from humans, retain the same reduced subspace for object grasping and manipulation. To detect object in the scene, a simplified perception pipeline is used that leverages the RANSAC algorithm with Euclidean clustering and SVM for object segmentation and recognition respectively. Further, the comparative analysis of kernelized synergies with other state of art approaches is made to confirm their flexibility and effectiveness on the robotic manipulation tasks. The experiments conducted on the robot hand confirm the robustness of modified kernelized synergies framework against the uncertainties related to the perception of environment.

During the planning phase of industrial robot workplaces, hazard analyses are required so that potential hazards for human workers can be identified and appropriate safety measures can be implemented. Existing hazard analysis methods use human reasoning, checklists and/or abstract system models, which limit the level of detail. We propose a new approach that frames hazard analysis as a search problem in a dynamic simulation environment. Our goal is to identify workplace hazards by searching for simulation sequences that result in hazardous situations. We solve this search problem by placing virtual humans into workplace simulation models. These virtual humans act in an adversarial manner: They learn to provoke unsafe situations, and thereby uncover workplace hazards. Although this approach cannot replace a thorough hazard analysis, it can help uncover hazards that otherwise may have been overlooked, especially in early development stages. Thus, it helps to prevent costly re-designs at later development stages. For validation, we performed hazard analyses in six different example scenarios that reflect typical industrial robot workplaces.