ICRA 2021 论文列表

We present a model predictive controller (MPC) that automatically discovers collision-free locomotion while simultaneously taking into account the system dynamics, friction constraints, and kinematic limitations. A relaxed barrier function is added to the optimization’s cost function, leading to collision avoidance behavior without increasing the problem’s computational complexity. Our holistic approach does not require any heuristics and enables legged robots to find whole-body motions in the presence of static and dynamic obstacles. We use a dynamically generated euclidean signed distance field for static collision checking. Collision checking for dynamic obstacles is modeled with moving cylinders, increasing the responsiveness to fast-moving agents. Furthermore, we include a Kalman filter motion prediction for moving obstacles into our receding horizon planning, enabling the robot to anticipate possible future collisions. Our experiments1 demonstrate collision-free motions on a quadrupedal robot in challenging indoor environments. The robot handles complex scenes like overhanging obstacles and dynamic agents by exploring motions at the robot’s dynamic and kinematic limits.

Grasping a novel target object in constrained environments (e.g., walls, bins, and shelves) requires intensive reasoning about grasp pose reachability to avoid collisions with the surrounding structures. Typical 6-DoF robotic grasping systems rely on the prior knowledge about the environment and intensive planning computation, which is ungeneralizable and inefficient. In contrast, we propose a novel Collision-Aware Reachability Predictor (CARP) for 6-DoF grasping systems. The CARP learns to estimate the collision-free probabilities for grasp poses and significantly improves grasping in challenging environments. The deep neural networks in our approach are trained fully by self-supervision in simulation. The experiments in both simulation and the real world show that our approach achieves more than 75% grasping rate on novel objects in various surrounding structures. The ablation study demonstrates the effectiveness of the CARP, which improves the 6-DoF grasping rate by 95.7%.

Obstacle avoidance is crucial for autonomous surface vehicles (ASVs) in the sea because rescue is extremely difficult there. OceanVoy, a sailboat toward long range energy-saving voyage, has to overcome the dual challenges, i.e. from the environmental interference and its low mobility preventing from precise obstacle avoidance. We propose a control scheme based on real-time collision risk assessment and a hybrid propulsion system to enhance safety of OceanVoy. A novel sailboat safety zone (SSZ) has been designed to warn the potential collision during its sailing. Both intrinsic characteristics of OceanVoy and environmental factors have been considered in SSZ. We use lateral and axial thrusters to provide emergency propulsion. A collision avoidance algorithm is executed to coordinate motors in rudder, sail and thrusters based on SSZ. Both simulation and experiments have been conducted and the results have validated our system and collision avoidance scheme.

Physical human-robot interaction is known to be a crucial aspect in modern lightweight robotics. Herein, the estimation of external interactions is essential for the effective and safe collaboration. In this work, an extended momentum-based disturbance observer is presented which includes the sensing redundancy related to additional force-torque measurements. The observer eliminates the need for acceleration measurements/estimates and it is able to accurately reconstruct multiple simultaneous contact locations. Moreover, it provides uncoupled, configuration-independent, and singularity-free estimates of the external forces. The performance of the approach is experimentally validated on the SARA robot, the new generation of DLR lightweight robots, involving high resolution force-torque sensors in a redundant arrangement.

We present a hierarchical control approach for maneuvering an autonomous vehicle (AV) in tightly-constrained environments where other moving AVs and/or human driven vehicles are present. A two-level hierarchy is proposed: a high-level data-driven strategy predictor and a lower-level model-based feedback controller. The strategy predictor maps an encoding of a dynamic environment to a set of high-level strategies via a neural network. Depending on the selected strategy, a set of time-varying hyperplanes in the AV’s position space is generated online and the corresponding halfspace constraints are included in a lower-level model-based receding horizon controller. These strategy-dependent constraints drive the vehicle towards areas where it is likely to remain feasible. Moreover, the predicted strategy also informs switching between a discrete set of policies, which allows for more conservative behavior when prediction confidence is low. We demonstrate the effectiveness of the proposed data-driven hierarchical control framework in a two-car collision avoidance scenario through simulations and experiments on a 1/10 scale autonomous car platform where the strategy-guided approach outperforms a model predictive control baseline in both cases.

We present a decentralized control algorithm for robots to aid in carrying an unknown load. Coordination occurs solely through sensing of the forces on or movement of the shared load. Robots prevent undesired motion of the load while permitting movement in the task-relevant subspace, and stabilize against unexpected events by a transient decrease in compliance. The algorithm requires no direct communication between agents, and minimal knowledge of the system or task. We demonstrate the approach in simulation using a commercially available compliant robotic platform.

The efficiency and accuracy of mapping are crucial in a large scene and long-term AR applications. Multi-agent cooperative SLAM is the precondition of multi-user AR interaction. The cooperation of multiple smart phones has the potential to improve efficiency and robustness of task completion and can complete tasks that a single agent cannot do. However, it depends on robust communication, efficient location detection, robust mapping, and efficient information sharing among agents. We propose a multi-intelligence collaborative monocular visual-inertial SLAM deployed on multiple ios mobile devices with a centralized architecture. Each agent can independently explore the environment, run a visual-inertial odometry module online, and then send all the measurement information to a central server with higher computing resources. The server manages all the information received, detects overlapping areas, merges and optimizes the map, and shares information with the agents when needed. We have verified the performance of the system in public datasets and real environments. The accuracy of mapping and fusion of the proposed system is comparable to VINS-Mono which requires higher computing resources.

Most of the existing saliency prediction research focuses on either single images or videos (or more precisely multiple images in sequence). However, to apply saliency prediction to drone exploration that has to consider multiple images from different view angles or localizations to determine the direction to explore, saliency prediction over multiple discontinuous images is required. In this paper, we propose a deep relative saliency model (MS-Net) for such an application. MS-Net starts with a single-image saliency feature extraction network for each image separately and then integrate these images by using a GCN-based mechanism called multi-image saliency fusion that learns relative saliency information among all the images. Finally, it predicts the saliency of each image by considering the relative information. Because there are no existing saliency prediction datasets with such multiple discontinuous images, we randomly cropped a large number of sub-images from 360° images of the existing 360° image saliency datasets to build our own dataset for both training and evaluation. Experimental results showed that the proposed MSNet considerably outperformed both single-image and video saliency prediction methods and could achieve comparative performance to that of 360° image saliency prediction even with only limited field-of-views, i.e., five sub-images, considered.

Older adults who age in place face many health problems and need to be taken care of. Fall is a serious problem among elderly people. In this paper, we present the design and implementation of collaborative fall detection using a wearable device and a companion robot. First, we developed a wearable device by integrating a camera, an accelerometer and a microphone. Second, a companion robot communicates with the wearable device to conduct collaborative fall detection. The robot is also able to contact caregivers in case of emergency. The collaborative fall detection method consists of motion data based preliminary detection on the wearable device and video-based final detection on the companion robot. Both convolutional neural network (CNN) and long short-term memory (LSTM) are used for video-based fall detection. The experimental results show that the overall accuracy of video-based algorithm is 84%. We also investigated the relation between the accuracy and the number of image frames. Our method improves the accuracy of fall detection while maximizing the battery life of the wearable device. In addition, our method significantly increases the sensing range of the companion robot.

This paper aims to mitigate straggler effects in synchronous distributed learning for multi-agent reinforcement learning (MARL) problems. Stragglers arise frequently in a distributed learning system, due to the existence of various system disturbances such as slow-downs or failures of compute nodes and communication bottlenecks. To resolve this issue, we propose a coded distributed learning framework, which speeds up the training of MARL algorithms in the presence of stragglers, while maintaining the same accuracy as the centralized approach. As an illustration, a coded distributed version of the multi-agent deep deterministic policy gradient (MADDPG) algorithm is developed and evaluated. Different coding schemes, including maximum distance separable (MDS) code, random sparse code, replication-based code, and regular low density parity check (LDPC) code are also investigated. Simulations in several multi-robot problems demonstrate the promising performance of the proposed framework.

In this work, we present a lightweight, tightly-coupled deep depth network and visual-inertial odometry (VIO) system, which can provide accurate state estimates and dense depth maps of the immediate surroundings. Leveraging the proposed lightweight Conditional Variational Autoencoder (CVAE) for depth inference and encoding, we provide the network with previously marginalized sparse features from VIO to increase the accuracy of initial depth prediction and generalization capability. The compact representation of dense depth, termed depth code, can be updated jointly with navigation states in a sliding window estimator in order to provide the dense local scene geometry. We additionally propose a novel method to obtain the CVAE’s Jacobian which is shown to be more than an order of magnitude faster than previous works, and we additionally leverage First-Estimate Jacobian (FEJ) to avoid recalculation. As opposed to previous works that rely on completely dense residuals, we propose to only provide sparse measurements to update the depth code and show through careful experimentation that our choice of sparse measurements and FEJs can still significantly improve the estimated depth maps. Our full system also exhibits state-of-the-art pose estimation accuracy, and we show that it can run in real-time with single-thread execution while utilizing GPU acceleration only for the network and code Jacobian.

We introduce a simple new method for visual imitation learning, which allows a novel robot manipulation task to be learned from a single human demonstration, without requiring any prior knowledge of the object being interacted with. Our method models imitation learning as a state estimation problem, with the state defined as the end-effector’s pose at the point where object interaction begins, as observed from the demonstration. By modelling a manipulation task as a coarse, approach trajectory followed by a fine, interaction trajectory, this state estimator can be trained in a self-supervised manner, by automatically moving the end-effector’s camera around the object. At test time, the end-effector is moved to the estimated state through a linear path, at which point the demonstration’s end-effector velocities are simply repeated, enabling convenient acquisition of a complex interaction trajectory without actually needing to explicitly learn a policy. Real-world experiments on 8 everyday tasks show that our method can learn a diverse range of skills from just a single human demonstration, whilst also yielding a stable and interpretable controller. Videos at: www.robot-learning.uk/coarse-to-fine-imitation-learning.

Existing work on sequential manipulation planning and trajectory optimization typically assumes the robot, environment and tools to be given. However, in particular in industrial applications, it is highly interesting to ask, what would be an optimal robot design, tool shape, or robot station geometry for a particular ensemble of manipulation tasks. To tackle this problem we propose a formulation to jointly optimize over static design parameters and the sequential manipulation trajectory. We can include optimization objectives such as penalizing velocities (path length) and joint torques. Our evaluations show that design optimization can significantly improve on such metrics. For instance, in a wrench tool demonstration scenario we show that the shape of the wrench tool as well as design of the robot can be optimized to allow for exerting a necessary external torque with minimal effort.

It is often desired to train 6D pose estimation systems on synthetic data because manual annotation is expensive. However, due to the large domain gap between the synthetic and real images, synthesizing color images is expensive. In contrast, this domain gap is considerably smaller and easier to fill for depth information. In this work, we present a system that regresses 6D object pose from depth information represented by point clouds, and a lightweight data synthesis pipeline that creates synthetic point cloud segments for training. We use an augmented autoencoder (AAE) for learning a latent code that encodes 6D object pose information for pose regression. The data synthesis pipeline only requires texture-less 3D object models and desired viewpoints, and it is cheap in terms of both time and hardware storage. Our data synthesis process is up to three orders of magnitude faster than commonly applied approaches that render RGB image data. We show the effectiveness of our system on the LineMOD, LineMOD Occlusion, and YCB Video datasets. The implementation of our system is available at: https://github.com/GeeeG/CloudAAE.

We propose a method for detecting structural changes in a city using images captured from vehicular mounted cameras over traversals at two different times. We first generate 3D point clouds for each traversal from the images and approximate GNSS/INS readings using Structure-from-Motion (SfM). A direct comparison of the two point clouds for change detection is not ideal due to inaccurate geo-location information and possible drifts in the SfM. To circumvent this problem, we propose a deep learning-based non-rigid registration on the point clouds which allows us to compare the point clouds for structural change detection in the scene. Furthermore, we introduce a dual thresholding check and post-processing step to enhance the robustness of our method. We collect two datasets for the evaluation of our approach. Experiments show that our method is able to detect scene changes effectively, even in the presence of viewpoint and illumination differences.

In this paper, we introduce Cirrus, a new long-range bi-pattern LiDAR public dataset for autonomous driving tasks such as 3D object detection, critical to highway driving and timely decision making. Our platform is equipped with a high-resolution video camera and a pair of LiDAR sensors with a 250-meter effective range, which is significantly longer than existing public datasets. We record paired point clouds simultaneously using both Gaussian and uniform scanning patterns. Point density varies significantly across such a long range, and different scanning patterns further diversify object representation in LiDAR. In Cirrus, eight categories of objects are exhaustively annotated in the LiDAR point clouds for the entire effective range. To illustrate the kind of studies supported by this new dataset, we introduce LiDAR model adaptation across different ranges, scanning patterns, and sensor devices. Promising results show the great potential of this new dataset to the robotics and computer vision communities.

Quadrupedal robots are skillful at locomotion tasks while lacking manipulation skills, not to mention dexterous manipulation abilities. Inspired by the animal behavior and the duality between multi-legged locomotion and multi-fingered manipulation, we showcase a circus ball challenge on a quadrupedal robot, ANYmal. We employ a model-free reinforcement learning approach to train a deep policy that enables the robot to balance and manipulate a light-weight ball robustly using its limbs without any contact measurement sensor. The policy is trained in the simulation, in which we randomize many physical properties with additive noise and inject random disturbance force during manipulation, and achieves zero-shot deployment on the real robot without any adjustment. In the hardware experiments, dynamic performance is achieved with a maximum rotation speed of 15 °/s, and robust recovery is showcased under external poking. To our best knowledge, it is the first work that demonstrates the dexterous dynamic manipulation on a real quadrupedal robot.

The feasibility of using chip-based RFID designs as wireless sensor tags open a wide range of application possibilities in the field of robotics. However, multi-step lithography manufacturing and/or MEMS techniques are often required for the industrial-grade fabrication of such sensors. In this paper, we present a simple, home-based, two-step fabrication process to produce chipless RF-based wireless sensors. We use an office-based inkjet printer to produce the antenna traces using silver conductive ink. Kirigami-inspired designs are used to produce four sensors responsive to various mechanical stimuli commonly used for DIY robotic projects (contact, compression, extension, and bend). We demonstrate sensing and wireless transmission of the detected mechanical stimuli through the proposed chipless sensor tags with reliable consistency. The tags can be replicated quickly with the inkjet-printing method. This paper also contains analyses on the effects of varying the dimensions and electrical parameters of the tags. The developed antennas in this work can be used as wireless mechano-responsive sensors for robotic applications.

We present a novel algorithm for combined task assignment and path planning on a roadmap with stochastic costs. In this problem, the initially unassigned robots and tasks are located at known positions in a roadmap. We want to assign a unique task to each robot and compute a path for the robot to go to the task location. Given the means and variances of travel cost, our goal is to develop algorithms that guarantee that for each robot, with high probability, the total travel cost is below a minimum value in any realization of the stochastic travel costs. We prove that the solution can be obtained by solving (a) a chance-constrained shortest path problems for all robot-task pairs and (b) a linear bottleneck assignment problem in which the cost of an assignment is equal to the optimal objective value of the former problem. We propose algorithms for solving the chance-constrained shortest path problem either optimally or approximately by solving a number of deterministic shortest path problems that minimize some linear combination of means and variances of edge costs. We present simulation results on randomly generated networks and data to demonstrate that our algorithm is scalable with the number of robots (or tasks) and the size of the network.