IROS 2021 论文列表

It is challenging for a mobile robot to navigate through human crowds. Existing approaches usually assume that pedestrians follow a predefined collision avoidance strategy, like social force model (SFM) or optimal reciprocal collision avoidance (ORCA). However, their performances commonly need to be further improved for practical applications, where pedestrians follow multiple different collision avoidance strategies. In this paper, we propose a map-based deep reinforcement learning approach for crowd-aware robot navigation with various pedestrians. We use the sensor map to represent the environmental information around the robot, including its shape and observable appearances of obstacles. We also introduce the pedestrian map that specifies the movements of pedestrians around the robot. By applying both maps as inputs of the neural network, we show that a navigation policy can be trained to better interact with pedestrians following different collision avoidance strategies. We evaluate our approach under multiple scenarios both in the simulator and on an actual robot. The results show that our approach allows the robot to successfully interact with various pedestrians and outperforms compared methods in terms of the success rate.

Mobile devices are increasingly expected to sup-port high-performance cyber-physical applications in small form factors, e.g., drones and rovers. However, the gap between hardware limitations of these devices and application requirements is still prohibitive – conflicting goals such as robust, accurate, and efficient execution must be managed carefully to achieve acceptable operation. In this paper, we explore the tradeoff between performance and efficiency in such cyber-physical systems, specifically with respect to localization (a core task for any mobile autonomous device). We perform a design space exploration (DSE) given a number of configurable parameters for both localization algorithm and platform layers. Given the configuration space, we formulate a cross-layer multi-objective optimization problem to explore the tradeoff between localization accuracy and power consumption. We then propose a predictive model for robust execution that can be used to determine desirable configurations at runtime in the face of environmental changes.

Advances in sensing and learning algorithms have led to increasingly mature solutions for human detection by robots, particularly in selected use-cases such as pedestrian detection for self-driving cars or close-range person detection in consumer settings. Despite this progress, the simple question which sensor-algorithm combination is best suited for a person detection task at handƒ remains hard to answer. In this paper, we tackle this issue by conducting a systematic cross-modal analysis of sensor-algorithm combinations typically used in robotics. We compare the performance of state-of-the-art person detectors for 2D range data, 3D lidar, and RGB-D data as well as selected combinations thereof in a challenging industrial use-case.We further address the related problems of data scarcity in the industrial target domain, and that recent research on human detection in 3D point clouds has mostly focused on autonomous driving scenarios. To leverage these methodological advances for robotics applications, we utilize a simple, yet effective multi-sensor transfer learning strategy by extending a strong image-based RGB-D detector to provide cross-modal supervision for lidar detectors in the form of weak 3D bounding box labels.Our results show a large variance among the different approaches in terms of detection performance, generalization, frame rates and computational requirements. As our use-case contains difficulties representative for a wide range of service robot applications, we believe that these results point to relevant open challenges for further research and provide valuable support to practitioners for the design of their robot system.

Detecting and tracking objects in 3D scenes play crucial roles in autonomous driving. Successfully recognizing objects through space and time hinges on a strong detector and a reliable association scheme. Recent 3D detection and tracking approaches widely represent objects as points when associating detection results with trajectories. Despite the demonstrated success, these approaches do not fully exploit the rich appearance information of objects. In this paper, we present a conceptually simple yet effective algorithm, named AlphaTrack, which considers both the location and appearance changes to perform joint 3D object detection and tracking. To achieve this, we propose a cross-modal fusion scheme that fuses camera appearance feature with LiDAR feature to facilitate 3D detection and tracking. We further attach an additional branch to the 3D detector to output instance-aware appearance embedding, which significantly improves tracking performance with our designed association mechanisms. Extensive validations on large-scale autonomous driving dataset demonstrate the effectiveness of the proposed algorithm in comparison with state-of-the-art approaches. Notably, the proposed algorithm ranks first on the nuScenes tracking leaderboard to date.

Real-time dense reconstruction has been extensively studied for its wide applications in computer vision and robotics, meanwhile much effort has been made for the multi-robot system which plays an irreplaceable role in complicated but time-critical scenarios, e.g., search and rescue tasks. In this paper, we propose an efficient system named Coxgraph for multi-robot collaborative dense reconstruction in real-time. In our system, each client performs volumetric mapping in a producer-consumer manner. To facilitate transmission, we propose a compact 3D representation which transforms the SDF submap to mesh packs. During the recovery of submaps from mesh packs, the system can perform loop closure outlier rejection based on geometry consistency, trajectory collision and fitness check. Then we develop a robust map fusion method through joint optimization of trajectories and submaps. Extensive experiments demonstrate that our system can produce a globally consistent dense map in real-time with less transmission load, which is available as open-source software 1.

The correct characterization of uncertainty when predicting human motion is equally important as the accuracy of this prediction. We present a new method to correctly predict the uncertainty associated with the predicted distribution of future trajectories. Our approach, CovariaceNet, is based on a Conditional Generative Model with Gaussian latent variables in order to predict the parameters of a bi-variate Gaussian distribution. The combination of CovarianceNet with a motion prediction model results in a hybrid approach that outputs a uni-modal distribution. We will show how some state of the art methods in motion prediction become overconfident when predicting uncertainty, according to our proposed metric and validated in the ETH data-set [1]. CovarianceNet correctly predicts uncertainty, which makes our method suitable for applications that use predicted distributions, e.g., planning or decision making.

A bio-inspired way to model locomotion is using a network of coupled phase oscillators to create a Central Pattern Generator (CPG). The recently developed feedback control method tegotae includes exteroceptive force feedback into the governing phase update equations, leading to gait limit cycles. However, the oscillator coupling weights are often determined empirically. Here, we first investigate how the coupling coefficients influence the limit cycle convergence behavior on a 2- and 3-limbed structure in simulation. We find that the convergence with tegotae can be improved by introducing appropriate cross-couplings. This results in a smoother convergence and steady-state behavior where each individual oscillator drives the full network to a common convergence state in comparison to competing convergence states with ill-chosen cross-couplings. We then validate the findings in hardware and hypothesize how the appropriate couplings could be derived directly from the morphology, potentially eliminating the empiric determination.

Volumetric deep learning approach towards stereo matching aggregates a cost volume computed from input left and right images using 3D convolutions. Recent works showed that utilization of extracted image features and a spatially varying cost volume aggregation complements 3D convolutions. However, existing methods with spatially varying operations are complex, cost considerable computation time, and cause memory consumption to increase. In this work, we construct Guided Cost volume Excitation (GCE) and show that simple channel excitation of cost volume guided by image can improve performance considerably. Moreover, we propose a novel method of using top-k selection prior to soft-argmin disparity regression for computing the final disparity estimate. Combining our novel contributions, we present an end-to-end network that we call Correlate-and-Excite (CoEx). Extensive experiments of our model on the SceneFlow, KITTI 2012, and KITTI 2015 datasets demonstrate the effectiveness and efficiency of our model and show that our model outperforms other speed-based algorithms while also being competitive to other state-of-the-art algorithms. Codes will be made available at https://github.com/antabangun/coex.

Accurate localization is one of the biggest challenges in underwater robotics. The primary reasons behind that are unavailability of satellite-based positioning below the surface, and lack of clear features in natural water bodies for visually aided localization. As such, the common method of choice for external position referencing in underwater robots is the use of acoustic signals for computing range or direction of arrival. To that end, we have developed an acoustic range based navigation system with floating, movable beacons. In this paper, we present an approach for planning the trajectory of acoustic beacons in a way that they provide the best possible navigation support for a group of underwater vehicles. We use an information theoretic approach to beacon path planning that minimizes the group’s position uncertainty. We evaluate our approach with realistic simulations calibrated using real-world data, and present results.

Similar to human work, robotic tasks sometimes require two hands to be accomplished. This requires coordinated motion planning and control. While fulfilling the task in a coordinated manner is already a big challenge, the task at hand becomes even harder when obstacles are introduced in the environment that need to be avoided. Furthermore in the case of dynamic environments, contacts cannot be avoided all the time, even with robust planning. In addition to geometric constraints, bimanual systems need to be able to detect and react to contacts during task execution. To this aim, we integrate a vector-field based planning scheme, that is able to avoid obstacles, with contact detection and reactive control methods based on contact wrench estimation such as admittance control. We also fuse the real contact forces into the planner directly together with the circular repulsive fields. The resulting planner-controller combination is capable of obstacle avoidance planning as well as reaction control in the case of unforeseen contacts that can also be used in situations where the manipulation needs to be guided by the environment such as landing control in only roughly known environments. We evaluate our approach on the torque-controlled Kobo bimanual set-up and also perform rigorous simulation studies.

We propose a method to determine control input on the basis of minimizing the risk-sensitive cost function and show the results of an experiment in which the method was applied to a cooperative transportation robot system that we have developed. In the robot system, two robots hold a work object to transport without an external fixing device. The mechanism yields the force interaction between the robots and the object, which results in unexpected random errors in transportation. We add a stochastic term to the system model to describe such errors and solve the stochastic differential equation numerically to estimate the cost function. Utilizing the risk-sensitive cost function enables us to find the control input that balances efficiency and safety. The experimental results revealed that the chance of a robot colliding with an obstacle during transportation decreased from 90% to 7% compared with the optimal control.

This paper presents a novel methodology that allows a swarm of robots to perform a cooperative transportation task. Our approach consists of modeling the swarm as a Gibbs Random Field (GRF), taking advantage of this framework’s locality properties. By setting appropriate potential functions, robots can dynamically navigate, form groups, and perform co- operative transportation in a completely decentralized fashion. Moreover, these behaviors emerge from the local interactions without the need for explicit communication or coordination. To evaluate our methodology, we perform a series of simulations and proof-of-concept experiments in different scenarios. Our results show that the method is scalable, adaptable, and robust to failures and changes in the environment.

Widespread adoption of autonomous vehicles will not become a reality until solutions are developed that enable these intelligent agents to co-exist with humans. This includes safely and efficiently interacting with human-driven vehicles, especially in both conflictive and competitive scenarios. We build up on the prior work on socially-aware navigation and borrow the concept of social value orientation from psychology —that formalizes how much importance a person allocates to the welfare of others— in order to induce altruistic behavior in autonomous driving. In contrast with existing works that explicitly model the behavior of human drivers and rely on their expected response to create opportunities for cooperation, our Sympathetic Cooperative Driving (SymCoDrive) paradigm trains altruistic agents that realize safe and smooth traffic flow in competitive driving scenarios only from experiential learning and without any explicit coordination. We demonstrate a significant improvement in both safety and traffic-level metrics as a result of this altruistic behavior and importantly conclude that the level of altruism in agents requires proper tuning as agents that are too altruistic also lead to sub-optimal traffic flow. The code and supplementary material are available at: https://symcodrive.toghi.net/

Cognitive cooperative assistance in robot-assisted surgery holds the potential to increase quality of care in minimally invasive interventions. Automation of surgical tasks promises to reduce the mental exertion and fatigue of surgeons. In this work, multi-agent reinforcement learning is demonstrated to be robust to the distribution shift introduced by pairing a learned policy with a human team member. Multi-agent policies are trained directly from images in simulation to control multiple instruments in a sub task of the minimally invasive removal of the gallbladder. These agents are evaluated individually and in cooperation with humans to demonstrate their suitability as autonomous assistants. Compared to human teams, the hybrid teams with artificial agents perform better considering completion time (44.4% to 71.2% shorter) as well as number of collisions (44.7% to 98.0% fewer). Path lengths, however, increase under control of an artificial agent (11.4% to 33.5% longer). A multi-agent formulation of the learning problem was favored over a single-agent formulation on this surgical sub task, due to the sequential learning of the two instruments. This approach may be extended to other tasks that are difficult to formulate within the standard reinforcement learning framework. Multi-agent reinforcement learning may shift the paradigm of cognitive robotic surgery towards seamless cooperation between surgeons and assistive technologies.

The lack of GPS signal in the underwater environment poses limitations in terms of localization and navigation of mobile robots. Strategies based on acoustic localization systems are employed to improve underwater navigation. In this paper we describe a first step towards the development of a marine system of systems involving autonomous mobile nodes. The approach relies on communication networking between an Autonomous Surface Vehicle (ASV), equipped with an Ultra Short BaseLine (USBL) device, and an Autonomous Underwater Vehicle (AUV). An active acoustic communication protocol prioritizes the AUV positioning rate, whereas a model-based navigation filter handles the delayed measurements caused by the acoustic communication latency. The system has been tested in a real marine environment to analyze its behavior and the quality of the navigation estimation. The experimental results show that the navigation algorithm on-board the AUV provides an estimate of its position with an error of a few meters with respect to the GPS ground-truth, over a total path of approximately 210m, exploiting acoustic positioning data provided by the ASV in order to limit drift problems.

Natural dynamics, nonlinear optimization, and, more recently, convex optimization are available methods for stiffness design of energy-efficient series elastic actuators. Natural dynamics and general nonlinear optimization only work for a limited set of load kinetics and kinematics, cannot guarantee convergence to a global optimum, or depend on initial conditions to the numerical solver. Convex programs alleviate these limitations and allow a global solution in polynomial time, which is useful when the space of optimization variables grows (e.g., when designing optimal nonlinear springs or co-designing spring, controller, and reference trajectories). Our previous work introduced the stiffness design of series elastic actuators via convex optimization when the transmission dynamics are negligible, which is an assumption that applies mostly in theory or when the actuator uses a direct or quasi-direct drive. In this work, we extend our analysis to include friction at the transmission. Coulomb friction at the transmission results in a non-convex expression for the energy dissipated as heat, but we illustrate a convex approximation for stiffness design. We experimentally validated our framework using a series elastic actuator with specifications similar to the knee joint of the Open Source Leg, an open-source robotic knee-ankle prosthesis.

We present a formulation for linear temporal logic (LTL)-based task planning of nonlinear dynamical systems. We consider pick-and-place task planning as a typical example of the planning task that can be modeled as a hybrid system that includes the states of robots and objects. LTL-based planning for hybrid systems is solved as a mixed-integer problem (MIP), especially a mixed-integer linear programming problem (MILP). Due to the formulation by the MILP, we could only deal with linear systems and linear constraints. In our proposed method, we apply a convex approximation to systems that have bilinear terms and quadratic terms in their dynamics. And we incorporate nonlinear systems into existing LTL-based planning as an MILP. We demonstrate the effectiveness through numerical simulations of a simple robot arm system and drone system.

We present a control-aware design optimization method for quadrupedal robots. In particular, we show that it is possible to analytically differentiate typical, inverse dynamics-based whole body controllers with respect to design parameters, and that gradient-based methods can be used to efficiently improve an initial morphological design according to well-established metrics. We apply our design optimization method to various types of quadrupedal robots, including designs that feature closed kinematic chains. The methodology we present enables a principled comparison of different types of optimized legged robot designs. Our experiments, for example, suggest that mechanically-coupled three-link leg designs present notable advantages in terms of performance and efficiency over the common two-link leg designs used in most quadrupedal robots today.

Hybrid robots incorporate the advantages of both aerial-only and terrestrial-only vehicles to achieve enhanced mobility and better energy efficiency. Among hybrid vehicles, spherical robots offer the best maneuverability. While operating on uneven surfaces is one of the main benefits of spherical robots, the current literature only covers control of these robots on flat surfaces. This work presents two control algorithms to track a desired trajectory and angular velocity of spherical robots on uneven terrains. The proposed control algorithms can be used when the terrain is known analytically or empirically (i.e., point cloud). By allowing the controller to use empirical information about the terrain profile, this work broadens the implementation of spherical robots in real applications.