IROS 2021 Papers

Optical flow can be leveraged in robotic systems for obstacle detection where low latency solutions are critical in highly dynamic settings. While event-based cameras have changed the dominant paradigm of sending by encoding stimuli into spike trails, offering low bandwidth and latency, events are still processed with traditional convolutional networks in GPUs defeating, thus, the promise of efficient low capacity low power processing that inspired the design of event sensors. In this work, we introduce a shallow spiking neural network for the computation of optical flow consisting of Leaky Integrate and Fire neurons.Optical flow is predicted as the synthesis of motion orientation selective channels. Learning is accomplished by Back-propapagation Through Time. We present promising results on events recorded in real "in the wild" scenes that has the capability to use only a small fraction of the energy consumed in CNNs deployed on GPUs.

We introduce Self-supervised Online Reward Shaping (SORS), which aims to improve the sample efficiency of any RL algorithm in sparse-reward environments by automatically densifying rewards. The proposed framework alternates between classification-based reward inference and policy update steps—the original sparse reward provides a self-supervisory signal for reward inference by ranking trajectories that the agent observes, while the policy update is performed with the newly inferred, typically dense reward function. We introduce theory that shows that, under certain conditions, this alteration of the reward function will not change the optimal policy of the original MDP, while potentially increasing learning speed significantly. Experimental results on several sparse-reward environments demonstrate that, across multiple domains, the proposed algorithm is not only significantly more sample efficient than a standard RL baseline using sparse rewards, but, at times, also achieves similar sample efficiency compared to when hand-designed dense reward functions are used.

Humans learn to imitate by observing others. However, robot imitation learning generally requires expert demonstrations in the first-person view (FPV). Collecting such FPV videos for every robot could be very expensive.Third-person imitation learning (TPIL) is the concept of learning action policies by observing other agents in a third-person view (TPV), similar to what humans do. This ultimately allows utilizing human and robot demonstration videos in TPV from many different data sources, for the policy learning. In this paper, we present a TPIL approach for robot tasks with egomotion. Although many robot tasks with ground/aerial mobility often involve actions with camera egomotion, study on TPIL for such tasks has been limited. Here, FPV and TPV observations are visually very different; FPV shows egomotion while the agent appearance is only observable in TPV. To enable better state learning for TPIL, we propose our disentangled representation learning method. We use a dual auto-encoder structure plus representation permutation loss and time-contrastive loss to ensure the state and viewpoint representations are well disentangled. Our experiments show the effectiveness of our approach.

Programmable matter is a material that can change its physical properties at will, whether it is its shape, density or conductivity. It can be implemented as an ensemble of micro-robots arranged in space to form a specific shape and having their own computing power. This technology behaves as a distributed system. Each micro-robot is called a module and the whole forms a modular robot. This paper tackles the self-reconfiguration problem by presenting a deterministic planning algorithm that can decide which positions can be filled over multiple iterations using virtual forces. The proposed algorithm implements the Hungarian method to optimize the planning by minimizing the total number of movements of the robots and preventing positions from being blocked. Each module embeds the same algorithm and coordinates with the others using neighbor-to-neighbor communications. Simulation results are conducted to show the effectiveness of the proposed approach.

In this paper, we present an accelerometer-based kinematic calibration algorithm to accurately estimate the pose of multiple sensor units distributed along a robot body. Our approach is self-contained, can be used on any robot provided with a Denavit-Hartenberg kinematic model, and on any skin equipped with Inertial Measurement Units (IMUs). To validate the proposed method, we first conduct extensive experimentation in simulation and demonstrate a sub-cm positional error from ground truth data—an improvement of six times with respect to prior work; subsequently, we then perform a real-world evaluation on a seven degrees-of-freedom collaborative platform. For this purpose, we additionally introduce a novel design for a stand-alone artificial skin equipped with an IMU for use with the proposed algorithm and a proximity sensor for sensing distance to nearby objects. In conclusion, in this work, we demonstrate seamless integration between a novel hardware design, an accurate calibration method, and preliminary work on applications: the high positional accuracy effectively enables to locate distributed proximity data and allows for a distributed avoidance controller to safely avoid obstacles and people without the need of additional sensing.

Autonomous driving with imitation learning is vulnerable to the quality of an expert dataset. Typical driving involves situations or online data that are biased toward specific scenarios such as lane following or stop. This property causes an imbalance in the driving dataset, and it is highly likely to deteriorate the performance of autonomous driving with imitation learning. In this paper, we propose a dataset self-balancing system with biased online data and an imbalanced dataset. By estimating the probability distribution of a dataset, we compute the probability and novelty of online data and then filter only qualified novel data. In addition, using the computed probability distribution, we determine the data that are non-informative in the current dataset and then exchange them with novel online data. At last, by retraining the driving neural network with high-entropy data batches, our method achieves incremental driving intelligence. We demonstrated the effectiveness of our method through open-loop evaluation and ablation studies in a CARLA simulator; the results show that our proposed system effectively balances the dataset with 100 scenarios and decreases test loss over time.

We address the problem of completing per pixel dense depth map using a single RGB image and the sparse point cloud of the scene. Depth prediction from RGB image is a hard problem and while dense point clouds obtained from LiDAR sensors can be used in addition to RGB image, the cost of such sensors is a significant barrier. Having LiDAR sensors which capture sparse point clouds is a reasonable middle ground. We propose a novel architecture which incorporates geometric primitives and self attention mechanisms, to improve the prediction. The motivation of self attention is to capture the correlations between scene and object elements, e.g. between the right and left window of car, early on in the network. While that for using geometric primitives is to have a high level clustering cue to enable the network to exploit similar correlations. In addition, we enforce complimentarity in the predictions made with RGB and sparse LiDAR respectively, this forces the two corresponding branches to focus on hard areas which are not already well predicted by the other branch. With exhaustive experiments on KITTI depth completion benchmark, NYU v2 and Matterport3D we show that the proposed method provides state-of-the-art results.

Learned visuomotor policies have shown considerable success as an alternative to traditional, hand-crafted frameworks for robotic manipulation. Surprisingly, an extension of these methods to the multiview domain is relatively unexplored. A successful multiview policy could be deployed on a mobile manipulation platform, allowing the robot to complete a task regardless of its view of the scene. In this work, we demonstrate that a multiview policy can be found through imitation learning by collecting data from a variety of viewpoints. We illustrate the general applicability of the method by learning to complete several challenging multi-stage and contact-rich tasks, from numerous viewpoints, both in a simulated environment and on a real mobile manipulation platform. Furthermore, we analyze our policies to determine the benefits of learning from multiview data compared to learning with data collected from a fixed perspective. We show that learning from multiview data results in little, if any, penalty to performance for a fixed-view task compared to learning with an equivalent amount of fixed-view data. Finally, we examine the visual features learned by the multiview and fixed-view policies. Our results indicate that multiview policies implicitly learn to identify spatially correlated features.

Agent control among pedestrians is often approached in one of the three following ways: using predefined behaviors for agent navigation, learning navigation behaviors from data, or search-based planning on a graph where each edge is a feasible action chosen from a set of predefined actions. While the first approach often produces natural looking motions and the second learns and utilizes complex interactions with pedestrians, both lack global reasoning about how to sequence these behaviors to achieve the overall goal. The third approach, namely search-based planning, does incorporate global reasoning but relies on predefined actions that do not involve any interactions with pedestrians or assume predefined interactions that cannot model complex interactions. This is a significant drawback since many situations such as going through a doorway blocked by other people require complex interactions in order to avoid highly suboptimal behaviors or not being able to get to the goal at all. To this end, we propose a search-based planning framework that constructs and searches a graph wherein each edge can be either a predefined action or a learned behavior. We further extend it to deal with the uncertainty arising from introducing learned behaviors. We present the algorithm, go over its theoretical analysis, and present experimental results.

In this work we tackle the path planning problem for a 21-dimensional snake robot-like manipulator, navigating a cluttered gas turbine for the purposes of inspection. Heuristic search based approaches are effective planning strategies for common manipulation domains. However, their performance on high dimensional systems is heavily reliant on the effectiveness of the action space and the heuristics chosen. The complex nature of our system, reachability constraints, and highly cluttered turbine environment renders naive choices of action spaces and heuristics ineffective. To this extent we have developed i) a methodology for dynamically generating actions based on online optimization that help the robot navigate narrow spaces, ii) a technique for lazily generating these computationally expensive optimization actions to effectively utilize resources, and iii) heuristics that reason about the homotopy classes induced by the blades of the turbine in the robot workspace and a Multi-Heuristic framework which guides the search along the relevant classes. The impact of our contributions is presented through an experimental study in simulation, where the 21 DOF manipulator navigates towards regions of inspection within a turbine.

Multi-object tracking is a critical component in autonomous navigation, as it provides valuable information for decision-making. Many researchers tackled the 3D multi-object tracking task by filtering out the frame-by-frame 3D detections; however, their focus was mainly on finding useful features or proper matching metrics. Our work focuses on a neglected part of the tracking system: score refinement and tracklet termination. We show that manipulating the scores depending on time consistency while terminating the tracklets depending on the tracklet score improves tracking results. We do this by increasing the matched tracklets’ score with score update functions and decreasing the unmatched tracklets’ score. Compared to count-based methods, our method consistently produces better AMOTA and MOTA scores when utilizing various detectors and filtering algorithms on different datasets. The improvements in AMOTA score went up to 1.83 and 2.96 in MOTA. We also used our method as a late-fusion ensembling method, and it performed better than voting-based ensemble methods by a solid margin. It achieved an AMOTA score of 67.6 on nuScenes test evaluation, which is comparable to other state-of-the-art trackers. Code is publicly available at: https://github.com/cogsys-tuebingen/CBMOT.

In recent studies, the widespread of deep learning has made many kinds of large-scale image datasets available and it has enabled to improve the performance of image-based 3-D scene reconstruction. Several studies estimate whole 3-D scenes including occluded or unseen parts consistent with the obtained partial observations by integrating prior knowledge from training datasets with them, under no camera parameters nor image landmark correspondence are known. Although they generate “discrete” scene instances, they cannot represent and treat their “ambiguity” at all.This paper proposes a novel deep-learning-based framework that can directly represent and treat the ambiguity of scene reconstructions. We introduce a neural network which encodes a target scene as a descriptor. The network takes partial observations as input and outputs a parametric set of the scene descriptors containing all scenes consistent with given observations. The input observations may be “incomplete” in the sense that they do not have enough pieces of information to uniquely determine the whole scene due to neither geometry in-formation nor landmark correspondences available (ill-defined cases). The network is trained based on the dataset of the complete 3-D scenes and possible partial observations so that it can predict the unseen parts from incomplete observations. The paper introduces the method to induce such a descriptor space into the encoder/decoder architecture by employing novel definitions of loss functions measuring “validity”, “consistency” and “reproducibility”. When the series of partial and incom-plete observations for the same 3-D scene is obtained, the reconstruction ambiguity is explicitly treated by parametrically integrating the descriptor set for each observation into one descriptor set.

We develop a Multi-Agent Reinforcement Learning (MARL) method to learn scalable control policies for target tracking. Our method can handle an arbitrary number of pursuers and targets; we show results for tasks consisting up to 1000 pursuers tracking 1000 targets. We use a decentralized, partially-observable Markov Decision Process framework to model pursuers as agents receiving partial observations (range and bearing) about targets which move using fixed, unknown policies. An attention mechanism is used to parameterize the value function of the agents; this mechanism allows us to handle an arbitrary number of targets. Entropy-regularized off-policy RL methods are used to train a stochastic policy, and we discuss how it enables a hedging behavior between pursuers that leads to a weak form of cooperation in spite of completely decentralized control execution. We further develop a masking heuristic that allows training on smaller problems with few pursuers-targets and execution on much larger problems. Thorough simulation experiments and comparisons to state of the art algorithms are performed to study the scalability of the approach and robustness of performance to varying numbers of agents and targets.

In multi-robot multi-target tracking, robots coordinate to monitor groups of targets moving about an environment. We approach planning for such scenarios by formulating a receding-horizon, multi-robot sensing problem with a mutual information objective. Such problems are NP-Hard in general. Yet, our objective is submodular which enables certain greedy planners to guarantee constant-factor suboptimality. However, these greedy planners require robots to plan their actions in sequence, one robot at a time, so planning time is at least proportional to the number of robots. Solving these problems becomes intractable for large teams, even for distributed implementations. Our prior work proposed a distributed planner (RSP) which reduces this number of sequential steps to a constant, even for large numbers of robots, by allowing robots to plan in parallel while ignoring some of each others’ decisions. Although that analysis is not applicable to target tracking, we prove a similar guarantee, that RSP planning approaches performance guarantees for fully sequential planners, by employing a novel bound which takes advantage of the independence of target motions to quantify effective redundancy between robots’ observations and actions. Further, we present analysis that explicitly accounts for features of practical implementations including approximations to the objective and anytime planning. Simulation results—available via open source release—for target tracking with ranging sensors demonstrate that our planners consistently approach the performance of sequential planning (in terms of position uncertainty) given only 2–8 planning steps and for as many as 96 robots with a 24x reduction in the number of sequential steps in planning. Thus, this work makes planning for multi-robot target tracking tractable at much larger scales than before, for practical planners and general tracking problems.

In this paper, we propose a new scheme based on the Sampson distance (SD) to describe visual feature residuals for visual-inertial odometry (VIO). Unlike the epipolar-constraint-based SD for visual odometry (VO), the proposed SD is formulated based on the perspective projection constraint. We proved in theory that the proposed SD retains the good properties of those earlier SD criteria in the literature of VO and it represents a visual feature residual more accurately than the prevailing transfer distance (TD) in existing VIO methods. We formulate three distance criteria, including TD, reprojection error (RE), and SD, and compared their performances by simulation. The results show that the SD is much more accurate than the TD and it is a very accurate estimate of the gold standard criteria—RE. Based on the SD, we modified VINS-Mono by replacing its TD-based visual residuals with the SD-based residuals and study the SD's efficacy in pose estimation by experiments with several public datasets. The results reveal that the SD-based VINS-Mono has a substantial improvement over the original VINS-Mono in pose estimation accuracy. This indicates that the SD is a better distance criterion than the TD for representing visual feature residuals. The proposed SD may find its applications to broader areas in computer vision and robotics.

Planning for robotic systems is frequently formulated as an optimization problem. Instead of manually tweaking the parameters of the cost function, they can be learned from human demonstrations by Inverse Reinforcement Learning (IRL). Common IRL approaches employ a maximum entropy trajectory distribution that can be learned with soft reinforcement learning, where the reward maximization is regularized with an entropy objective. The consideration of safety constraints is of paramount importance for human-robot collaboration. For this reason, our work addresses maximum entropy IRL in constrained environments. Our contribution to this research area is threefold: (1) We propose Constrained Soft Reinforcement Learning (CSRL), an extension of soft reinforcement learning to Constrained Markov Decision Processes (CMDPs). (2) We transfer maximum entropy IRL to CMDPs based on CSRL. (3) We show that using importance sampling in maximum entropy IRL in constrained environments introduces a bias and fails to achieve feature matching. In our evaluation we consider the tactical lane change decision of an autonomous vehicle in a highway scenario modeled in the SUMO traffic simulation.

Visual servoing control schemes, such as Image-Based (IBVS), Pose Based (PBVS) or Hybrid-Based (HBVS) have been extensively developed over the last decades making possible their uses in a large number of applications. It is well-known that the main problems to be handled concern the presence of local minima or singularities, the visibility constraint, the joint limits, etc. Recently, Model Predictive Path Integral (MPPI) control algorithm has been developed for autonomous robot navigation tasks. In this paper, we propose a MPPI-VS framework applied for the control of a 6-DoF robot with 2D point, 3D point, and Pose Based Visual Servoing techniques. We performed intensive simulations under various operating conditions to show the potential advantages of the proposed control framework compared to the classical schemes. The effectiveness, the robustness and the capability in coping easily with the system constraints of the control framework are shown.

Developing an agent in reinforcement learning (RL) that is capable of performing complex control tasks directly from high-dimensional observation such as raw pixels is a challenge as efforts still need to be made towards improving sample efficiency and generalization of RL algorithm. This paper considers a learning framework for a Curiosity Contrastive Forward Dynamics Model (CCFDM) to achieve a more sample-efficient RL based directly on raw pixels. CCFDM incorporates a forward dynamics model (FDM) and performs contrastive learning to train its deep convolutional neural network-based image encoder (IE) to extract conducive spatial and temporal information to achieve a more sample efficiency for RL. In addition, during training, CCFDM provides intrinsic rewards, produced based on FDM prediction error, and encourages the curiosity of the RL agent to improve exploration. The diverge and less-repetitive observations provided by both our exploration strategy and data augmentation available in contrastive learning improve not only the sample efficiency but also the generalization . Performance of existing model-free RL methods such as Soft Actor-Critic built on top of CCFDM outperforms prior state-of-the-art pixel-based RL methods on the DeepMind Control Suite benchmark.

This paper proposes an admittance controller-based teleoperation system for contact-rich tasks. Based on the analysis of the motivating task (deposited iron lump removal task in the steel mill), the system concept is focused on the practical aspects of the system, and various components are combined to enhance the safety of the teleoperation of the robot. To connect the large inertia difference between the teleoperated robot and the command device, the admittance control is utilized in the teleoperation system, and the virtual spring saturation is adapted with the damping injection to ensure safe motion during the task. Lastly, the inertia and damping adaptation rule based on the contact force frequency is developed so that the system can selectively dissipate energy when the system shows oscillatory behavior. The proposed techniques have shown their effectiveness through the experiments. Although this study has started from a specific target, it suggests a practical solution for various contact-rich teleoperated tasks in the hazardous industrial workspace.