IROS 2021 论文列表

In robotic bin-picking applications, the perception of texture-less, highly reflective parts is a valuable but challenging task. The high glossiness can introduce fake edges in RGB images and inaccurate depth measurements, especially in heavily cluttered bin scenarios. In this paper, we present the ROBI (Reflective Objects in BIns) dataset, a public dataset for 6D object pose estimation and multi-view depth fusion in robotic bin-picking scenarios. The ROBI dataset includes a total of 63 bin-picking scenes captured with two active stereo cameras: a high-cost Ensenso sensor and a low-cost RealSense sensor. For each scene, the monochrome/RGB images and depth maps are captured from sampled view spheres around the scene, and are annotated with accurate 6D poses of visible objects and an associated visibility score. For evaluating the performance of depth fusion, we captured the ground truth depth maps by high-cost Ensenso camera with objects coated in anti-reflective scanning spray. To show the utility of the dataset, we evaluated the representative algorithms of 6D object pose estimation and multi-view depth fusion on the full dataset. Evaluation results demonstrate the difficulty of highly reflective objects, especially in difficult cases due to the degradation of depth data quality, severe occlusions, and cluttered scenes. The ROBI dataset is available online at https://www.trailab.utias.utoronto.ca/robi.

Simultaneous Localization and Mapping (SLAM) is considered to be an essential capability for intelligent vehicles and mobile robots. However, most of the current lidar SLAM approaches are based on the assumption of a static environment. Hence the localization in a dynamic environment with multiple moving objects is actually unreliable. The paper proposes a dynamic SLAM framework RF-LIO, building on LIO-SAM, which adds adaptive multi-resolution range images and uses tightly-coupled lidar inertial odometry to first remove moving objects, and then match lidar scan to the submap. Thus, it can obtain accurate poses even in high dynamic environments. The proposed RF-LIO is evaluated on both self-collected datasets and open Urbanloco datasets. The experimental results in high dynamic environments demonstrate that, compared with LOAM and LIO-SAM, the absolute trajectory accuracy of the proposed RF-LIO can be improved by 90% and 70%, respectively. RF-LIO is one of the state-of-the-art SLAM systems in high dynamic environments.

Exploring an unknown environment without colliding with obstacles is one of the essentials of autonomous vehicles to perform diverse missions such as structural inspections, rescues, deliveries, and so forth. Therefore, unmanned aerial vehicles (UAVS), which are fast, agile, and have high degrees of freedom, have been widely used. However, previous approaches have two limitations: a) First, they may not be appropriate for exploring large-scale environments because they mainly depend on random sampling-based path planning that causes unnecessary movements. b) Second, they assume the pose estimation is accurate enough, which is the most critical factor in obtaining an accurate map. In this paper, to explore and map unknown large-scale environments rapidly and accurately, we propose a novel exploration method that combines the pre-calculated Peacock Trajectory with graph-based global exploration and active loop-closing. Because the two-step trajectory that considers the kinodynamics of UAVs is used, obstacle avoidance is guaranteed in the receding-horizon manner. In addition, local exploration that considers the frontier and global exploration based on the graph maximizes the speed of exploration by minimizing unnecessary revisiting. In addition, by actively closing the loop based on the likelihood, pose estimation performance is improved. The proposed method’s performance is verified by exploring 3D simulation environments in comparison with the state-of-the-art methods. Finally, the proposed approach is validated in a real-world experiment.

Spiking Neural Networks (SNNs) aim at providing energy-efficient learning capabilities when implemented on neuromorphic chips with event-based Dynamic Vision Sensors (DVS). This paper studies the robustness of SNNs against adversarial attacks on such DVS-based systems, and proposes R-SNN, a novel methodology for robustifying SNNs through efficient DVS-noise filtering. We are the first to generate adversarial attacks on DVS signals (i.e., frames of events in the spatio-temporal domain) and to apply noise filters for DVS sensors in the quest for defending against adversarial attacks. Our results show that the noise filters effectively prevent the SNNs from being fooled. The SNNs in our experiments provide more than 90% accuracy on the DVS-Gesture and NMNIST datasets under different adversarial threat models.

This paper reports methods to compute the equilibrium stances of a new snake-like robot designed to stabilize its head on a free water surface. To adjust rapidly the stability of the robot, this bio-inspired robot can rotate independently each body-shell, and modify the level of immersion of each module. To predict the stable stance accessible by this additional degree of freedom, a model is developed to compute the equilibrium configurations of the robot from a given parametrization of the body shape. Then, an algorithm is introduced to compute a sequence of controlled body deformations, such that the head configuration relatively to the water surface remains unchanged. Finally, we explore in simulation stances and quasi-static gaits, and investigate to what extent the buoyancy and the body deformations can be used to stabilize the head of the snake-like robot.

Simple template models have proven useful for understanding the underlying dynamics of legged locomotion. The most common one, the SLIP model, considers the legs as linear springs with constant stiffness, and it explains well the radial dynamics of the legs. However, in order to study the influence of the leg swing dynamics and leg segmentation on gait stability, more complex models are required. This paper introduces a novel template model for quadrupedal gait, which considers these additional aspects. The dynamic behavior of the model is analyzed via numerical simulation, using a continuation approach. By conducting a parametric analysis on the trotting gait and analyzing its stability, we identify the influence of the main model parameters, leading to marginally unstable limit cycles. These numerical results are applicable to the design of more efficient elastic quadrupedal robots.

This paper presents a control strategy for quadruped robots to hop on their rear legs in three-dimensional space. The proposed approach generates nominal center of mass (CoM) trajectories based on a template spring-loaded inverted pendulum (SLIP) model. Tracking this reference remains a challenge due to the underactauted nature of balance with point feet. To address this challenge, a control-Lyapunov function based quadratic programming (CLF-QP) controller is proposed, which modulates nominal ground reaction forces (GRFs) to balance the torso while considering friction limits. The CLF construction is guided by a variational-based linearization (VBL) applied to a reduced-order single-rigid-body (SRB) model, and treats underactuation via solving a Riccati equation to obtain the CLF. A new balance control approach is presented that effectively decouples sagittal plane control (via re-planning) with lateral and rotational control (via the CLF and VBL). The proposed approach shows more robust balancing performance than the conventional CLF-QP approach. Simulations of the Mini Cheetah demonstrate in-place hopping with up to a 0.71m apex height.

Teleoperation systems with haptic feedback allow a human user to remotely interact with a dangerous or inac-cessible environment, perform various tasks, and perceive the haptic feedback. To ensure system stability while maintaining the best possible quality of experience (QoE), different teleoperation control schemes and haptic communication strategies need to be selected to adapt to varying network conditions and teleoperation tasks. In this paper, we propose a QoE-driven control scheme switching approach, which adaptively selects the control scheme that provides the best possible QoE for varying communication delay. A transition period is designed to moderate the artifacts during the switching phase. Haptic data reduction approaches are developed for the switching strategy to match the characteristics of each control scheme. Our experiments verify the feasibility of the proposed scheme. Subjective tests confirm that the proposed adaptive switching scheme is able to achieve a superior user QoE in contrast to a fixed control scheme in the presence of varying communication delay up to 200 ms.

In autonomous driving applications, reinforcement learning agents often have to perform complex behavior, which can translate into optimizing multiple objectives while following certain rules. Encoding traffic rules and desires such as safety and comfort via classical methods based on reward shaping (i.e. a weighted combination of different objectives in the reward signal) or Lagrangian methods (including auxiliary losses in the optimization) can be very hard and cumbersome. In this work, we propose to instead shape the action-space at the maximization step of Q-learning. We further introduce a formulation for fixed-horizon estimation of auxiliary costs under the current target-policy based on truncated value- functions to encode the desire of comfortable driving ensuring interpretable behavior. We compare our algorithm to reward shaping and Lagrangian methods in the application of high- level decision making in autonomous driving, considering rules for safety, keeping right and comfort. We train and evaluate our agent in the open-source simulator SUMO on a variety of scenarios with different driver types and traffic situations. Additionally, we apply our method on the real HighD data set, showing the real-world applicability and simplicity of Q- learning with Action-space Shaping.

We propose a novel constrained reinforcement learning method for finding optimal policies in Markov Decision Processes while satisfying temporal logic constraints with a desired probability throughout the learning process. An automata-theoretic approach is proposed to ensure the probabilistic satisfaction of the constraint in each episode, which is different from penalizing violations to achieve constraint satisfaction after a sufficiently large number of episodes. The proposed approach is based on computing a lower bound on the probability of constraint satisfaction and adjusting the exploration behavior as needed. We present theoretical results on the probabilistic constraint satisfaction achieved by the proposed approach. We also numerically demonstrate the proposed idea in a drone scenario, where the constraint is to perform periodically arriving pick-up and delivery tasks and the objective is to fly over high-reward zones to simultaneously perform aerial monitoring.

Humans can robustly follow a visual trajectory defined by a sequence of images (i.e. a video) regardless of substantial changes in the environment or the presence of obstacles. We aim at endowing similar visual navigation capabilities to mobile robots solely equipped with a RGB fisheye camera. We propose a novel probabilistic visual navigation system that learns to follow a sequence of images with bidirectional visual predictions conditioned on possible navigation velocities. By predicting bidirectionally (from start towards goal and vice versa) our method extends its predictive horizon enabling the robot to go around unseen large obstacles that are not visible in the video trajectory. Learning how to react to obstacles and potential risks in the visual field is achieved by imitating human teleoperators. Since the human teleoperation commands are diverse, we propose a probabilistic representation of trajectories that we can sample to find the safest path. We evaluate our navigation system quantitatively and qualitatively in multiple simulated and real environments and compare to state-of-the-art baselines. Our approach outperforms the most recent visual navigation methods with a large margin with regard to goal arrival rate, subgoal coverage rate, and success weighted by path length (SPL). Our method also generalizes to new robot embodiments never used during training.

This work addresses the problem of predicting the motion trajectories of dynamic objects in the environment. Recent advances in predicting motion patterns often rely on machine learning techniques to extrapolate motion patterns from observed trajectories, with no mechanism to directly incorporate known rules. We propose a novel framework, which combines probabilistic learning and constrained trajectory optimisation. The learning component of our framework provides a distribution over future motion trajectories conditioned on observed past coordinates. This distribution is then used as a prior to a constrained optimisation problem which enforces chance constraints on the trajectory distribution. This results in constraint-compliant trajectory distributions which closely resemble the prior. In particular, we focus our investigation on collision constraints, such that extrapolated future trajectory distributions conform to the environment structure. We empirically demonstrate on real-world and simulated datasets the ability of our framework to learn complex probabilistic motion trajectories for motion data, while directly enforcing constraints to improve generalisability, producing more robust and higher quality trajectory distributions.

This paper proposes a framework for learning task specifications from demonstrations, while ensuring that the learned specifications do not violate safety constraints. Furthermore, we show how these specifications can be used in a planning problem to control the robot under environments that can be different from those encountered during the learning phase. We formulate the specification learning problem as a grammatical inference problem, using probabilistic automata to represent specifications. The edge probabilities of the resulting automata represent the demonstrator's preferences. The main novelty in our approach is to incorporate the safety property during the learning process. We prove that the resulting automaton always respects a pre-specified safety property, and furthermore, the proposed method can easily be included in any Evidence-Driven State Merging (EDSM)-based automaton learning scheme. Finally, we introduce a planning algorithm that produces the most desirable plan by maximizing the probability of an accepting trace of the automaton. Case studies show that our algorithm learns the true probability distribution most accurately while maintaining safety. Since, specification is detached from the robot's environment model, a satisfying plan can be synthesized for a variety of different robots and environments including both mobile robots and manipulators.

Optimal control is often used in robotics for planning a trajectory to achieve some desired behavior, as expressed by the cost function. Most works in optimal control focus on finding a single optimal trajectory, which is then typically tracked by another controller. In this work, we instead consider trajectory distribution as the solution of an optimal control problem, resulting in better tracking performance and a more stable controller. A Gaussian distribution is first obtained from an iterative Linear Quadratic Regulator (iLQR) solver. A short horizon Model Predictive Control (MPC) is then used to track this distribution. We show that tracking the distribution is more cost-efficient and robust as compared to tracking the mean or using iLQR feedback control. The proposed method is validated with kinematic control of 7-DoF Panda manipulator and dynamic control of 6-DoF quadcopter in simulation.

For autonomous service robots to successfully perform long horizon tasks in the real world, they must act intelligently in partially observable environments. Most Task and Motion Planning approaches assume full observability of their state space, making them ineffective in stochastic and partially observable domains that reflect the uncertainties in the real world. We propose an online planning and execution approach for performing long horizon tasks in partially observable domains. Given the robot’s belief and a plan skeleton composed of symbolic actions, our approach grounds each symbolic action by inferring continuous action parameters needed to execute the plan successfully. To achieve this, we formulate the problem of joint inference of action parameters as a Hybrid Constraint Satisfaction Problem (H-CSP) and solve the H-CSP using Belief Propagation. The robot executes the resulting parameterized actions, updates its belief of the world and replans when necessary. Our approach is able to efficiently solve partially observable tasks in a realistic kitchen simulation environment. Our approach outperformed an adaptation of the state-of-the-art method across our experiments.

Indoor exploration using mobile robots has typically focused on exploring the entire environment without considering deadlines. This paper introduces a priority-based exploration algorithm for situations with an initially unknown and dynamically assigned deadline. The goal of our exploration strategy is to determine the geometric structure of an unknown environment as rapidly as possible and return to the home location. This is necessary for dangerous environments where an initial rapid robot exploration provides critical information about the layout for subsequent operations. For example, firefighters, for whom time is of the essence, can utilize the map generated by this robotic exploration to navigate a building on fire. We present a three-part strategy to solve this problem. First, we represent the physical environment as an exploration graph, whose vertices represent the local environment with its geometric and semantic information. Second, we assign priority values to these vertices based on their environment regions. Third, we present a graph exploration algorithm that employs the vertex priorities. Simulation experiments on a set of graph environments and Gazebo environments demonstrate that, in contrast to prior approaches that ignore the semantic information, our priority-based exploration algorithm enables the robot to efficiently explore more of the environment while satisfying its deadline constraints.

Autonomous Underwater Vehicles (AUVs) can be effective collaborators to human scuba divers in many applications, such as environmental surveying, mapping, or infrastructure repair. However, for these applications to be realized in the real world, it is essential that robots are able to both lead and follow their human collaborators. Current algorithms for diver following are not robust to non-uniform changes in the motion of the diver, and no framework currently exists for robots to lead divers. One method to improve the robustness of diver following and enable the capability of diver leading is to predict the future motion of a diver. In this paper, we present a vision-based approach for AUVs to predict the future motion trajectory of divers, utilizing the Vanilla-LSTM and Social-LSTM temporal deep neural networks. We also present a dense optical flow-based method to stabilize the input annotations from the dataset and reduce the effects of camera ego-motion. We analyze the results of these models on scenarios ranging from swimming pools to the open ocean and present the model’s accuracy at varying prediction lengths. We find that our LSTM models can generate predictions with significant accuracy 1.5 seconds into the future and that stabilizing LSTM models significantly improves trajectory prediction performance.

This paper introduces a novel approach for the grasping and precise placement of various known rigid objects using multiple grippers within highly cluttered scenes. Using a single depth image of the scene, our method estimates multiple 6D object poses together with an object class, a pose distance for object pose estimation, and a pose distance from a target pose for object placement for each automatically obtained grasp pose with a single forward pass of a neural network.By incorporating model knowledge into the system, our approach has higher success rates for grasping than state-of-the-art model-free approaches. Furthermore, our method chooses grasps that result in significantly more precise object placements than prior model-based work.

Micro-nano-robots are considered to be a promising platform for drug delivery in biological organisms, but there are still urgent technical problems in biocompatibility and degradability of 3D-printed-based micro-robots that need to be solved. Therefore, in this paper, we design a magnetized bio-hybrid robot, which uses mouse macrophages as carriers, and allowed it to swallow Fe2O3 particles with a diameter of 10 nm. The robot takes advantage of macrophage’s natural biocompatibility and targeting characteristics to reach and function in complex environments such as: eye, knee, tumor, etc., and finally being able to be actively metabolized by the organism. More importantly, the cell robot can move precisely along a preplanned path under the control of a three-dimensional magnetic control system built in this study, and be delivered accurately to the vicinity of cancer cells in vitro environment. In future work, cellular robots could be allowed to carry anti-cancer drugs and release them in a targeted manner at the lesion. These microrobots have shown great potential for tumor reginal targeted drug delivery.