ICRA 2021 论文列表

This paper proposes a novel visual SLAM method with dense planar reconstruction using a monocular camera: TT-SLAM. The method exploits planar template-based trackers (TT) to compute camera poses and reconstructs a multi-planar scene representation. Multiple homographies are estimated simultaneously by clustering a set of template trackers supported by superpixelized regions. Compared to RANSAC-based multiple homographies method [1], data association and keyframe selection issues are handled by the continuous nature of template trackers. A non-linear optimization process is applied to all the homographies to improve the precision in pose estimation. Experiments show that the proposed method outperforms RANSAC-based multiple homographies method [1] as well as other dense method SLAM techniques such as LSD-SLAM or DPPTAM, and competes with keypoint-based techniques like ORB-SLAM while providing dense planar reconstructions of the environment.

The ability to simultaneously track and reconstruct multiple objects moving in the scene is of the utmost importance for robotic tasks such as autonomous navigation and interaction. Virtually all of the previous attempts to map multiple dynamic objects have evolved to store individual objects in separate reconstruction volumes and track the relative pose between them. While simple and intuitive, such formulation does not scale well with respect to the number of objects in the scene and introduces the need for an explicit occlusion handling strategy. In contrast, we propose a map representation that allows maintaining a single volume for the entire scene and all the objects therein. To this end, we introduce a novel multi-object TSDF formulation that can encode multiple object surfaces at any given location in the map. In a multiple dynamic object tracking and reconstruction scenario, our representation allows maintaining accurate reconstruction of surfaces even while they become temporarily occluded by other objects moving in their proximity. We evaluate the proposed TSDF++ formulation on a public synthetic dataset and demonstrate its ability to preserve reconstructions of occluded surfaces when compared to the standard TSDF map representation. Code is available at https://github.com/ethz-asl/tsdf-plusplus.

Practical industrial assembly scenarios often require robotic agents to adapt their skills to unseen tasks quickly. While transfer reinforcement learning (RL) could enable such quick adaptation, much prior work has to collect many samples from source environments to learn target tasks in a model-free fashion, which still lacks sample efficiency on a practical level. In this work, we develop a novel transfer RL method named TRANSfer learning by Aggregating dynamics Models (TRANS-AM). TRANS-AM is based on model-based RL (MBRL) for its high-level sample efficiency, and only requires dynamics models to be collected from source environments. Specifically, it learns to aggregate source dynamics models adaptively in an MBRL loop to better fit the state-transition dynamics of target environments and execute optimal actions there. As a case study to show the effectiveness of this proposed approach, we address a challenging contact-rich peg-in-hole task with variable hole orientations using a soft robot. Our evaluations with both simulation and real-robot experiments demonstrate that TRANS-AM enables the soft robot to accomplish target tasks with fewer episodes compared when learning the tasks from scratch.

Semantic segmentation of point clouds is a key component of scene understanding for robotics and autonomous driving. In this paper, we introduce TORNADO-Net - a neural network for 3D LiDAR point cloud semantic segmentation. We incorporate a multi-view (bird-eye and range) projection feature extraction with an encoder-decoder ResNet architecture with a novel diamond context block. Current projection-based methods do not take into account that neighboring points usually belong to the same class. To better utilize this local neighbourhood information and reduce noisy predictions, we introduce a combination of Total Variation, Lovász-Softmax, and Weighted Cross-Entropy losses. We also take advantage of the fact that the LiDAR data encompasses 360◦ field of view and use circular padding. We demonstrate state-of-the-art results on the SemanticKITTI dataset and also provide thorough quantitative evaluations and ablation results.

Biological observations on tetrapods locomotion deduce that anti-phase synchronization (APS) between fore and rear parts is beneficial for achieving a high-speed walking. On the other hand, theoretical analysis and experimental studies on quadruped robots suggest that a flexible spine potentially improves the gait efficiency and adaptability via smoothing the ground collisions. However, these two mechanisms have never been placed together by a comprehensive investigation in terms of their synergetic effect. Namely, an advanced principle is still lacking in combining the APS and the spine flexibility for quadruped walking robots. To address this issue, we construct a mathematical model for a quadruped dynamic walker under different spine conditions. First, the APS effect is generated via entrainment-based control method under a rigid spine condition. Then, flexible spines realized by three kinds of springs are compared with the rigid one via theoretical analysis. The results suggest that the APS mechanism and the flexible spine can be synergized via an appropriate deformation control. The theoretical findings not only uncover locomotion control mechanisms for quadruped walking robots, but also provide additional understandings of tetrapods dynamic walking from a mechanical engineering point of view.

A two-wheeled self-balancing robot is a statically unstable non-linear system with strong coupling dynamics. Common practices in the development of control systems for such robots are either to linearise the region of application to be used with linear controllers or to use complex nonlinear controllers such as Fuzzy logic, Sliding Mode, and Neural Networks. Nonetheless, in this paper, we are proposing a novel to this field concept of switching control that would adjust its approach depending on the evaluation of the current states. The performance of the proposed controller was assessed against exemplary solely linear and solely non-linear controllers in simulated tests. The tested were evaluated against dynamic criteria (distance traveled, max. angular deviation, etc.), control criteria (settling time, % overshoot, etc.), and environmental criterion of energy consumption. The results showed an interesting behavior of the proposed controller, with superior performance in many cases.

Minimally invasive surgery mainly consists of a series of sub-tasks, which can be decomposed into basic gestures or contexts. As a prerequisite of autonomic operation, surgical gesture recognition can assist motion planning and decision-making, and build up context-aware knowledge to improve the surgical robot control quality. In this work, we aim to develop an effective surgical gesture recognition approach with an explainable feature extraction process.A Bidirectional Multi-Layer independently RNN (BMLindRNN) model is proposed in this paper, while spatial feature extraction is implemented via fine-tuning of a Deep Convolutional Neural Network (DCNN) model constructed based on the VGG architecture. To eliminate the black-box effects of DCNN, Gradient-weighted Class Activation Mapping (Grad-CAM) is employed. It can provide explainable results by showing the regions of the surgical images that have a strong relationship with the surgical gesture classification results.The proposed method was evaluated based on the suturing task with data obtained from the public available JIGSAWS database. Comparative studies were conducted to verify the proposed framework. Results indicated that the testing accuracy for the suturing task based on our proposed method is 87.13%, which outperforms most of the state-of-the-art algorithms.

Surface robots can have many applications due to multiple degrees of freedom. Accordingly, many open research questions arise due to the limited number of realized cases and insufficient theory foundation. For a surface robot that performs deformations with shear deformation, the inner product calculation in a local coordinate system on the robot generally depends on the shear angle. However, the shear angle cannot be accurately measured, making it difficult to control the robot with coordinate transformation. We present herein a geometric robot with coordinates that are as locally orthogonal as possible. This robot is embodied by adding thickness to a special circle group, called the S-isothermic surface. We propose the utilization of inverse kinematics in S-isothermic-surface robots and demonstrate the results of actual realization.

In this paper, we study the problem of autonomous and reliable positioning of a camera w.r.t. an object when only this latter is known but not the rest of the scene. We propose to combine the advantages and efficiency of a visual servoing scheme and the generalization ability of a generative adversarial network. The paper describes how to efficiently create a synthetic dataset in order to train a network that predicts an intermediate visual keyframe between two images. Subsequent predictions are used as visual features to autonomously converge towards the desired pose even for large displacements. We show that the proposed method can be used without any prior knowledge on the scene appearance except for the object itself, while being robust to various lighting conditions and specular surfaces. We provide experimental results, both in simulation and using a real service robot platform to validate and evaluate the effectiveness, robustness, and accuracy of our approach.

Robotic automation in surgery requires precise tracking of surgical tools and mapping of deformable tissue. Previous works on surgical perception frameworks require significant effort in developing features for surgical tool and tissue tracking. In this work, we overcome the challenge by exploiting deep learning methods for surgical perception. We integrated deep neural networks, capable of efficient feature extraction, into the tissue tracking and surgical tool tracking processes. By leveraging transfer learning, the deep-learning-based approach requires minimal training data and reduced feature engineering efforts to fully perceive a surgical scene. The framework was tested on three publicly available datasets, which use the da Vinci® Surgical System, for comprehensive analysis. Experimental results show that our framework achieves state-of-the-art tracking performance in a surgical environment by utilizing deep learning for feature extraction.

Straight line features have been increasingly utilized in visual SLAM and 3D reconstruction systems. The straight lines’ parameterization, parallel constraint, and coplanar constraint are studied in many recent works. In this paper, we explore the novel intersection constraint of straight lines for structure reconstruction. First, a minimum parameterized representation of ray-point-ray (RPR) structures is proposed to represent the intersection of two straight lines in the 3D space. Second, an efficient solver is designed for the camera pose estimation, which leverages the perpendicularity and intersection of straight lines. Third, we build a stereo visual odometry based on RPR features and evaluate it on the simulation and real datasets. The experimental results verify that the intersection constraints from RPR can effectively improve the accuracy and efficiency of line-based SLAM and reconstruction system.

In this paper, we present a meta-algorithm intended to accelerate many existing path optimization algorithms. The central idea of our work is to strategically break up a waypoint path into consecutive groupings called "pods," then optimize over various pods concurrently using parallel processing. Each pod is assigned a color, either blue or red, and the path is divided in such a way that adjacent pods of the same color have an appropriate buffer of the opposite color between them, reducing the risk of interference between concurrent computations. We present a path splitting algorithm to create blue and red pod groupings and detail steps for a meta-algorithm that optimizes over these pods in parallel. We assessed how our method works on a testbed of simulated path optimization scenarios using various optimization tasks and characterize how it scales with additional threads. We also compared our meta-algorithm on these tasks to other parallelization schemes. Our results show that our method more effectively utilizes concurrency compared to the alternatives, both in terms of speed and optimization quality.

Depth completion is an important task in computer vision and robotics applications, which aims at predicting accurate dense depth from a single RGB-LiDAR image. Convolutional neural networks (CNNs) have been widely used for depth completion to learn a mapping function from sparse to dense depth. However, recent methods do not exploit any 3D geometric cues during the inference stage and mainly rely on sophisticated CNN architectures. In this paper, we present a cascade and geometrically inspired learning framework for depth completion, consisting of three stages: view extrapolation, stereo matching, and depth refinement. The first stage extrapolates a virtual (right) view using a single RGB (left) and its LiDAR data. We then mimic the binocular stereo-matching, and as a result, explicitly encode geometric constraints during depth completion. This stage augments the final refinement process by providing additional geometric reasoning. We also introduce a distillation framework based on teacher-student strategy to effectively train our network. Knowledge from a teacher model privileged with real stereo pairs is transferred to the student through feature distillation. Experimental results on KITTI depth completion benchmark demonstrate that the proposed method is superior to state-of-the-art methods.

This paper presents a stereo object matching method that exploits both 2D contextual information from images as well as 3D object-level information. Unlike existing stereo matching methods that exclusively focus on the pixel-level correspondence between stereo images within a volumetric space (i.e., cost volume), we exploit this volumetric structure in a different manner. The cost volume explicitly encompasses 3D information along its disparity axis, therefore it is a privileged structure that can encapsulate the 3D contextual information from objects. However, it is not straightforward since the disparity values map the 3D metric space in a non-linear fashion. Thus, we present two novel strategies to handle 3D objectness in the cost volume space: selective sampling (RoISelect) and 2D-3D fusion (fusion-by-occupancy), which allow us to seamlessly incorporate 3D object-level information and achieve accurate depth performance near the object boundary regions. Our depth estimation achieves competitive performance in the KITTI dataset and the Virtual-KITTI 2.0 dataset.

A well known and well-studied feature of boats’ dynamic is the effect of steering-induced roll. This property is used by a technique called Rudder Roll Stabilization (RRS) to stabilise ships in waves in order to make the navigation safer and more pleasant. This technique is based on the generation of induced roll. Because of its specific application, studies have been limited to commercial vessels using a Single Propeller-Rudder System (SPRS). This study not only broadens the technique to any propulsion and steering mechanisms that can be used with RRS by introducing the thrust asymmetry, but it also incorporates the effect of the centrifugal forces that were previously left off. To prove the capabilities of the new concept, a test is effectuated employing an RC demonstrator fitted with a Differential Jet Pump System (DJPS), performing a turning circle test manoeuvre.

Soft robots can operate effectively inside confined spaces because their soft bodies can adapt to accommodate the geometry around them. When they interact with the environment, the presence of contact forces can dramatically change the dynamics of the robots. If a soft robot is in contact and the contact force is not known, the control action is still targeted for a free robot. Hence the robot may perform improper actions. Because a soft robot is deformable, it is quite challenging to determine the contact forces and the system states from sensor measurements. This paper proposes an observer design to estimate the states of a fabric-reinforced inflatable soft robot as well as the external contact forces. The soft robot is represented by the disc-thread model which results in a set of ordinary differential equations (ODEs). A linear parameter-varying (LPV) system including some subsystems is formed to represent the nonlinear robot. The observer is based on the sliding mode approach and includes a set of sub-observers corresponding to the subsystems in the LPV system. The observer is validated through simulations and an experiment. The simulation results show that the observer can estimate the angular positions and their rate of changes as well as assumed contact forces with no error in steady states. The experiment results display good tracking of the robot’s configurations compared to the ground truth data from the motion tracking system.

This paper introduces a general state estimation framework fusing multiple sensor information for hybrid wheeled-legged robots performing mobile manipulation tasks. At the core of the state estimator is a novel unified odometry for hybrid locomotion which can seamlessly maintain tracking and has no need to switch between stepping and rolling modes. To the best of our knowledge, the proposed odometry is the first work in this area. It is calculated based on the robot kinematics and instantaneous contact points of wheels with sensor inputs from IMU, joint encoders, joint torque sensors estimating wheel contact status, as well as RGB-D camera detecting geometric features of the terrain (e.g. elevation and surface normal vector). Subsequently, the odometry output is utilized as the motion model of a 3D Lidar map-based Monte Carlo Localization module for drift-free state estimation. As part of the framework, visual localization is integrated to provide high precision guidance for the robot movement relative to an object of interest. The proposed approach was verified thoroughly by two experiments conducted on the Pholus robot with OptiTrack measurements as ground truth.

Motion prediction of multiple agents in a dynamic scene is a crucial component in many real applications, including intelligent monitoring and autonomous driving. Due to the complex interactions among the agents and their interactions with the surrounding scene, accurate trajectory prediction is still a great challenge. In this paper, we propose a new method for robust trajectory prediction of multiple intelligent agents in a dynamic scene. The input of the method includes the observed trajectories of all agents, and optionally, the planning of the ego-agent and the surrounding high definition map at every time steps. Given observed trajectories, an efficient approach in a star computational topology is utilized to compute both the spatiotemporal interaction features and the current interaction features between the agents, where the time complexity scales linearly to the number of agents. Moreover, on an autonomous vehicle, the proposed prediction method can make use of the planning of ego-agent to improve the modeling of the interaction between surrounding agents. To increase the robustness to upstream perception noises, at the training stage, we randomly mask out the input data, a.k.a. the points on the observed trajectories of agents and the lane sequence. Experiments on autonomous driving and pedestrian-walking datasets demonstrate that the proposed method is not only effective when the planning of ego-agent and the high definition map are provided, but also achieves state-of-the-art performance with only the observed trajectories.

Staircase is a typical obstacle for the legged robot to overcome in buildings. This paper studies the stair climbing capability-based dimensional synthesis for a hexapod legged robot, i.e., exploring how to determine the leg length and the longitudinal body length concerning the target staircase in the mechanical design stage. In climbing a staircase, leg-staircase interference is one of the predominant issues. The three possible interference cases are illustrated in detail with a 2-DOF (degree of freedom) leg mechanism and the staircase size, based on the predefined tripod gait sequence. The mathematical relationships between the leg length, longitudinal body length, and the target staircase size are derived. The leg length and the body length are finally determined with the target staircase size. The virtual simulations and prototype experiments verify the effectiveness of the dimensional synthesis for the hexapod robot.