ICRA 2020 Papers

This paper presents the design and performance of a new locomotion strategy for serpentine robots using screw propulsion. The ARCSnake robot comprises serially linked, identical modules, each incorporating an Archimedes' screw for propulsion and a universal joint (U-Joint) for orientation control. When serially chained, these modules form a versatile serpentine robot platform which enables the robot to reshape its body configuration for varying environments, typical of a snake. Furthermore, the Archimedes' screws allow for novel omni-wheel drive-like motions by speed controlling their screw threads. This paper considers the mechanical and electrical design, as well as the software architecture for realizing a fully integrated system. The system includes 3N actuators for N segments, each controlled using a BeagleBone Black with a customized power-electronics cape, a 9 Degrees of Freedom (DoF) Inertial Measurement Unit (IMU), and a scalable communication channel over ROS. This robot serves as the first proof-of-concept demonstration of the NASA-JPL Exobiology Extant Life Surveyor (EELS) program that aims to deliver scientific instrumentation deep within the plume vents, caves, and ice sheets of Enceladus and Europa in search for extant lifeforms*.

Surgical scene understanding and multi-tasking learning are crucial for image-guided robotic surgery. Training a real-time robotic system for the detection and segmentation of high-resolution images provides a challenging problem with the limited computational resource. The perception drawn can be applied in effective real-time feedback, surgical skill assessment, and human-robot collaborative surgeries to enhance surgical outcomes. For this purpose, we develop a novel end-to-end trainable real-time Multi-Task Learning (MTL) model with weight-shared encoder and task-aware detection and segmentation decoders. Optimization of multiple tasks at the same convergence point is vital and presents a complex problem. Thus, we propose an asynchronous task-aware optimization (ATO) technique to calculate task-oriented gradients and train the decoders independently. Moreover, MTL models are always computationally expensive, which hinder real-time applications. To address this challenge, we introduce a global attention dynamic pruning (GADP) by removing less significant and sparse parameters. We further design a skip squeeze and excitation (SE) module, which suppresses weak features, excites significant features and performs dynamic spatial and channel-wise feature re-calibration. Validating on the robotic instrument segmentation dataset of MICCAI endoscopic vision challenge, our model significantly outperforms state-of-the-art segmentation and detection models, including best-performed models in the challenge.

Deep Convolutional Neural Networks (DCNNs) are used extensively in medical image segmentation and hence 3D navigation for robot-assisted Minimally Invasive Surgeries (MISs). However, current DCNNs usually use down sampling layers for increasing the receptive field and gaining abstract semantic information. These down sampling layers decrease the spatial dimension of feature maps, which can be detrimental to image segmentation. Atrous convolution is an alternative for the down sampling layer. It increases the receptive field whilst maintains the spatial dimension of feature maps. In this paper, a method for effective atrous rate setting is proposed to achieve the largest and fully-covered receptive field with a minimum number of atrous convolutional layers. Furthermore, a new and full resolution DCNN - Atrous Convolutional Neural Network (ACNN), which incorporates cascaded atrous II-blocks, residual learning and Instance Normalization (IN) is proposed. Application results of the proposed ACNN to Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) image segmentation demonstrate that the proposed ACNN can achieve higher segmentation Intersection over Unions (IoUs) than U-Net and Deeplabv3+, but with reduced trainable parameters.

Exploration in environments with sparse feed-back remains a challenging research problem in reinforcement learning (RL). When the RL agent explores the environment randomly, it results in low exploration efficiency, especially in robotic manipulation tasks with high dimensional continuous state and action space. In this paper, we propose a novel method, called Augmented Curiosity-Driven Experience Replay (ACDER), which leverages (i) a new goal-oriented curiosity-driven exploration to encourage the agent to pursue novel and task-relevant states more purposefully and (ii) the dynamic initial states selection as an automatic exploratory curriculum to further improve the sample-efficiency. Our approach complements Hindsight Experience Replay (HER) by introducing a new way to pursue valuable states. Experiments conducted on four challenging robotic manipulation tasks with binary rewards, including Reach, Push, Pick&Place and Multi-step Push. The empirical results show that our proposed method significantly outperforms existing methods in the first three basic tasks and also achieves satisfactory performance in multi-step robotic task learning.

In order to relate information across cameras in a Dynamic Camera Cluster (DCC), an accurate time-varying set of extrinsic calibration transformations need to be determined. Previous calibration approaches rely solely on collecting measurements from a known fiducial target which limits calibration accuracy as insufficient excitation of the gimbal is achieved. In this paper, we improve DCC calibration accuracy by collecting measurements over the entire configuration space of the gimbal and achieve a 10X improvement in pixel re-projection error. We perform a joint optimization over the calibration parameters between any number of cameras and unknown joint angles using a pose-loop error optimization approach, thereby avoiding the need for overlapping fields-of-view. We test our method in simulation and provide a calibration sensitivity analysis for different levels of camera intrinsic and joint angle noise. In addition, we provide a novel analysis of the degenerate parameters in the calibration when joint angle values are unknown, which avoids situations in which the calibration cannot be uniquely recovered. The calibration code will be made available at https://github.com/TRAILab/AC-DCC.

With the increasing global popularity of self-driving cars, there is an immediate need for challenging real-world datasets for benchmarking and training various computer vision tasks such as 3D object detection. Existing datasets either represent simple scenarios or provide only day-time data. In this paper, we introduce a new challenging A*3D dataset which consists of RGB images and LiDAR data with a significant diversity of scene, time, and weather. The dataset consists of high-density images (≈ 10 times more than the pioneering KITTI dataset), heavy occlusions, a large number of nighttime frames (≈ 3 times the nuScenes dataset), addressing the gaps in the existing datasets to push the boundaries of tasks in autonomous driving research to more challenging highly diverse environments. The dataset contains 39K frames, 7 classes, and 230K 3D object annotations. An extensive 3D object detection benchmark evaluation on the A*3D dataset for various attributes such as high density, day-time/night-time, gives interesting insights into the advantages and limitations of training and testing 3D object detection in real-world setting.

Obstacle detection by semantic segmentation shows a great promise for autonomous navigation in unmanned surface vehicles (USV). However, existing methods suffer from poor estimation of the water edge in presence of visual ambiguities, poor detection of small obstacles and high false-positive rate on water reflections and wakes. We propose a new deep encoder-decoder architecture, a water-obstacle separation and refinement network (WaSR), to address these issues. Detection and water edge accuracy are improved by a novel decoder that gradually fuses inertial information from inertial measurement unit (IMU) with the visual features from the encoder. In addition, a novel loss function is designed to increase the separation between water and obstacle features early on in the network. Subsequently, the capacity of the remaining layers in the decoder is better utilised, leading to a significant reduction in false positives and increased true positives. Experimental results show that WaSR outperforms the current state-of-the-art by a large margin, yielding a 14% increase in F-measure over the second-best method.

The need for users' safety and technology accept-ability has incredibly increased with the deployment of co-bots physically interacting with humans in industrial settings, and for people assistance. A well-studied approach to meet these requirements is to ensure human-like robot motions. Classic solutions for anthropomorphic movement generation usually rely on optimization procedures, which build upon hypotheses devised from neuroscientific literature, or capitalize on learning methods. However, these approaches come with limitations, e.g. limited motion variability or the need for high dimensional datasets. In this work, we present a technique to directly embed human upper limb principal motion modes computed through functional analysis in the robot trajectory optimization. We report on the implementation with manipulators with redundant anthropomorphic kinematic architectures - although dissimilar with respect to the human model used for functional mode extraction - via Cartesian impedance control. In our experiments, we show how human trajectories mapped onto a robotic manipulator still exhibit the main characteristics of human-likeness, e.g. low jerk values. We discuss the results with respect to the state of the art, and their implications for advanced human-robot interaction in industrial co-botics and for human assistance.

A methodology for implementing arbitrary foot shapes in the passive walking dynamics of biped robots is developed. The dynamic model of a walking robot is defined in a way that allows shape-dependent foot kinetics to contribute to the robot's dynamics, for all convex foot shapes regardless of the exact foot geometry: for the developed method, only the set of points describing the foot profile curve is needed. The method is mathematically derived and then showcased with an application. The open-source pose estimation system OpenPose is used to determine the foot profile that enables the rigid-foot passive robot to reproduce the ankle trajectory of the actively powered, multi-DOF human foot complex. The passive gait of the biped robot walking on the specified foot shape is simulated and analyzed, and a stable walking cycle is found and evaluated. The proposed model enables the study of the effects of foot shape on the walking dynamics of biped robots, eliminating the necessity of solely using simple, and analytically defined geometric shapes as the walking robots' feet. The method can be used for foot shape optimization towards achieving any desired walking pattern in walking robots.

This paper addresses the problem of efficiently achieving visual predictive control tasks. To this end, a memory of motion, containing a set of trajectories built off-line, is used for leveraging precomputation and dealing with difficult visual tasks. Standard regression techniques, such as k-nearest neighbors and Gaussian process regression, are used to query the memory and provide on-line a warm-start and a way point to the control optimization process. The proposed technique allows the control scheme to achieve high performance and, at the same time, keep the computational time limited. Simulation and experimental results, carried out with a 7-axis manipulator, show the effectiveness of the approach.

We describe and experimentally demonstrate a practical method for indoor localization using measurements obtained from resource-constrained devices with limited sensing capabilities. We focus on handheld/mobile devices but the method can be useful for a variety of wearable devices. Our system works with sparse WiFi or image-based measurements, avoiding laborious site surveying for dense signal maps and runs in real-time. It uses Conditional Random Fields to infer the most probable sequence of agent positions from a known floor plan, dead reckoning and sparse absolute position estimates. Our solution leverages known topology of the environment by pre-computing allowed motion sequences of an agent, which are then used to constraint the motion inferred from the sensory data. The system is evaluated in a typical office building, demonstrating good accuracy and robustness to sparse, low-quality measurements.

Beyond the ultimate goal of prosthetics, repairing all the capabilities of amputees, the development line of upper-limb prostheses control mainly relies on three aspects: the robustness, the intuitiveness and the reduction of mental fatigue. Many complex structures and algorithms are proposed but no one question a common open-loop nature, where the user is the one in charge of correcting errors. Yet, closing the control loop at the prosthetic level may help to improve the three main lines of research cited above. One major issue to build a closed-loop control is the definition of a reliable error signal; this paper proposes to use body compensations, naturally exhibited by prostheses users when the motion of their device is inaccurate, as such. The described control scheme measures these compensatory movements and makes the prosthesis move in order to bring back the user into an ergonomic posture. The function of the prosthesis is no longer to perform a given motion but rather to correct the posture of its user while s/he focus on performing an endpoint task. This concept was validated and compared to a standard open-loop scheme, for the control of a prosthetic wrist, with five healthy subjects completing a dedicated task with a customized transradial prosthesis. Results show that the presented closed-loop control allows for more intuitiveness and less mental burden without enhancing body compensation.

This paper proposes a new kind of light-weight manipulators suitable for safe interactions. The proposed manipulators use anti-parallelogram joints in series, referred to as X-joints. Each X-joint is remotely actuated with cables and springs in parallel, thus realizing a tensegrity one-degree-of-freedom mechanism. As compared to manipulators built with simple revolute joints in series, manipulators with tensegrity X-joint offer a number of advantages, such as an intrinsic stability, variable stiffness and lower inertia. This new design was inspired by the musculosleketon architecture of the bird neck that is known to have remarkable features such as a high dexterity. The paper analyzes in detail the kinetostatics of a X-joint and proposes a 3-degree-of-freedom manipulator made of three such joints in series. Both simulation results and experiment results conducted on a test-bed prototype are presented and discussed.

We present a novel framework for object-level 3D underwater reconstruction using imaging sonar sensors. We demonstrate that imaging sonar reconstruction is analogous to the problem of confocal non-line-of-sight (NLOS) reconstruction. Drawing upon this connection, we formulate the problem as one of solving for volumetric albedo, where the scene of interest is modeled as a directionless albedo field. After discretization, reconstruction reduces to a convex linear optimization problem, which we can augment with a variety of priors and regularization terms. We show how to solve the resulting regularized problems using the alternating direction method of multipliers (ADMM) algorithm. We demonstrate the effectiveness of the proposed approach in simulation and on real-world datasets collected in a controlled, test tank environment with several different sonar elevation apertures.

This paper presents a visual positioning system (VPS) for real-time pose estimation of a robotic navigation aid (RNA) for assistive navigation. The core of the VPS is a new method called depth-enhanced visual-inertial odometry (DVIO) that uses an RGB-D camera and an inertial measurement unit (IMU) to estimate the RNA's pose. The DVIO method extracts the geometric feature (the floor plane) from the camera's depth data and integrates its measurement residuals with that of the visual features and the inertial data in a graph optimization framework for pose estimation. A new measure based on the Sampson error is introduced to describe the measurement residuals of the near-range visual features with a known depth and that of the far-range visual features whose depths are unknown. The measure allows for the incorporation of both types of visual features into graph optimization. The use of the geometric feature and the Sampson error improves pose estimation accuracy and precision. The DVIO method is paired with a particle filter localization (PFL) method to locate the RNA in a 2D floor plan and the information is used to guide a visually impaired person. The PFL reduces the RNA's position and heading error by aligning the camera's depth data with the floor plan map. Together, the DVIO and the PFL allow for accurate pose estimation for wayfinding and 3D mapping for obstacle avoidance. Experimental results demonstrate the usefulness of the RNA in assistive navigation in indoor spaces.

In today's automation driven manufacturing environments, emerging technologies like cobots (collaborative robots) and augmented reality interfaces can help integrating humans into the production workflow to benefit from their adaptability and cognitive skills. In such settings, humans are expected to work with robots side by side and physically interact with them. However, the trade-off between stability and transparency is a core challenge in the presence of physical human robot interaction (pHRI). While stability is of utmost importance for safety, transparency is required for fully exploiting the precision and ability of robots in handling labor intensive tasks. In this work, we propose a new variable admittance controller based on fractional order control to handle this trade-off more effectively. We compared the performance of fractional order variable admittance controller with a classical admittance controller with fixed parameters as a baseline and an integer order variable admittance controller during a realistic drilling task. Our comparisons indicate that the proposed controller led to a more transparent interaction compared to the other controllers without sacrificing the stability. We also demonstrate a use case for an augmented reality (AR) headset which can augment human sensory capabilities for reaching a certain drilling depth otherwise not possible without changing the role of the robot as the decision maker.

Soft continuum robots have many applications such as medical surgeries, service industries, rescue tasks, and underwater exploration. Flexibility and good accessibility of such robots are the key reasons for their popularity. However, the complexity of their structural design and control systems limit their broader applications. In this paper, a novel variable stiffness soft continuum robot based on pre-charged air, particle jamming, and origami is proposed. The robot is a bellow-like origami structure with internal chambers. A spine-like chamber is filled with particles, and three identical chambers surrounding the spine chamber are filled with pressurized air. When the origami structure is compressed, the particles are jammed by the compression force and the increased pressure of the three air chambers, thus increasing the overall stiffness of the robot. The robot expansion-contraction and bending are controlled by three tendons. An analytical model of the proposed stiffness variation mechanism has been developed. The effects of various parameters on the lateral and axial stiffness of the soft continuum robot have been investigated by experimental studies. A prototype robot has been fabricated to demonstrate grasping operations.

This paper presents a novel framework for dense piecewise semantic reconstruction in dynamic scenes containing complex background and moving objects via exploiting superpixel relations. We utilize two kinds of superpixel relations: motion relations and spatial relations, each having three subcategories: coplanar, hinge, and crack. Spatial relations provide constraints on the spatial locations of neighboring superpixels and thus can be used to reconstruct dynamic scenes. However, spatial relations can not be estimated directly with epipolar geometry due to moving objects in dynamic scenes. We synthesize the results of semantic instance segmentation and motion relations to estimate spatial relations. Given consecutive frames, we mainly develop our method in five main stages: preprocessing, motion estimation, superpixel relation analysis, reconstruction and refinement. Extensive experiments on various datasets demonstrate that our method outperforms competitors in reconstruction quality. Furthermore, our method presents a feasible way to incorporate semantic information in Structure-from-Motion (SFM) based reconstruction pipelines.

Soft robotic pneumatic grippers have been shown to be versatile, robust to impacts, and safe for use on delicate objects. One type, fluidic elastomer grippers, are characterized by fingers with an inextensible gripping surface backed by extensible pneumatic chambers; when inflated, this mismatch in extensibility results in the finger curling. However, one drawback of these simple fingers is that they have one preprogrammed grasp, usually a simple constant-curvature wrap. While well-suited for finger-sized round objects, they do not grasp flat or small objects well. Here, we present an adaptable tri-stable soft robotic finger that can form either a pinch or wrap grasp based on the shape of the grasped object. We enable this by incorporating two bi-stable springs into the inextensible layer. The three stable positions are: i) open (unpressurized), ii) pinch (with only the proximal section bending), and iii) wrap (with the entire finger bending). We present a simple model of the behavior of our finger and experimental results verifying the model. Further, we apply forces and moments to grasped objects, and show that the tri-stable finger increases the grasping performance when compared to a control gripper with equal gripping force. Our work presents a novel design modification that is unobtrusive, simple, and passive. Our introduction of inexpensive programmable hardware advances the versatility and adaptability of soft grippers.