ICRA 2020 论文列表

Mechanical self-stability is often useful for controlling systems in uncertain and unstructured environments because it can regulate processes without explicit state observation or feedback computation. However, the performance of such systems is often not optimised, which begs the question how their dynamics can be naturally augmented by a control law to improve performance metrics. We propose a minimalistic approach to controlling mechanically self-stabilising systems by utilising model-based, feedforward bang-bang control at a global level and self-stabilizing dynamics at a local level. We demonstrate the approach in the height control problem of a sphere hovering in a vertical air jet - the so-called Bernoulli Ball. After developing a model to study the system and theoretically proving global asymptotic stability, we present the augmented controller and show how to enhance performance measures and plan behaviour. Our physical experiments show that the proposed control approach has a reduced time-to-target compared to the uncontrolled system without loss of stability (ranging from a 2.4 to 4.4 fold improvement) and that we can plan sequences of target positions at will.

Newborn infants are naturally attracted to human faces, a crucial source of information for social interaction. In robotics, acquisition of such information is crucial and social robots should also learn to exhibit such social skill. Deep learning algorithms are valid candidates to address the problem of face localisation. However, a major drawback of these methods is the large amount of data and human supervision needed in the training procedure. In this work, we propose a cognitive architecture to address autonomous learning from raw sensory signals without supervision. We demonstrate the success of our cognitive framework for the task of face localisation. The proposed cognitive architecture builds on existing work and uses audiovisual attention and a proactive stereo vision mechanism to autonomously direct a robot’s attentive focus towards human faces. The gathered information is used to incrementally generate a dataset that can be used to train a state-of-the-art deep network. The learning system imitates the typical learning process of infants and enhances the learning generalization process by leveraging on the interaction experience with people. The integration of HRI with machine learning, inspired by early development in humans, constitutes an innovative approach for improving autonomous learning in robots.

This paper proposes an end-to-end self-supervised feature representation network named Attentive Task-Net or AT-Net for video-based task imitation. The proposed AT-Net incorporates a novel multi-level spatial attention module to highlight spatial features corresponding to the intended task demonstrated by the expert. The neural connections in AT-Net ensure the relevant information in the demonstration is amplified and the irrelevant information is suppressed while learning task-specific feature embeddings. This is achieved by a weighted combination of multiple intermediate feature maps of the input image at different stages of the CNN pipeline. The weights of the combination are given by the compatibility scores, predicted by the attention module for respective feature maps. The AT-Net is trained using a metric learning loss which aims to decrease the distance between the feature representations of concurrent frames from multiple view points and increase the distance between temporally consecutive frames. The AT-Net features are then used to formulate a reinforcement learning problem for task imitation. Through experiments on the publicly available Multi-view pouring dataset, it is demonstrated that the output of the attention module highlights the task-specific objects while suppressing the rest of the background. The efficacy of the proposed method is further validated by qualitative and quantitative comparison with a state-of-the-art technique along with intensive ablation studies. The proposed method is implemented to imitate a pouring task where an RL agent is learned with the AT-Net in Gazebo simulator. Our findings show that the AT-Net achieves 6.5% decrease in alignment error along with a reduction in the number of training iterations by almost 155k over the state-of-the-art while satisfactorily imitating the intended task.

The real-time segmentation of surgical instruments plays a crucial role in robot-assisted surgery. However, it is still a challenging task to implement deep learning models to do real-time segmentation for surgical instruments due to their high computational costs and slow inference speed. In this paper, we propose an attention-guided lightweight network (LWANet), which can segment surgical instruments in real-time. LWANet adopts encoder-decoder architecture, where the encoder is the lightweight network MobileNetV2, and the decoder consists of depthwise separable convolution, attention fusion block, and transposed convolution. Depthwise separable convolution is used as the basic unit to construct the decoder, which can reduce the model size and computational costs. Attention fusion block captures global contexts and encodes semantic dependencies between channels to emphasize target regions, contributing to locating the surgical instrument. Transposed convolution is performed to upsample feature maps for acquiring refined edges. LWANet can segment surgical instruments in real-time while takes little computational costs. Based on 960x544 inputs, its inference speed can reach 39 fps with only 3.39 GFLOPs. Also, it has a small model size and the number of parameters is only 2.06 M. The proposed network is evaluated on two datasets. It achieves state-of-the- art performance 94.10% mean IOU on Cata7 and obtains a new record on EndoVis 2017 with a 4.10% increase on mean IOU.

Automatic surveillance and monitoring using Unmanned Aerial Systems (UAS) require the development of perception systems that robustly work under different illumination conditions. Event cameras are neuromorphic sensors that capture the illumination changes in the scene with very low latency and high dynamic range. Although recent advances in eventbased vision have explored the use of event cameras onboard UAS, most techniques group events in frames and, therefore, do not fully exploit the sequential and asynchronous nature of the event stream. This paper proposes a fully asynchronous scheme for intruder monitoring using UAS. It employs efficient event clustering and feature tracking modules and includes a sampling mechanism to cope with the computational cost of event-by-event processing adapting to on-board hardware computational constraints. The proposed scheme was tested on a real multirotor in challenging scenarios showing significant accuracy and robustness to lighting conditions.

This paper introduces a 2-DOF spatial remote center of motion (RCM) tensegrity mechanism, based on a double parallelogram system, dedicated for percutaneous needle insertion. The originality of this mechanism is its ability to be reconfigured and its capacity to perform a decoupled modulation of its stiffness in an asynchronous way. To do so, an analytical stiffness model of the robot is established, and a control methodology is proposed. A prototype of the robot is developed and assessed experimentally. The position tracking is evaluated using a 6-DOF magnetic tracker sensor showing a root mean square error less than 0.8° in both directions of the needle guide.

Multilateration systems reconstruct the location of a target that transmits electromagnetic or acoustic signals. The employed measurements for localization are the times of arrival (TOAs) of the transmitted signal, measured by a number of spatially distributed receivers at known positions. We present a novel multilateration algorithm to localize multiple targets that transmit indistinguishable signals at unknown times. That is, each receiver measures merely a set of TOAs with no association to the targets. Our method does not need any prior information. Therefore, it can provide uncorrelated, static measurements to be introduced into a separate tracker subsequently, or an initialization routine for multi target trackers.

Autonomous robots have the potential to serve as versatile caregivers that improve quality of life for millions of people worldwide. Yet, conducting research in this area presents numerous challenges, including the risks of physical interaction between people and robots. Physics simulations have been used to optimize and train robots for physical assistance, but have typically focused on a single task. In this paper, we present Assistive Gym, an open source physics simulation framework for assistive robots that models multiple tasks. It includes six simulated environments in which a robotic manipulator can attempt to assist a person with activities of daily living (ADLs): itch scratching, drinking, feeding, body manipulation, dressing, and bathing. Assistive Gym models a person's physical capabilities and preferences for assistance, which are used to provide a reward function. We present baseline policies trained using reinforcement learning for four different commercial robots in the six environments. We demonstrate that modeling human motion results in better assistance and we compare the performance of different robots. Overall, we show that Assistive Gym is a promising tool for assistive robotics research.

This paper proposes a relatively simple function of assistive force for a belt-type hip assist suit developed by the authors' group. The function, which is inspired by the muscle force of the rectus femoris, contains only two parameters, the magnitude and a phase shift factor. Thus, it can reduce the amount of calculation in generating the desired assistive force during walking. Tests were performed on three healthy subjects to confirm its effect and to investigate its influence on the motions of hip, knee and ankle joints. It was demonstrated that the effect of the assist depended greatly on the phase shift factor, i.e., the location of the peak of the assistive force in a swing period. A large effect was observed when the peak of the assistive force came at mid-swing phase. The results of the tests showed that the proposed force function could help to increase walk ratio (the ratio of step length to the number of steps per minute) by an average value of 11.2% at a force magnitude of 35 N, which could produce an assistive torque of the same order as the magnitude of the muscle force of the rectus femoris around the hip joint.

In industry assembly lines, parts feeding machines are widely employed as the prologue of the whole procedure. They play the role of sorting the parts randomly placed in bins to the state with specified pose. With the help of the parts feeding machines, the subsequent assembly processes by robot arm can always start from the same condition. Thus it is expected that function of parting feeding machine and the robotic assembly can be integrated with one robot arm. This scheme can provide great flexibility and can also contribute to reduce the cost. The difficulties involved in this scheme lie in the fact that in the part feeding phase, the pose of the part after grasping may be not proper for the subsequent assembly. Sometimes it can not even guarantee a stable grasp. In this paper, we proposed a method to integrate parts feeding and assembly within one robot arm. This proposal utilizes a specially designed gripper tip mounted on the jaws of a two-fingered gripper. With the modified gripper, in-hand manipulation of the grasped object is realized, which can ensure the control of the orientation and offset position of the grasped object. The proposal in this paper is verified by a simulated assembly in which a robot arm completed the assembly process including parts picking from bin and a subsequent peg-in-hole assembly.

Many manipulation tasks require coordinated motions for arm and fingers. Complexity increases when the task requires to control for the force at contact against a non-flat surface; This becomes even more challenging when this contact is done on a human. All these challenges are regrouped when one, for instance, massages a human limb. When massaging, the robotic arm is required to continuously adapt its orientation and distance to the limb while the robot fingers exert desired patterns of forces and motion on the skin surface. To address these challenges, we adopt a Dynamical System (DS) approach that offers a unified motion-force control approach and enables to easily coordinate multiple degrees of freedom. As each human limb may slightly differ, we learn a model of the surface using support vector regression (SVR) which enable us to obtain a distance-to-surface mapping. The gradient of this mapping, along with the DS, generates the desired motions for the interaction with the surface. A DS-based impedance control for the robotic fingers allows to control separately for force along the normal direction of the surface while moving in the tangential plane. We validate our approach using the KUKA IIWA robotic arm and Allegro robotic hand for massaging a mannequin arm covered with a skin-like material. We show that our approach allows for 1) reactive motion planning to reach for an unknown surface, 2) following desired motion patterns on the surface, and 3) exerting desired interaction forces profiles. Our results show the effectiveness of our approach; especially the robustness toward uncertainties for shape and the given location of the surface.

Service robots should be able to operate autonomously in dynamic and daily changing environments over an extended period of time. While Simultaneous Localization And Mapping (SLAM) is one of the most fundamental problems for robotic autonomy, most existing SLAM works are evaluated with data sequences that are recorded in a short period of time. In real-world deployment, there can be out-of-sight scene changes caused by both natural factors and human activities. For example, in home scenarios, most objects may be movable, replaceable or deformable, and the visual features of the same place may be significantly different in some successive days. Such out-of-sight dynamics pose great challenges to the robustness of pose estimation, and hence a robot’s long-term deployment and operation. To differentiate the forementioned problem from the conventional works which are usually evaluated in a static setting in a single run, the term lifelong SLAM is used here to address SLAM problems in an ever-changing environment over a long period of time. To accelerate lifelong SLAM research, we release the OpenLORIS-Scene datasets. The data are collected in real-world indoor scenes, for multiple times in each place to include scene changes in real life. We also design benchmarking metrics for lifelong SLAM, with which the robustness and accuracy of pose estimation are evaluated separately. The datasets and benchmark are available online at lifelong-robotic-vision.github.io/dataset/scene.

Current clinical endovascular interventions rely on 2D guidance for catheter manipulation. Although an aortic 3D surface is available from the pre-operative CT/MRI imaging, it cannot be used directly as a 3D intra-operative guidance since the vessel will deform during the procedure. This paper aims to reconstruct the live 3D aortic deformation by fusing the static 3D model from the pre-operative data and the 2D live imaging from fluoroscopy. In contrast to some existing deformation reconstruction frameworks which require 3D observations such as RGB-D or stereo images, fluoroscopy only presents 2D information. In the proposed framework, a 2D-3D registration is performed and the reconstruction process is formulated as a non-linear optimization problem based on the deformation graph approach. Detailed simulations and phantom experiments are conducted and the result demonstrates the reconstruction accuracy and robustness, as well as the potential clinical value of this framework.

In order to solve complex, long-horizon tasks, intelligent robots need to carry out high-level, abstract planning and reasoning in conjunction with motion planning. However, abstract models are typically lossy and plans or policies computed using them can be unexecutable. These problems are exacerbated in stochastic situations where the robot needs to reason about, and plan for multiple contingencies. We present a new approach for integrated task and motion planning in stochastic settings. In contrast to prior work in this direction, we show that our approach can effectively compute integrated task and motion policies whose branching structures encoding agent behaviors handling multiple execution-time contingencies. We prove that our algorithm is probabilistically complete and can compute feasible solution policies in an anytime fashion so that the probability of encountering an unresolved contingency decreases over time. Empirical results on a set of challenging problems show the utility and scope of our methods.

Object detection and motion parameters estimation are crucial tasks for self-driving vehicle safe navigation in a complex urban environment. In this work we propose a novel real-time approach of temporal context aggregation for motion detection and motion parameters estimation based on 3D point cloud sequence. We introduce an ego-motion compensation layer to achieve real-time inference with performance comparable to a naive odometric transform of the original point cloud sequence. Not only is the proposed architecture capable of estimating the motion of common road participants like vehicles or pedestrians but also generalizes to other object categories which are not present in training data. We also conduct an in-deep analysis of different temporal context aggregation strategies such as recurrent cells and 3D convolutions. Finally, we provide comparison results of our state-of-the-art model with existing solutions on KITTI Scene Flow dataset.

A service robot is expected to provide proactive service for visitors who require its help. In contrast to passive service, e.g., providing service only after being spoken to, proactive service initiates an interaction at an early stage, e.g., talking to potential visitors who need the robot’s help in advance. This paper addresses how to anticipate the start of user interaction. We propose an approach using only a single RGB camera that anticipates whether a visitor will come to the robot for interaction or just pass it by. In the proposed approach, we (i) utilize the visitor’s pose information from captured images incorporating facial information, (ii) train a CNN-LSTM–based model in an end-to-end manner with an exponential loss for early anticipation, and (iii) during the training, the network branch for facial keypoints acquired as the part of the human pose information is taught to mimic the branch trained with the face image from a specialized face detector with a human verification. By virtue of (iii), at the inference, we can run our model in an embedded system processing only the pose information without an additional face detector and typical accuracy drop.We evaluated the proposed approach on our collected real world data with a real service robot and publicly available JPL interaction dataset and found that it achieved accurate anticipation performance.

It is common knowledge that tractor-trailer vehicles are affected by jackknifing, a phenomenon that consists in the divergence of the trailer hitch angle and ultimately causes the vehicle to fold up. For the case of backwards motion, in which jackknifing can also occur at low speeds, we present a control method that drives the vehicle along a reference Cartesian trajectory while avoiding the divergence of the hitch angle. In particular, a feedback control law is obtained by combining two actions: a tracking term, computed using input-output linearization, and a corrective term, generated via IS-MPC, an intrinsically stable MPC scheme which is effective for stable inversion of nonminimum-phase systems. The proposed method has been verified in simulation and experimentally validated on a purposely built prototype.

Autonomous vehicles are sharing the road with human drivers. In order to facilitate interactive driving and cooperative behavior in dense traffic, a thorough understanding and representation of other traffic participants' behavior are necessary. Cost functions (or reward functions) have been widely used to describe the behavior of human drivers since they can not only explicitly incorporate the rationality of human drivers and the theory of mind (TOM), but also share similarity with the motion planning problem of autonomous vehicles. Hence, more human-like driving behavior and comprehensible trajectories can be generated to enable safer interaction and cooperation. However, the selection of cost functions in different driving scenarios is not trivial, and there is no systematic summary and analysis for cost function selection and learning from a variety of driving scenarios. In this work, we aim to investigate to what extent cost functions are suitable for explaining and imitating human driving behavior. Further, we focus on how cost functions differ from each other in different driving scenarios. Towards this goal, we first comprehensively review existing cost function structures in literature. Based on that, we point out required conditions for demonstrations to be suitable for inverse reinforcement learning (IRL). Finally, we use IRL to explore suitable features and learn cost function weights from human driven trajectories in three different scenarios.

Serial manipulator kinematics provide a mapping between joint variables in joint-space coordinates, and end-effector configurations in task-space Cartesian coordinates. Velocity mappings are represented via the manipulator Jacobian produced by direct differentiation of the forward kinematics. Acquisition of acceleration, jerk, and snap expressions, typically utilized for accurate trajectory-tracking, requires the computation of high-order Jacobian derivatives. As compared to conventional numerical/D-H approaches, this paper proposes a novel methodology to derive the Jacobians and their high-order derivatives symbolically, based on Lie theory, which requires that the derivatives are calculated with respect to each joint variable and time. Additionally, the technique described herein yields a mathematically sound solution to the high-order Jacobian derivatives, which distinguishes it from other relevant works. Performing computations with respect to the two inertial-fixed and body-fixed frames, the analytical form of the spatial and body Jacobians are derived, as well as their higher-order derivatives, without resorting to any approximations, whose expressions would depend explicitly on the joint state and the choice of reference frames. The proposed method provides more tractable computation of higher-order Jacobian derivatives, while its effectiveness has been verified by conducting a comparative analysis based on experimental data extracted from a KUKA LRB iiwa7 R800 manipulator.