ICRA 2019 论文列表

State-of-the-art deep neural network recognition systems are designed for a static and closed world. It is usually assumed that the distribution at test time will be the same as the distribution during training. As a result, classifiers are forced to categorise observations into one out of a set of predefined semantic classes. Robotic problems are dynamic and open world; a robot will likely observe objects that are from outside of the training set distribution. Classifier outputs in robotic applications can lead to real-world robotic action and as such, a practical recognition system should not silently fail by confidently misclassifying novel observations. We show how a deep metric learning classification system can be applied to such open set recognition problems, allowing the classifier to label novel observations as unknown. Further to detecting novel examples, we propose an open set active learning approach that allows a robot to efficiently query a user about unknown observations. Our approach enables a robot to improve its understanding of the true distribution of data in the environment, from a small number of label queries. Experimental results show that our approach significantly outperforms comparable methods in both the open set recognition and active learning problems.

3D multi-object detection and tracking are crucial for traffic scene understanding. However, the community pays less attention to these areas due to the lack of a standardized benchmark dataset to advance the field. Moreover, existing datasets (e.g., KITTI [1]) do not provide sufficient data and labels to tackle challenging scenes where highly interactive and occluded traffic participants are present. To address the issues, we present the Honda Research Institute 3D Dataset (H3D), a large-scale full-surround 3D multi-object detection and tracking dataset collected using a 3D LiDAR scanner. H3D comprises of 160 crowded and highly interactive traffic scenes with a total of 1 million labeled instances in 27,721 frames. With unique dataset size, rich annotations, and complex scenes, H3D is gathered to stimulate research on full-surround 3D multi-object detection and tracking. To effectively and efficiently annotate a large-scale 3D point cloud dataset, we propose a labeling methodology to speed up the overall annotation cycle. A standardized benchmark is created to evaluate full-surround 3D multi-object detection and tracking algorithms. 3D object detection and tracking algorithms are trained and tested on H3D. Finally, sources of errors are discussed for the development of future algorithms.

A robot cannot be warm and friendly - or can it? To explore whether a robot can be a medical receptionist, we developed a robotic system for interacting with patients at a doctor's clinic, including acting friendly. We designed the robot to interact naturally with patients at the start and finish of a clinic visit. We investigated people's perceptions to the robot in a wizard-of-Oz study, where the participants interacted with the robot over four interactions. 40 participants evaluated the robot. The results indicate the participants thought the robot could be a friendly receptionist, especially after repeated interactions with the robot. However, the participants mainly thought the robot was friendly in a “professional” way, rather than a personal friend.

Myoelectric prosthetic hand users have difficulty with, and frequently avoid, grasping fragile objects with their prosthesis. While the sense of touch is known to be critical for human hand dexterity, it has been virtually absent in prosthetic hands. In this study, a standard myoelectric prosthetic hand was modified with tactile sensors and a simple tactile reflex to inhibit excessive forces on contact. The tactile sensors were made from an open-cell self-skinning polyurethane foam that produced a detectable increase in air pressure inside the foam when contacted. This contact signal was then used by an inhibitory reflex controller which served to reduce the gain of weaker closing signals after contact but allow stronger closing signals to pass through. Four unilateral myoelectric prosthesis users completed five trials of three different timed grasping tasks with fragile and rigid items. Subjects performed each task in three different scenarios: with their sound side limb, their current myoelectric hand, and the modified prosthesis with tactile reflex. Findings demonstrated that grasping performance with fragile objects was significantly enhanced using the modified prosthesis, even nearing the performance of subject's sound side limb. Results suggest that this approach can substantially improve the speed and success of grasping fragile items, leading to improved use patterns, decreased cognitive effort, and improved user confidence.

Current minimally-invasive surgical tools suffer from lack of scalability and restricted access to some surgical sites using a laparoscopic probe. This paper introduces a proof-of-concept prototype of the first completely wireless surgical scissors capable of dexterous motion and cutting in a remote environment as a mobile microrobotic device. The 15 mm untethered surgical scissors are custom made from sharpened titanium sheets with a magnet on each blade for actuating force and control. A super-elastic nitinol wire acts as a restoring spring and results in a simple design with no pin joint which is difficult to fabricate at small sizes. To actuate and control the scissors, a 3D magnetic coil system is used here for testing and demonstration. An external magnetic flux density of 20 mT can be generated using the coils and is used for cutting as well as orienting, moving and closing the scissors. In this first prototype setup, the scissors can generate up to 75 mN of cutting force, and we demonstrate the cutting of agar. As a proof of concept demonstration of the potential use of the scissors as a completely untethered surgical tool, we robotically maneuver the scissors to a target location in a confined environment where they cut through agar and return to their initial position.

Performing ultrasound procedures from a remote site is a challenging task since both a stable behavior, for the safety of the patient, and a high-level of usability, to exploit the sonographer's expertise, need to be guaranteed. Furthermore, a teleoperation system that provides such requirements has to deal with communication delays as well. To address this issue, we use the two-layer algorithm: a passivity-based bilateral teleoperation architecture able to guarantee stability despite unknown and time-varying delay. Its flexibility allows to implement different kinds of control laws. In a Tele-Echography system, the slave manipulator has to apply significant forces needed by the procedure whereas the haptic device at the master side should be very light to avoid tiring the operator. Therefore, the energy needed by these two robots to perform their movements is very different and the energy injected into the system by the operator is often not sufficient to implement the desired action at the slave side. Methods to overcome this problem require to perfectly know the dynamical models of the robots. The solution proposed in this paper does not require such knowledge and is based on properly scaling the energy exchanged between the master and the slave side. We show the effectiveness of this approach in a real setup using a TOUCH haptic device and a WAM Barrett robot holding an ultrasound probe.

The minimalist robot designs typically employed in swarms and teams can fall and get trapped when traversing irregular terrain. To protect against this contingency the design could add a specialized escape actuator, but each actuator drives up cost multiplicatively for the whole team. Instead, the emergency actuator can be found for free in the form of another teammate. Teammate pushing can be efficiently directed by careful shaping of the robot's exterior hull. This approach is illustrated by designing a shell for VelociRoACH robots that enables them to roll pronated comrades back onto their feet. The designed maneuver can be performed in open-loop with 87% success and an average time of 0.7 seconds.

In this paper, we introduce an approach which enables manipulator robots to write handwritten characters or line drawings. Given an image of just-drawn handwritten characters, the robot infers a plan to replicate the image with a writing utensil, and then reproduces the image. Our approach draws each target stroke in one continuous drawing motion and does not rely on handcrafted rules or on predefined paths of characters. Instead, it learns to write from a dataset of demonstrations. We evaluate our approach in both simulation and on two real robots. Our model can draw handwritten characters in a variety of languages which are disjoint from the training set, such as Greek, Tamil, or Hindi, and also reproduce any stroke-based drawing from an image of the drawing.

Our goal is to develop a principled and general algorithmic framework for task-driven estimation and control for robotic systems. State-of-the-art approaches for controlling robotic systems typically rely heavily on accurately estimating the full state of the robot (e.g., a running robot might estimate joint angles and velocities, torso state, and position relative to a goal). However, full state representations are often excessively rich for the specific task at hand and can lead to significant computational inefficiency and brittleness to errors in state estimation. In contrast, we present an approach that eschews such rich representations and seeks to create task-driven representations. The key technical insight is to leverage the theory of information bottlenecks to formalize the notion of a “task-driven representation” in terms of information theoretic quantities that measure the minimality of a representation. We propose novel iterative algorithms for automatically synthesizing (offline) a task-driven representation (given in terms of a set of task-relevant variables (TRVs)) and a performant control policy that is a function of the TRVs. We present online algorithms for estimating the TRVs in order to apply the control policy. We demonstrate that our approach results in significant robustness to unmodeled measurement uncertainty both theoretically and via thorough simulation experiments including a spring-loaded inverted pendulum running to a goal location.

The great promise of modular robots is the ability to create on demand robots; however, choosing the “right” design based on a task is still a challenging problem. In this paper, we present an approach to automatically synthesize both the design and control for modular robots from a task description. In particular, we focus on manipulators composed of one degree-of-freedom (DoF) modules. Our approach is able to handle partially infeasible tasks by either identifying the infeasible part and finding a design that satisfies the feasible part or searching for multiple designs that together satisfy the entire task. We compare our approach to a baseline genetic algorithm in a series of increasingly complex environments.

This work studies the problem of shape reconstruction and object localization using a vision-based tactile sensor, GelSlim. The main contributions are the recovery of local shapes from contact, an approach to reconstruct the tactile shape of objects from tactile imprints, and an accurate method for object localization of previously reconstructed objects. The algorithms can be applied to a large variety of 3D objects and provide accurate tactile feedback for in-hand manipulation. Results show that by exploiting the dense tactile information we can reconstruct the shape of objects with high accuracy and do on-line object identification and localization, opening the door to reactive manipulation guided by tactile sensing. We provide videos and supplemental information in the project's website web.mit.edu/mcube/research/tactile localization.html.

We describe the design and physical realization of a novel branching continuum robot, aimed at inspection and cleaning operations in hard-to-reach environments at depths greater than human arm lengths. The design, based on a hybrid concentric-tube/tendon actuated continuum trunk core, features two pairs of fully retractable continuum branches. The retractable nature of the branches allows the robot to actively change its topology, allowing it to penetrate narrow openings and expand to adaptively engage complex environmental geometries. We detail and discuss the realization of a physical prototype of the design, and its testing in a simulated glove box environment.

Fault-tolerant architectures are mandatory to ensure the robustness of autonomous robots performing missions in complex and uncertain environments. The first step of a fault-tolerant mechanism is the detection of a faulty behavior of the system. It is then important to provide tools to help robot developers specify relevant observers. It is moreover crucial to guarantee a correct implementation of the observers, i.e. that the observers do not miss data and do not trigger unsuitable recovery actions in case of false detection. In this paper, we propose a specification language for observers that uses Past-Time LTL to express complex formulas on data produced by software components, and timed constraints on the evaluations of these formulas. We moreover provide an implementation of this specification that guarantees a real-time evaluation of the observers. We briefly describe the observers we have specified for a patrolling mission, and we evaluate the performance of our approach compared to state of the art on a benchmark in which we detect errors on a laser range sensor.

Securing multi-robot teams against malicious activity is crucial as these systems accelerate towards widespread societal integration. This emerging class of “physical networks” requires research into new methods of security that exploit their physical nature. This paper derives a theoretical framework for securing multi-agent consensus against the Sybil attack by using the physical properties of wireless transmissions. Our frame-work uses information extracted from the wireless channels to design a switching signal that stochastically excludes potentially untrustworthy transmissions from the consensus. Intuitively, this amounts to selectively ignoring incoming communications from untrustworthy agents, allowing for consensus to the true average to be recovered with high probability if initiated after a certain observation time T0 that we derive. This work is different from previous work in that it allows for arbitrary malicious node values and is insensitive to the initial topology of the network so long as a connected topology over legitimate nodes in the network is feasible. We show that our algorithm will recover consensus and the true graph over the system of legitimate agents with an error rate that vanishes exponentially with time.

Omnidirectional depth sensing has its advantage over the conventional stereo systems since it enables us to recognize the objects of interest in all directions without any blind regions. In this paper, we propose a novel wide-baseline omnidirectional stereo algorithm which computes the dense depth estimate from the fisheye images using a deep convolutional neural network. The capture system consists of multiple cameras mounted on a wide-baseline rig with ultra-wide field of view (FOV) lenses, and we present the calibration algorithm for the extrinsic parameters based on the bundle adjustment. Instead of estimating depth maps from multiple sets of rectified images and stitching them, our approach directly generates one dense omnidirectional depth map with full 360° coverage at the rig global coordinate system. To this end, the proposed neural network is designed to output the cost volume from the warped images in the sphere sweeping method, and the final depth map is estimated by taking the minimum cost indices of the aggregated cost volume by SGM. For training the deep neural network and testing the entire system, realistic synthetic urban datasets are rendered using Blender. The experiments using the synthetic and real-world datasets show that our algorithm outperforms the conventional depth estimation methods and generate highly accurate depth maps.

The real-time and robust surgical instrument segmentation is an important issue for endoscopic vision. We propose an instrument segmentation method fusing the convolutional neural networks (CNN) prediction and the kinematic pose information. First, the CNN model ToolNet-C is designed, which cascades a convolutional feature extractor trained over numerous unlabeled images and a pixel-wise segmentor trained on few labeled images. Second, the silhouette projection of the instrument body onto the endoscopic image is implemented based on the measured kinematic pose. Third, the particle filter with the shape matching likelihood and the weight suppression is proposed for data fusion, whose estimate refines the kinematic pose. The refined pose determines an accurate silhouette mask, which is the final segmentation output. The experiments are conducted with a surgical navigation system, several animal-tissue backgrounds, and a debrider instrument.

Achieving high surface reconstruction accuracy in dense mapping has been a desirable target for both robotics and vision communities. In the robotics literature, simultaneous localization and mapping (SLAM) systems use RGB-D cameras to reconstruct a dense map of the environment. They leverage the depth input to provide accurate local pose estimation and a locally consistent model. However, drift in the pose tracking over time leads to misalignments and artifacts. On the other hand, offline computer vision methods, such as the pipeline that combines structure-from-motion (SfM) and multi-view stereo (MVS), estimate the camera poses by performing batch optimization. These methods achieve global consistency, but suffer from heavy computation loads. We propose a novel approach that integrates both methods to achieve locally and globally consistent reconstruction. First, we estimate poses of keyframes in the offline SfM pipeline to provide strong global constraints at relatively low cost. Afterwards, we compute odometry between frames driven by off-the-shelf SLAM systems with high local accuracy. We fuse the two pose estimations using factor graph optimization to generate accurate camera poses for dense reconstruction. Experiments on real-world and synthetic datasets demonstrate that our approach produces more accurate models comparing to existing dense SLAM systems, while achieving significant speedup with respect to state-of-the-art SfM-MVS pipelines.

The high agility of legged systems allows them to operate in rugged outdoor environments. In these situations, knowledge about the terrain geometry is key for foothold planning to enable safe locomotion. However, on penetrable or highly compliant terrain (e.g. grass) the visibility of the supporting ground surface is obstructed, i.e. it cannot directly be perceived by depth sensors. We present a method to estimate the underlying terrain topography by fusing haptic information about foot contact closure locations with exteroceptive sensing. To obtain a dense support surface estimate from sparsely sampled footholds we apply Gaussian process regression. Exteroceptive information is integrated into the support surface estimation procedure by estimating the height of the penetrable surface layer from discrete penetration depth measurements at the footholds. The method is designed such that it provides a continuous support surface estimate even if there is only partial exteroceptive information available due to shadowing effects. Field experiments with the quadrupedal robot ANYmal show how the robot can smoothly and safely navigate in dense vegetation.

Recent techniques in self-supervised monocular depth estimation are approaching the performance of supervised methods, but operate in low resolution only. We show that high resolution is key towards high-fidelity self-supervised monocular depth prediction. Inspired by recent deep learning methods for Single-Image Super-Resolution, we propose a subpixel convolutional layer extension for depth super-resolution that accurately synthesizes high-resolution disparities from their corresponding low-resolution convolutional features. In addition, we introduce a differentiable flip-augmentation layer that accurately fuses predictions from the image and its horizontally flipped version, reducing the effect of left and right shadow regions generated in the disparity map due to occlusions. Both contributions provide significant performance gains over the state-of-the-art in self-supervised depth and pose estimation on the public KITTI benchmark. A video of our approach can be found at https://youtu.be/jKNgBeBMx0I.