ICRA 2017 Papers

A multilayer fabrication method has been used to create crawling soft robots based on dielectric elastomer actuators. These actuators are created without the need for pre-stretch, eliminating the need for rigid components. A four-legged, multi-gait capable crawler can be fabricated in a matter of hours and shows promise for future untethered systems. Studies on inchworm robots show them to be the fastest dielectric elastomer actuator-based systems reported to date, capable of traveling faster than 1 body length/second for the best elastomer available. Most importantly, the devices are primarily soft and deformable, with few rigid attachments.

This paper addresses the problem of grasping unknown objects with a humanoid robot. Conventional approaches fail when the shape, dimension or pose of the objects are missing. We propose a novel approach in which the grasping problem is solved by modeling the object and the volume graspable by the hand with superquadric functions. The object model is computed in real-time using stereo vision. Pose computation is formulated as a nonlinear constrained optimization problem, which is solved in real-time using the Ipopt software package. Notably, our method finds solutions in which the fingers are located on portions of the object that are occluded by vision. The performance of our approach has been assessed on a real robotic system, the iCub humanoid robot. The experiments show that the proposed method computes proper poses, suitable for grasping even small objects, while avoiding hitting the table with the fingers.

We present a global controller for tracking nominal trajectories with a flying wing tailsitter vehicle. The control strategy is based on a first-principles model of the vehicle dynamics that captures all relevant aerodynamic effects, and we apply an onboard parameter learning scheme in order to estimate unknown aerodynamic parameters. A cascaded control architecture is used: Based on position and velocity errors an outer control loop computes a desired attitude keeping the vehicle in coordinated flight, while an inner control loop tracks the desired attitude using a lookup table with precomputed optimal attitude trajectories. The proposed algorithms can be implemented on a typical microcontroller and the performance is demonstrated in various experiments.

Piezoelectric materials are an attractive option for electromechanical transduction on the mesoscale due to their intrinsic high force production, large bandwidth, and favorable scaling characteristics. However, the small displacements they inherently produce are typically too small to be directly used in robotic systems, and thus displacement amplification is needed. Here we present a piezoelectric actuator that uses geometric amplification to achieve 20 × the nominal piezoelectric displacement. Actuator performance is described in terms of blocked force (20 mN), displacement (115 μm), bandwidth (3 kHz), and power density (172 W/kg). The actuator is fabricated using printed circuit MEMS, an emerging mesoscale manufacturing paradigm. Expected applications include locomotion for terrestrial crawling robots and flapping wing micro-air vehicles.

The goal of human-robot motion retargeting is to let a robot follow the movements performed by a human subject. This is traditionally achieved by applying the estimated poses from a human pose tracking system to a robot via explicit joint mapping strategies. In this paper, we present a novel approach that combine the human pose estimation and the motion retarget procedure in a unified generative framework. A 3D parametric human-robot model is proposed that has the specific joint and stability configurations as a robot while its shape resembles a human subject. Using a single depth camera to monitor human pose, we use its raw depth map as input and drive the human-robot model to fit the input 3D point cloud. The calculated joint angles of the fitted model can be applied onto the robots for retargeting. The robot's joint angles, instead of fitted individually, are fitted globally so that the transformed surface shape is as consistent as possible to the input point cloud. The robot configurations including its skeleton proportion, joint limitation, and DoF are enforced implicitly in the formulation. No explicit and pre-defined joints mapping strategies are needed. This framework is tested with both simulations and real robots that have different skeleton proportion and DoFs compared with human to show its effectiveness for motion retargeting.

The multi-agent meeting (MAM) problem asks for a meeting location for multiple heterogeneous agents such that the agents can get together within a given time or budget, possibly using different modes of transportation, subject to some constraints and preferences to visit specific locations on their ways to the meeting location. We mathematically model MAM as a graph problem, prove its intractability, and introduce a novel formal method to solve it and its variations using logic-based AI methods. We experimentally evaluate our approach with artificial benchmarks randomly generated over a grid, and show its real world applicability with instances generated over maps of Istanbul and Hong Kong.

With the integration of Human-Robot Collaboration (HRC) in industrial assembly scenarios, robot systems face numerous challenges. In contrast to classic robot systems which follow a pre-programmed and fixed sequence of actions, an interaction scenario with humans in the loop requires mutual adaptation. In this paper a framework based on game theory is presented that allows robots to choose appropriate actions with respect to the action of human coworkers when collaborating in close proximity. The proposed framework models HRC scenarios as iterative games and selects action-strategies for the Human-Robot Team (HRT) by finding the Nash-Equilibria (NEs) of these games. In contrast to most common approaches, our proposed HRC-game treats the decision-making behavior equally for all agents involved. Therefore, the concept of game theory is applied to evaluate the mutual interference of all actions on the HRT to obtain pareto-optimal NEs, i.e. team-optimal action-allocations. The general framework of the proposed HRC-game is realized on an interactive pick-and-place scenario in close proximity. This exemplary HRC-game is tested in a human subject experiment of a KUKA LWR 4+ robot and a human coworker assembling toy-bricks in close proximity. The experimental measurements and statistically significant improvements in the subjective feedback hold as a proof-of-concept of the proposed HRC-game model.

Advances in optical imaging, and probe-based Confocal Laser Endomicroscopy (pCLE) in particular, offer real-time cellular level information for in-vivo tissue characterization. However for large area coverage, the limited field-of-view necessitates the use of a technique known as mosaicking to generate usable information from the incoming image stream. Mosaicking also needs a continuous stream of good quality images, but this is challenging as the probe needs to be maintained within an optimal working range and the contact force controlled to minimize tissue deformation. Robotic manipulation presents a potential solution to these challenges, but the lack of haptic feedback in current surgical robot systems hinders the technology's clinical adoption. This paper proposes a sensorless alternative based on processing the incoming image stream and deriving a quantitative measure representative of the image quality. This measure is then used by a controller, designed using model-free reinforcement learning techniques, to maintain optimal contact autonomously. The developed controller has shown near real-time performance in overcoming typical loss-of-contact and excess-deformation scenarios experienced during endomicroscopy scanning procedures.

When treating highly-infectious diseases such as Ebola, health workers are at high risk of infection during the doffing of Personal Protective Equipment (PPE). This is due to factors such as fatigue, hastiness, and inconsistency in training. The introduction of a semi-autonomous robot doffing assistant has the potential to increase the safety of the doffing procedure by assisting the human during high-risk sub-tasks. However, using a robotic assistant requires transforming a purely human task into a sequence of safe and effective human-robot collaborative actions. Since diseases like Ebola can spread through the mucous membranes of the face our goal in synthesizing these actions is to keep the human's hands away from his or her face as much as possible. As a secondary goal, we also seek to minimize the human's effort. We segment the doffing procedure into a sequence of human and robot actions such that the robot only assists when necessary and the human performs the more intricate parts of the procedure. Our framework then synthesizes assistive motions for the robot that perform parts of the tasks. Our experiments on five doffing tasks suggest that the introduction of a robot assistant improves the safety of the procedure in three out of four of the high-risk doffing tasks while reducing effort in all five tasks.

We propose a task-independent adaptive teleoperation methodology that seeks to improve operator performance and efficiency by concurrently modeling user intent and adapting the set of available actions according to the predicted intent. User input selects a robot motion from a finite set of dynamically feasible and safe motions, represented as a motion primitive library. User intent is modeled as a probabilistic distribution with respect to future actions that represents the likelihood of action selection given recent user input, which can be formulated independent of task, environment, or user. As the intent model becomes increasingly confident, the action set is adapted in order to reduce the error between the intended and actual performance. Experimental evaluation of teleoperating a quadrotor for nonaggressive, single-intent maneuvers such as following a racetrack and conducting a free-hand helix motion shows improved performance, validating that the approach provides efficient adaptation towards achieving the user intent.

A critical issue for Bowden-cable transmission is bending-dependent hysteresis that degrades control performance and limits the application. The relationship between input and output of the Bowden-cable varies with the changing bending angle of the cable. This paper proposes a novel method that can compensate the varying hysteresis of the Bowden-cable using an embedded auxiliary wire, called a sensing wire. The accumulated bend angle of the Bowden-cable can be estimated by measuring the displacement change of the sensing wire and the input tension of the actuation wire. The estimated bending angle of the sheath can be used to compensate the hysteresis of the Bowden-cable with a feedforward control scheme, even though it is subject to the varying shape of the Bowden-cable. The proposed system has a simple and fully embedded design inside the sheath and does not require any external shape sensor or output tension sensor, which keeps the size and complexity of the system low. The results show that the control error of output tension decreased from 4.22 N to 0.69 N when the bend angle of the Bowden-cable changes from 0° to 560°.

Braitenberg vehicles have been used in multiple real world robotic implementations of bio-inspired local navigation. While sensor based control strategies - including existing models of Braitenberg vehicles - typically neglect sensor noise, real robot implementations suffer from different levels of noise, especially in the case of low cost robots and highly stochastic environments. This paper presents a novel drift-diffusion model of Braitenberg vehicle 3a - a bio-inspired source seeking controller for non-holonomic robots - accounting for the sensor noise. The stochastic differential equations obtained provide means to accurately simulate the behaviour of this bio-inspired control mechanism. Although these equations do not have analytic solutions in general, under some simplifying assumptions, we obtain the deterministic equations for the average and dispersion of the vehicles while performing source seeking. Moreover, we found an analytic bound for the distribution of the heading direction of the robots. Simulations illustrate and confirm the theoretical results presented.

One of the major challenges from both a theoretical and practical perspectives, for effectively establishing unattended operation of automated manufacturing systems (AMSs), is to resolve the deadlock. In the existing methods on deadlock problem, most of them are focused on the models with either flexible routes or assembly operations, whereas few method investigates them with both. Furthermore, applying these methods into large-scale systems is usually nontrivial, primarily because of the fact that they generally require the enumeration of all the states or siphons. The work presented in this paper proposes a Petri net-based model which can deal with both features and develops an innovative distributed strategy that provides an online and dynamic mechanism for deadlock resolution. Each step's execution depends on a search-based procedure that seeks to determine whether there exists a feasible event sequence bringing the currently-active process or sub-processes to the nearest global critical place when other processes stagnate. By taking our approach, system can benefit from minimal communication amount among different processes, better toleration to many contingencies like resource failure, and improved system performance like throughput.

This paper presents a novel multi-robot mapping algorithm which allows a large number of simple robots to map the discrete graphical structure underlying an environment of multiple disjoint subregions. Examples of such environments include rooms in a building, buildings in a town, chambers in a cave network, or islands in an archipelago. Each robot is limited to a small communication range, compass, GPS sensor, and a short-range proximity sensor (e.g. bump sensor). Furthermore memory is limited, so no metric map of the environment is stored. Instead the algorithm determines which robots inhabit the same subregion, and which of these groups of robots are able to communicate. This information is captured in a disk graph representation. It is proven that these simple capabilities are sufficient to guarantee that all agents will determine the graphical structure in a finite time. Two environment configurations were tested with a range of quantities of robots. These simulations confirm that processing time is polynomial in the number of robots and indicate that the number of steps to convergence is linear in the number of robots.

This paper proposes a discrete-time geometric attitude observer for fusing monocular vision with GPS velocity measurements. The observer takes the relative transformations obtained from processing monocular images with any visual odometry algorithm and fuses them with GPS velocity measurements. The objectives of this sensor fusion are twofold; first to mitigate the inherent drift of the attitude estimates of the visual odometry, and second, to estimate the orientation directly with respect to the North-East-Down frame. A key contribution of the paper is to present a rigorous stability analysis showing that the attitude estimates of the observer converge exponentially to the true attitude and to provide a lower bound for the convergence rate of the observer. Through experimental studies, we demonstrate that the observer effectively compensates for the inherent drift of the pure monocular vision based attitude estimation and is able to recover the North-East-Down orientation even if it is initialized with a very large attitude error.

Conventional camera calibration techniques rely on discrete reference points extracted from a set of input images. While these approaches have been applied successfully for a long time, omitting all image information apart from reference point positions at the initial stage of the calibration pipeline renders correct treatment of uncertainties difficult and gives rise to complications in timestamping measurements in applications where exposure time cannot be neglected. Drawing inspiration from visual state estimation, we employ a direct formulation of the camera measurement model. To this end, we render a view of the target given all calibration parameters, enabling a maximum likelihood estimator formulated on image intensities as measurements. We demonstrate the advantages of avoiding abstraction from image measurements for determining the line delay of a rolling shutter camera and by estimating camera exposure time from motion blur.

Convolutional Neural Networks (CNNs) trained on large scale RGB databases have become the secret sauce in the majority of recent approaches for object categorization from RGB-D data. Thanks to colorization techniques, these methods exploit the filters learned from 2D images to extract meaningful representations in 2.5D. Still, the perceptual signature of these two kind of images is very different, with the first usually strongly characterized by textures, and the second mostly by silhouettes of objects. Ideally, one would like to have two CNNs, one for RGB and one for depth, each trained on a suitable data collection, able to capture the perceptual properties of each channel for the task at hand. This has not been possible so far, due to the lack of a suitable depth database. This paper addresses this issue, proposing to opt for synthetically generated images rather than collecting by hand a 2.5D large scale database. While being clearly a proxy for real data, synthetic images allow to trade quality for quantity, making it possible to generate a virtually infinite amount of data. We show that the filters learned from such data collection, using the very same architecture typically used on visual data, learns very different filters, resulting in depth features (a) able to better characterize the different facets of depth images, and (b) complementary with respect to those derived from CNNs pre-trained on 2D datasets. Experiments on two publicly available databases show the power of our approach.

Reliable traffic light detection and classification is crucial for automated driving in urban environments. Currently, there are no systems that can reliably perceive traffic lights in real-time, without map-based information, and in sufficient distances needed for smooth urban driving. We propose a complete system consisting of a traffic light detector, tracker, and classifier based on deep learning, stereo vision, and vehicle odometry which perceives traffic lights in real-time. Within the scope of this work, we present three major contributions. The first is an accurately labeled traffic light dataset of 5000 images for training and a video sequence of 8334 frames for evaluation. The dataset is published as the Bosch Small Traffic Lights Dataset and uses our results as baseline. It is currently the largest publicly available labeled traffic light dataset and includes labels down to the size of only 1 pixel in width. The second contribution is a traffic light detector which runs at 10 frames per second on 1280×720 images. When selecting the confidence threshold that yields equal error rate, we are able to detect traffic lights as small as 4 pixels in width. The third contribution is a traffic light tracker which uses stereo vision and vehicle odometry to compute the motion estimate of traffic lights and a neural network to correct the aforementioned motion estimate.

We present a new public dataset with a focus on simulating robotic vision tasks in everyday indoor environments using real imagery. The dataset includes 20,000+ RGB-D images and 50,000+ 2D bounding boxes of object instances densely captured in 9 unique scenes. We train a fast object category detector for instance detection on our data. Using the dataset we show that, although increasingly accurate and fast, the state of the art for object detection is still severely impacted by object scale, occlusion, and viewing direction all of which matter for robotics applications. We next validate the dataset for simulating active vision, and use the dataset to develop and evaluate a deep-network-based system for next best move prediction for object classification using reinforcement learning. Our dataset is available for download at cs.unc.edu/~ammirato/active_vision_dataset_website/.