IROS 2017 Papers

We propose a probabilistic model that generates nod motions based on utterance categories estimated from the speech input. The model comprises two main blocks. In the first block, dialogue act-related categories are estimated from the input speech. Considering the correlations between dialogue acts and head motions, the utterances are classified into three categories having distinct nod distributions. Linguistic information extracted from the input speech is fed to a cluster of classifiers which are combined to estimate the utterance categories. In the second block, nod motion parameters are generated based on the categories estimated by the classifiers. The nod motion parameters are represented as probability distribution functions (PDFs) inferred from human motion data. By using speech energy features, the parameters are sampled from the PDFs belonging to the estimated categories. Subjective experiment results indicate that the motions generated by our proposed approach are considered more natural than those of a previous model using fixed nod shapes and hand-labeled utterance categories.

Creating accurate models of stochastically self-assembling systems is a key step in developing control strategies, centralized or distributed, for the self-assembly process. This paper comparatively studies several aspects of developing probabilistic models for programmable self-assembling systems of stochastically interacting modules. In particular, we systematically investigate Markov models as well as hidden Markov models to predict the self-assembly process dynamics. We consider a case study leveraging our fluidic self-assembly robotic system. The ground truth is obtained through a high-fidelity simulation, calibrated using real experimental data. We first consider Markov models and employ the formalism of chemical reaction networks. In order to compute the model parameters, i.e. the reaction rates in the network, three different methods are studied. We then investigate the validity of the underlying well-mixed assumption, and thus the Markov property, for our system through estimation of the diffusion coefficient, through two different approaches. The system is shown to be borderline well-mixed, motivating extension of the initial Markov models to more complex models in order to achieve improved model accuracy. We formulate an automatic method for creating a hidden Markov model starting from a Markov model, based on a previously existing systematic method. Sample trajectories of the models are realized using the Gillespie's method. The resulting hidden Markov model is shown to achieve an improved accuracy over the standard Markov model.

Allowing a cobot to predict what the human operator is about to do can definitely enhance the effectiveness of human-robot collaboration. This paper addresses the problem of inferring the most likely reaching target of the human hand. The method allows the robot to promptly recognise the intention of the human to reach a certain position within the scene and can be thus used by the controller of the robot to take the optimal decision on what to do. A novel method based on Bayesian statistics has been developed in this work and its applicability in a realistic context has been verified within an industrial use case, consisting of a commercial collaborative robot and a human operator performing a collaborative assembly task.

Probabilistic 3D map has been applied to object segmentation with multiple camera viewpoints, however, conventional methods lack of real-time efficiency and functionality of multilabel object mapping. In this paper, we propose a method to generate three-dimensional map with multilabel occupancy in real-time. Extending our previous work [1] in which only target label occupancy is mapped, we achieve multilabel object segmentation in a single looking around action. We evaluate our method by testing segmentation accuracy with 39 different objects, and applying it to a manipulation task of multiple objects in the experiments. Our mapping-based method outperforms the conventional projection-based method by 40-96% relative (12.6 mean IU3d), and robot successfuly recognizes (86.9%) and manipulates multiple objects (60.7%) in an environment with heavy occlusions.

Urban transportation is being transformed by mobility-on-demand (MoD) systems. One of the goals of MoD systems is to provide personalized transportation services to passengers. This process is facilitated by a centralized operator that coordinates the assignment of vehicles to individual passengers, based on location data. However, current approaches assume that accurate positioning information for passengers and vehicles is readily available. This assumption raises privacy concerns. In this work, we address this issue by proposing a method that protects passengers' drop-off locations (i.e., their travel destinations). Formally, we solve a batch assignment problem that routes vehicles at obfuscated origin locations to passenger locations (since origin locations correspond to previous drop-off locations), such that the mean waiting time is minimized. Our main contributions are two-fold. First, we formalize the notion of privacy for continuous vehicle-to-passenger assignment in MoD systems, and integrate a privacy mechanism that provides formal guarantees. Second, we present a polynomial-time iterative version of the Hungarian algorithm to allocate a redundant number of vehicles to a single passenger. This algorithm builds on the insight that even during peak rush hour there are unoccupied (redundant) traveling vehicles. This strategy allows us to reduce the performance deterioration induced by the privacy mechanism. In particular, it enables the exploration of the trade-off between privacy levels, waiting time, and deployed fleet size. We evaluate our methods on a real, large-scale data set consisting of over 11 million taxi rides (specifying vehicle availability and passenger requests), recorded over a month's duration, in the area of Manhattan, New York. Based on current traffic statistics, our evaluations indicate that privacy can be achieved without incurring a significant loss of performance, and that this loss can be further controlled by varying operator or us... Show More

This paper presents the design and preliminary investigation of a fully 3D printed soft robotic hand exoskeleton, Print-it-Yourself (PIY) Glove for stroke patients. The PIY Glove is fabricated with Fused Deposition Modeling (FDM) using consumer based 3D printing technology to lower fabrication costs and allow patients to 3D print a rehabilitative and assistive device at home. PIY Glove uses a novel, fold based design of 3D printed soft actuators to achieve bending motion in the fingers. Fabrication guidelines of the PIY Glove are laid out and characterization of the glove in terms of its range of motion and grip force are also presented. An EMG control system that achieves control of the PIY Glove is also described. This work paves the way for the implementation of printable pneumatics in real-world applications, particularly robot-assisted hand therapy.

This paper presents a near-future perception system named “Previewed Reality”. The system consists of an informationally structured environment (ISE), an immersive VR display, a stereo camera, an optical tracking system, and a dynamic simulator. In an ISE, a number of sensors are embedded, and information such as the position of furniture, objects, humans, and robots, is sensed and stored in a database. The position and orientation of the immersive VR display are also tracked by an optical tracking system. Therefore, we can forecast the next possible events using a dynamic simulator and synthesize virtual images of what users will see in the near future from their own viewpoint. The synthesized images, overlaid on a real scene by using augmented reality technology, are presented to the user. The proposed system can allow a human and a robot to coexist more safely by showing possible hazardous situations to the human intuitively in advance.

Flexible manipulator enables curvilinear accessibility through small incisions or natural orifices for minimally invasive surgery and diagnosis, which makes it a good choice for minimally invasive surgery. In order to control the robot precisely and safely, the real-time position and shape information of the robot need to be measured well. In this paper, we propose a magnetic tracking based tip pose and shape detection method for wire driven flexible robots. A permanent magnet is mounted at the distal end of the robot. Its magnetic field can be sensed with a sensor array. Therefore, position and orientation of the tip can be estimated utilizing the tracking method. A shape sensing algorithm is then carried out to estimate the real-time shape based on the tip pose. With the tip pose and shape display in the reconstructed visual environment, navigation can be achieved. This method provides the advantages that no sensors are needed to mount on the robot and has no line-of-sight problem. Experimental results verified the feasibility of the proposed method. A navigation error of 1.9mm is achieved.

We propose a novel two-tier gait recognizer using a minimal number of mechanical sensors built into a lower limb exoskeleton. The aim of this recognizer is to offer one-step selection of one of the actions ascending, descending, and level walking during five gaits (stair ascent/descent, slope ascent/descent, level walking). The proposed recognizer is executed at the moment of foot contact as estimated by an inertial measurement unit (IMU) on the pelvis without using direct foot sensors. The proposed recognizer selects the gait by using the relations between the angles formed by the hip and knee joints during the last step. As this study constitutes preliminary work for lower limb exoskeletons, we used a hip exoskeleton integrated with two wireless IMUs on the shank to set up the lower limb exoskeleton sensor configuration. Experiments were used to evaluate IMU-based foot contact estimation with respect to a foot sensor based on a force-sensitive resistor (FSR) for stairs, slopes, and level ground. In addition, we evaluated the performance of the proposed two-tier gait recognizer by using two healthy male subjects in various gait environments.

This paper presents preliminary results on OCT (Optical Coherence Tomography) based control of a concentric tube robot (CTR). The potential use of OCT system is presented then its ability to perform an optical biopsy and an eye-to-hand configuration for robot control is highlighted. A prototype of CTR with three curved tubes is shortly described then a model-less OCT-based controller is detailed. The extraction of visual features from OCT images is performed by normalized cross-correlation. The proposed control scheme was experimentally validated. The positioning accuracy of CTR control was evaluated in benchtop trials. The error in tip placement can reach 10 μm along X and 74 μm along Y according to controller parameters. The 2D positioning (XY plan) is also tested then a final error about 55 μm is achieved.

Ride-sharing, or carpooling, systems with autonomous vehicles will provide efficient and reliable urban mobility on demand. In this work we present a method for dynamic vehicle routing that leverages historical data to improve the performance of a network of self-driving taxis. In particular, we describe a constrained optimization method capable of assigning requests to autonomous vehicles in an informed way, to minimize the expected cost of serving both current and future travel requests. We allow several passengers with independent trips to share a vehicle and allow vehicles to pick additional passengers as they progress through their route. Based on historical data, we compute a probability distribution over future demand. Then, samples from the learned probability distribution are incorporated into a decoupled vehicle routing and passenger assignment method to take into account the predicted future demand. This method consists of three steps, namely pruning of feasible trips, assignment of trips to vehicles and rebalancing of idle vehicles. We show the benefits and trade-offs of this predictive approach in an experimental evaluation with over three million rides extracted from a dataset of taxi trips in New York City. Our method produces routes and assignments that, in expectation, reduce the travel and waiting times for passengers, with respect to a purely reactive approach. Besides the mobility on demand application, the method we present is general and could also be applied to other multi-task multi-vehicle assignment and routing problems.

This paper proposes the use of a learning approach to predict air-to-ground (A2G) communication strength in support of the communication relay mission using UAVs in an urban environment. To plan an efficient relay trajectory, A2G communication link quality needs to be predicted between the UAV and ground nodes. However, due to frequent occlusions by buildings in the urban environment, modelling and predicting communication strength is a difficult task. Thus, a need for learning techniques such as Gaussian Process (GP) arises to learn about inaccuracies in a pre-defined communication model and the effect of line-of-sight obstruction. Two ways of combining GP with a relay trajectory planner are presented: i) scanning the area of interest with the UAV to collect communication strength data first and then using learned data in the trajectory planner and ii) collecting data and running the trajectory planner simultaneously. The performance of both approaches is compared with Monte Carlo simulations. It is shown that the first implementation results in slightly better predictions, however the second one benefits from being able to start the relay mission immediately.

In robotics, vision sensors are used to estimate the poses of objects in the environment. However, it is a fundamental problem that the estimated poses are not always accurate enough for a given robotic task. Proper sensor placement can mitigate this problem. We present a method which can predict the pose uncertainties in the Iterative Closest Point (ICP) algorithm, which is often used as the last critical pose refinement step in a pose estimation system. With our method we thus provide a crucial tool needed for the optimization of a robust pose estimation system. Our method relies on the generation of synthetic depth images in a Monte Carlo simulation. In this paper we demonstrate our method for depth sensors which rely on Kinect v1 like technology. We evaluate our method using real depth sensor recordings from the publicly available BigBird dataset. The evaluation shows that the uncertainty predictions of our method are in better correspondence with real world experimental results than the state of the art analytical method.

We introduce a novel pose graph-based localization technique for autonomous driving that incorporates both sparse point features as well as lane markings on the road. Unlike many commonly used filter methods, our graph-based localization takes a much larger history of the trajectory into account. In addition, we utilize a multiple-hypothesis approach for data association for increased robustness against the presence of outliers. By incorporating both sparse point features and lane markings, we are able to take advantage of both kinds of features widely available in urban environments as well as highways. Furthermore, by using high level features, we avoid having to store and match against large amounts of data such as dense 3D point clouds. Finally, we evaluate our approach on real driving sequences on city roads.

Robotic pouring is an important step in improving the safety, productivity and repeatability in the biotechnology industry and generally increasing the effectiveness of robotics in human based environments. In this work we present a method to autonomously dispense a precise amount of fluid using only visual feedback without using precision pouring instruments such as pipettes, syringes or pourers. We model circular and rectangular pouring container geometries. We prove that for square containers we can control the flow by only observing the fluid height in the receiving beaker. We show a systematic approach using a hybrid control scheme that is robust to the initial amount of fluid in the pouring container and inconsistent flow. Specifically we present (a) a model for pouring (b) a model based algorithm to drive a robot arm (c) visual feedback for regulating the pouring rate. We demonstrate this using the Rethink Robotics Sawyer manipulator and mvBluefox MLC202bc camera.

Vacuum-powered Soft Pneumatic Actuator (V-SPA) Modules have been described to afford advantages for rapid development of reconfigurable, multi-DoF soft pneumatic robots powered by vacuum by reducing their logistical complexity, however they also present new challenges in the control of resulting systems. This framework features modules joined together over a simple embedded pneumatic and serial communication network and requires a unique approach to both low-level control implementation and high-level control strategy. We describe the structure and activation characteristics of a V-SPA Module and present practical methods for its control. These methods utilize software generated PWM activation through a unique serial protocol designed for LED networks and a heuristic mapping strategy for simplifying the spherical control of 3-DoF actuator modules.

In human-robot collaboration the robot's behavior impacts the worker's safety, comfort and acceptance of the robotic system. In this paper we address the problem of how to improve the worker's posture during human-robot collaboration. Using postural assessment techniques, and a personalized human kinematic model, we optimize the model body posture to fulfill a task while avoiding uncomfortable or unsafe postures. We then derive a robotic behavior that leads the worker towards that improved posture. We validate our approach in an experiment involving a joint task with 39 human subjects and a Baxter torso-humanoid robot.

Present visual servoing approaches are instance specific i.e. they control camera motion between two views of the same object. However, in practical scenarios where a robot is required to handle various instances of a category, classical visual servoing techniques are less suitable. We formulate across instance visual servoing as a pose induction and pose alignment problem. Initially, the desired pose given for any known instance is transferred to the novel instance through pose induction. Then the pose alignment problem is solved by estimating the current pose using the part aware keypoints reconstruction followed by a pose based visual servoing (PBVS) iteration. To tackle large variation in appearance across object instances in a category, we employ visual features that uniquely correspond to locations of object's parts in images. These part-aware keypoints are learned from annotated images using a convolutional neural network (CNN). Advantages of using such part-aware semantics are two-fold. Firstly, it conceals the illumination and textural variations from the visual servoing algorithm. Secondly, semantic keypoints enables us to match descriptors across instances accurately. We validate the efficacy of our approach through experiments in simulation as well as on a quadcopter. Our approach results in acceptable desired camera pose and smooth velocity profile. We also show results for large camera transformations with no overlap between current and desired pose for 3D objects, which is desirable in servoing context.

Numerous therapeutic transendoscopic procedures exist to treat lesions in the GI tract. However, these procedures are limited by their difficulty and the amount of training required to successfully perform them. The surgeon is tasked with simultaneously steering the distal tip of the endoscope, applying tension to tissue to retract it, and manipulating electro-cautery tools with limited dexterity. We propose a device designed to assist with anchoring and tissue retraction during endoscopic surgical procedures. The designed solution decouples the tissue-grasping function from the movement of the endoscope tip, leaving the surgeon free to use the endoscope tip solely for positioning of electro-cautery or biopsy tools deployed through the endoscope working channel. The anchoring and retraction device uses pop-up book MEMS techniques, allowing for a “flat” structure to expand into a 3-D structure. The proposed device has three main integrated components: a rigid expandable geometric structure, inflatable pneumatic actuators, and a vacuum gripper. These inflatable actuators include internal rigid discs, allowing for resistance to buckling while maintaining the benefits of the established lightweight, low profile actuator design scheme. Proof-of-concept ex vivo testing demonstrates that the integrated device can be used to retract tissue to a height of 13.5 mm, providing access for endoscopy tools to contact a sample of porcine stomach tissue.