IROS 2015 论文列表

We present an approach that uses combinatorial optimization to decide which spatial relations between objects are relevant to accurately describe an indoor scene, made up of objects. We extract scene models from object configurations that are acquired during demonstration of actions, characteristic for a certain scene. We model scenes as graphs with Implicit Shape Models (ISMs), a Generalized Hough Transform variant. ISMs are limited to represent scenes as star-shaped topologies of object relations, leading to false positives in recognizing scenes. To describe other relation topologies, we introduced a representation of trees of ISMs in prior work together with a method to learn such ISM trees from demonstrations. Limited to creating topologies, corresponding to spanning trees, that method omits certain relations so that false positives still occur. In this paper, we introduce a method to convert any relation topology, corresponding to a connected graph, into an ISM tree using a heuristic depth-first-search. It allows using complete graphs as scene models. Despite causing no false positives, complete graphs are intractable for scene recognition. To achieve efficiency, we contribute a method that searches for an optimal relation topology by traversing the space of connected scene graphs, for a given set of objects, using an optimization similar to hill climbing. Optimality is defined as minimizing computational costs during scene recognition, while producing a minimum of false positives. Experiments with up to 15 objects show that both are achievable by the presented method. Costs, growing exponentially with the number of objects, are transferred from online recognition to offline optimization.

This paper presents a novel method for automated manipulation of individual paper fiber bonds using a microrobotic platform, a computer vision algorithm and a robotic software framework. This is a challenging task due to the three-dimensional, heterogeneous and complex morphology of the fiber bonds. The goal is to automatically grasp the fiber bond, and break it by pulling apart the fibers it consists of. We present the components of the microrobotic platform, and the different rules utilized in detecting suitable grasp points from a 3D reconstruction of the bond generated from an image pair. We demonstrate the functionality of the approach with bond breaking experiments of seven fiber bonds. The time required for grasping and breaking of a bond is 10 - 15 seconds making the approach much faster than the current state-of-the-art testing, which is based on manual manipulation. The success rate of the tests is as high as 80 %.

Fabrication of artificial blood vessels in micro scale significantly benefits the regeneration of functional human vascular networks. In this paper, we develop an efficient multi-microrobotic system with an innovative motorized sample holder (MSH) and two manipulators. Air is injected into the solution through a glass pipette fixed on one manipulator to create bubbles. These bubbles conduct a regular rising movement, which is utilized to assemble the 2D ring-shaped microgels fabricated in a simple micro fluidic device. With this novel bubble-based method and the robotic system, we achieve the automation of the assembly. A 1.2 mm long vascular-like microtube with an outer diameter of 200 µm is fabricated. The whole process of the bubble-based assembly is visually observed and analyzed with side view. Key parameters are characterized to improve the assembly. Results show that the automated bubble-based assembly success rate is 100% and average time cost of assembling every microgel is as low as 3.25s.

We present a novel, soft, tactile skin composed of a fabric-based, stretchable sensor technology based on the piezoresistive effect. Softness is achieved by a combination of a soft silicone padding covered by a skin of more durable, tearproof silicone with an imprinted surface pattern mimicking human glabrous skin, found e.g. in fingertips. Its very thin layer structure (starting from 2.5 mm) facilitates integration on existing robot surfaces, particularly on small and highly curved links. For example, we augmented our Shadow Dexterous Hand with 12 palm sensors, and 2 resp. 3 sensors in the middle resp. proximal phalanges of each finger. To demonstrate the usefulness and efficiency of the proposed sensor skin, we performed a challenging classification task distinguishing squeezed objects based on their varying stiffness.

This paper reports on an algorithm to support autonomous vehicles in reasoning about occluded regions of their environment to make safe, reliable decisions. In autonomous driving scenarios, other traffic participants are often occluded from sensor measurements by buildings or large vehicles like buses or trucks, which makes tracking dynamic objects challenging.We present a method to augment standard dynamic object trackers with means to 1) estimate the occluded state of other traffic agents and 2) robustly associate the occluded estimates with new observations after the tracked object reenters the visible region of the sensor horizon. We perform occluded state estimation using a dynamics model that accounts for the driving behavior of traffic agents and a hybrid Gaussian mixture model (hGMM) to capture multiple hypotheses over discrete behavior, such as driving along different lanes or turning left or right at an intersection. Upon new observations, we associate them to existing estimates in terms of the Kullback-Leibler divergence (KLD). We evaluate the proposed method in simulation and using a real-world traffic-tracking dataset from an autonomous vehicle platform. Results show that our method can handle significantly prolonged occlusions when compared to a standard dynamic object tracking system.

In this paper we present a framework for the application of augmented reality to a mobile robot, using non-central camera systems. Considering a virtual object in the world with known local 3D coordinates, the goal is to project this object into the image of a non-central catadioptric imaging device. We propose a solution to this problem which allows us to project textured objects to the image in real-time (up to 20 fps): projection of 3D segments to the image; occlusions; and illumination. In addition, since we are considering that the imaging device is on a mobile robot, one needs to take into account the real-time localization of the robot. To the best of our knowledge this is the first time that this problem is addressed (all state-of-the-art methods are derived for central camera systems). To evaluate the proposed framework we test the solution using a mobile robot and a non-central catadioptric camera (using a spherical mirror).

This paper addresses multimodal “scene understanding” for a robot using audio-visual and text information. Scene understanding is defined by extracting six-W information such as What, When, Where, Who, Why, and hoW on the surrounding environment. Although scene understanding for a robot has been studied in the fields of robot vision and audition, only the first four Ws except for why and how information were considered. We, thus, focus on extracting how information, in particular, on cooking scenes. In cooking scenes, we define how information as a cooking procedure, and it is useful that a robot gives appropriate advice for cooking. To realize such cooking support, we propose a multi-modal cooking procedure recognition framework consisting of Convolutional Neural Network (CNN), and Hierarchical Hidden Markov Model (HHMM). CNN is knows as one of the most advanced classifiers, and it is applied to recognize a cooking events from audio and visual information. HHMM models a cooking procedure represented by a sequence of cooking events, which is defined as a relationship between cooking events using text data obtained from web, and the cooking events classified with CNN. Therefore, our proposed framework integrates these three types of modalities. We constructed an interactive cooking support system based on the proposed framework, which advice a next step in the current cooking procedure through human-robot communication. Preliminary results with simulated and real recorded multi-modal scenes showed the robustness of the proposed framework in a noisy and/or occluded situation.

This paper presents an audio-visual beat-tracking method for an entertainment robot that can dance in synchronization with music and human dancers. Conventional music robots have focused on either music audio signals or dancing movements of humans for detecting and predicting beat times in real time. Since a robot needs to record music audio signals by using its own microphones, however, the signals are severely contaminated with loud environmental noise and reverberant sounds. Moreover, it is difficult to visually detect beat times from real complicated dancing movements that exhibit weaker repetitive characteristics than music audio signals do. To solve these problems, we propose a state-space model that integrates both audio and visual information in a probabilistic manner. At each frame, the method extracts acoustic features (audio tempos and onset likelihoods) from music audio signals and extracts skeleton features from movements of a human dancer. The current tempo and the next beat time are then estimated from those observed features by using a particle filter. Experimental results showed that the proposed multi-modal method using a depth sensor (Kinect) for extracting skeleton features outperformed conventional mono-modal methods by 0.20 (F measure) in terms of beat-tracking accuracy in a noisy and reverberant environment.

This paper presents a method for representing acoustic information with point clouds by tying it to geometrical features. The motivation is to create a representation of this information that is well suited for mobile robotic applications. In particular, the proposed approach is designed to take advantage of the use of multiple coordinate frames. As an illustrative example, we present a way to create an audio augmented point cloud by adding estimated audio power to the point cloud created by a RGB-D camera. A few applications of this method are presented.

Tracking of items on shelves is an important but time-consuming task in inventory control. In particular, books in public libraries are frequently borrowed and returned, even misplaced, and proves a challenge to be tracked on a daily basis. To track the books efficiently, we propose the Autonomous Robotic Shelf Scanning (AuRoSS) system. This paper elaborates on the key enabling robotic technology for the fully autonomous system, namely, a navigation system with surface tracking capability. A Radio Frequency Identification (RFID) reader is carried by the navigation system to identify the RFID tags embedded in each book. Based on the tag information, a tracking report that highlights missing and misplaced books is generated for the end users. To ensure successful identification, the surface tracking requires high accuracy, for which we propose a filtered Hough transform and a macro-mini manipulator structure. Tests of the AuRoSS system in a library show high accuracy in the scanning performance.

In recent years there has been increased interest in studies that explore integrative learning of language and other modalities by using neural network models. However, for practical application to human-robot interaction, the acquired semantic structure between language and meaning has to be available immediately and repeatably whenever necessary, just as in everyday communication. As a solution to this problem, this study proposes a method in which a recurrent neural network self-organizes cyclic attractors that reflect semantic structure and represent interaction flows in its internal dynamics. To evaluate this method we design a simple task in which a human verbally directs a robot, which responds appropriately. Training the network with training data that represent the interaction series, the cyclic attractors that reflect the semantic structure is self-organized. The network first receives a verbal direction, and its internal state moves according to the first half of the cyclic attractors with branch structures corresponding to semantics. After that, the internal state reaches a potential to generate appropriate behavior. Finally, the internal state moves to the second half and converges on the initial point of the cycle while generating the appropriate behavior. By self-organizing such an internal structure in its forward dynamics, the model achieves immediate and repeatable response to linguistic directions. Furthermore, the network self-organizes a fixed-point attractor, and so able to wait for directions. It can thus repeat the interaction flexibly without explicit turn-taking signs.

Mobile robots are nowadays used in a wide variety of missions especially for exploring environments and allowing scientists who study these environments to gather data about them. However the monolithic design of control laws is often a hindrance to adapting control solutions from a robotic application to another. We thus propose a new control description paradigm strongly focused on modularity as well as integrating constraints coming both from the system capabilities and from the concerns of control engineering (such as stability). In this paper, we present the main ideas behind our approach and illustrate it through Hardware-In-the-Loop simulation.

Apprenticeship learning based on inconsistent demonstrations is presented in this paper. We address a problem where given demonstrations are not directly applicable to reward function estimation due to the non-stationarity of an environment or the difference between the dynamics of a robot and a demonstrator. A basic idea of the proposed method is to use a subset of the trajectories sampled from the baseline policy as training data for inverse reinforcement learning. All consistent sample trajectories and inconsistent demonstrations are abstracted by an affine transformation invariant feature. Using the feature, the importance of each sample trajectory is estimated. Rating the sample trajectories based on importance, the training data for inverse reinforcement learning are identified. The validity of our approach is verified through simulation in two scenarios: inconsistency caused by variation of an environment and performance of a robot.

In this paper a study is made of the problem of classifying scenarios, in terms of semantic categories, based on data gathered from sensors mounted on-board mobile robots operating indoors. Once the data are transformed to feature space, supervised classification is performed by a probabilistic approach called Dynamic Bayesian Mixture Models (DBMM). This approach combines class-conditional probabilities from supervised learning models and incorporates past inferences. In this work, several experiments on multi-class semantic place classification are reported based on publicly available datasets. Such experiments were conducted in a such way that generalization aspects are emphasized, which is particularly important in real-world applications. Benchmark results show the effectiveness and competitive performance of the DBMM method, in terms of classification rates, using features extracted from 2D range data and from a RGB-D (Kinect) sensor.

We present a deep learning approach to estimation of the bead parameters in welding tasks. Our model is based on a four-hidden-layer neural network architecture. More specifically, the first three hidden layers of this architecture utilize Sigmoid function to produce their respective intermediate outputs. On the other hand, the last hidden layer uses a linear transformation to generate the final output of this architecture. This transforms our deep network architecture from a classifier to a non-linear regression model. We compare the performance of our deep network with a selected number of results in the literature to show a considerable improvement in reducing the errors in estimation of these values. Furthermore, we show its scalability on estimating the weld bead parameters with same level of accuracy on combination of datasets that pertain to different welding techniques. This is a nontrivial result that is counter-intuitive to the general belief in this field of research.

We study the problem in which a mobile sensing robot is tasked to travel among and gather intelligence at a set of spatially distributed points-of-interest (POIs). The quality of the information collected at a POI is characterized by some sensory (reward) function of time. With limited fuel, the robot must balance between spending time traveling to more POIs and performing time-consuming sensing activities at POIs to maximize the overall reward. In a dual formulation, the robot is required to acquire a minimum amount of reward with the least amount of time. We propose an anytime planning algorithm for solving these two NP-hard problems to arbitrary precision for arbitrary reward functions. The algorithm is effective on large instances with tens to hundreds of POIs, as demonstrated with an extensive set of computational experiments. Besides mobile sensor scheduling, our algorithm also applies to automation scenarios such as intelligent and optimal itinerary planning.

Inspired by geckos, we built a new configuration for climbing robot, the GPL model with one passive waist and four active axil legs. In this paper, further research was made to analysis its motion. We found the anomalies line between supporting feet and its effect on robot motion. A step further, the principia of configuration design and gait planning were proposed based on dynamic analysis. The prototype was developed and the climbing forces experiment was carried out. We testified the rationality of GPL model and waist trajectory planning. Also, energy analysis was proven by the driving forces measured on the supporting feet. In our experiment, the deceleration force during the robot climbing was only provided by the gravity, indicating a potential space for speed improvement without significant energy waste.

This paper presents a novel control architecture for controlling a moving remote center of motion in addition to the end-effector motion during robotic surgery. In minimally invasive surgery, it is common to require that the point at which the robot enters the body, called the incision point or the trocar, does not allow for any lateral motion. It is generally considered that no motion should be applied to this point in order to avoid inflicting damage to the patient's skin. However, in surgery, the patient's body may be moving, for example due to breathing or the beating of the heart. In order to compensate for this motion—or if we for some other reason want to leverage the possible motion of the incision point to improve performance in any other way—we derive a new framework which allows us to actively control the motion both at the incision point and the end effector. The novelty of the approach lies in the possibility of controlling both the incision point and the end effector to follow a trajectory, and that we find a Jacobian matrix that satisfies the velocity constraints in both the end-effector and the incision point frames. This allows us to formulate a framework that is not only suited for control, but also for analyzing the condition number of the Jacobian and avoid any singular configurations that may arise either as a result of the constrained motion or the manipulator geometry. The approach is verified experimentally on a redundant industrial manipulator.

Intuitive programming of industrial robots is especially important for small and medium-sized enterprises. We evaluated four different input modalities (touch, gesture, speech, 3D tracking device) regarding their preference, usability, and intuitiveness for robot programming.