IROS 2015 Papers

We present an optimization method to solve coupled redundant inverse kinematics problems and generate trajectories for humanoid robot full-body manipulation. The basic idea of our algorithm is to divide a manipulation task into a series of key postures, generate multiple diverse initial guesses for each key posture, and use optimization to find inverse kinematics solutions based on these initial guesses. We then find an optimal series of key postures and form a continuous trajectory. Our approach is implemented in a Gazebo simulation using the Atlas humanoid robot from Boston Dynamics.

This paper presents a framework for information sharing and fusion in cooperative tasks involving humans and robots. In this context, all information regarding the state of interest is recursively fused and maintained by each agent in a form of belief. For a robot agent, its belief is commonly and practically represented as a probability density function (pdf), formed by traditional sensor fusion and state estimation algorithms. In cooperative tasks with non-expert humans, a robot needs to effectively communicate its belief so that the gathered information can be easily processed and interpreted by the humans. The goal of this research is to provide two-way information exchange and fusion between robots and humans, the former operating on pdfs, while the latter on English sentences. This is achieved by considering two goodness measures: semantic correctness and information preservation. Based on the goodness measures studied, results show that the proposed framework is able to generate optimal statements describing the given belief pdfs and successfully recover the initial inputs used to generate them. Additionally, in order to describe complex belief pdfs, a Mixture of Statements (MoS) model is proposed such that the optimal expression can be generated through a composition of more than one statements. With a nonparametric Dirichlet Process MoS generation, it is found that the robot can determine correctly the number of statements as well as the corresponding reference parameters needed to describe all hypotheses underlying its belief.

Previous studies on human-pose estimation rely on the design of factors to represent underlying probability distributions. However, designing factors is laborious and yet, the designed factors may not represent the underlying probability distributions. In this paper, we propose to use a neural network to automatically design factors in one of the existing models called the action-mixture model (AMM). Factors that are designed automatically by neural networks can be adapted to different situations. The semantic meaning of random variables in AMM can be transferred to a neural network, rendering the semantic meaning of hidden neurons transparent to users. The design process consists of two stages: structure identification and parameter learning. In the structure identification, we propose a bottom-up approach to build a neural network while preserving the structure of AMM. In the parameter learning, we propose a part-based approach to learn synaptic weights by decomposing a neural network into parts. Synaptic weights that have been learnt in one part can be used as initial weights for learning synaptic weights in another part. Based on the concept of distributed representation, the proposed two-stage, neural-network-based design process is used to design a scalable neural network to realize an AMM. Experimental results showed that the scalable neural network outperformed AMM and some existing works.

Human-autonomy sensor fusion is an emerging technology with a wide range of applications, including object detection/recognition, surveillance, collaborative control, and prosthetics. For object detection, humans and computer-vision-based systems employ different strategies to locate targets, likely providing complementary information. However, little effort has been made in combining the outputs of multiple autonomous detectors and multiple human-generated responses. This paper presents a method for integrating several sources of human- and autonomy-generated information for rapid object detection tasks. Human electroencephalography (EEG) and button-press responses from rapid serial visual presentation (RSVP) experiments are fused with outputs from trained object detection algorithms. Three fusion methods—Bayesian, Dempster-Shafer, and Dynamic Dempster-Shafer—are implemented for comparison. Results demonstrate that fusion of these human classifiers with computer-vision-based detectors improves object detection accuracy over purely computer-vision-based detection (5% relative increase in mean average precision) and the best individual computer vision algorithm (28% relative increase in mean average precision). Computer vision fused with button press response and/or the XDAWN + Bayesian Linear Discriminant Analysis neural classifier provides considerable improvement, while computer vision fused with other neural classifiers provides little or no improvement. Of the three fusion methods, Dynamic Dempster-Shafer Theory (DDST) Fusion exhibits the greatest performance in this application.

Optical motion capture systems, which are used in broad fields of research, are costly; they need large installation space and calibrations. Applying this technology in typical homes and care centers is unrealistic. Low cost motion capture systems such as Microsoft Kinect are based on video, thus privacy issues might arise from their usage. Therefore we propose to use low cost contact force measurement systems to develop rehabilitation and healthcare monitoring tools that can be used widely. Here, we propose a novel algorithm for motion recognition using the feature vector from force data solely obtained during a daily exercise program. We recognized 7 types of movement (Radio Exercises) of 5 candidates (mean age 24, male). The results show that the average recognition rate for each motion has good score (mean: 75%), but leave room for improvement. The results also confirm that there is a dynamic signature in each movement allowing inter-personal recognition. Thus it is possible using inexpensive contact force measurement for motion analysis and motion recognition for applications such as rehabilitation or healthcare.

In this paper, we present an algorithm to infer the intent of a human operator's arm movements based on the observations from a Microsoft Kinect sensor. Intentions are modeled as goal locations in 3-dimensional (3D) space where the human is intending to reach. Human intention inference is a critical step towards realizing safe human-robot collaboration. This work models the human arm's nonlinear motion dynamics using an unknown nonlinear function with intentions modeled as parameters. The unknown model is learned using a neural network (NN). Based on the learned model, an approximate expectation-maximization (E-M) algorithm is developed to infer human intentions. Furthermore, an identifier-based online model learning algorithm is developed to adapt to variations in the arm motion dynamics, trajectory of motion, goal locations, and initial conditions of different human subjects. We show the results of our algorithm using two sets of experiments conducted on data obtained from different users.

The HippoCampus platform is a low-cost micro autonomous underwater vehicle for swarm robotics research. This paper presents the hardware and software design, the communication link, instrumentation, and control system. The quadrotor design enables the vehicle to perform agile maneuvers in a very confined test tank. Vehicle navigation is based on the on-board sensor suite. Autonomous path following is presented and experimental results are evaluated.

Recent advances in soft sensors using microfluidic liquid conductors enabled sensing of large deformation of soft structures. However, the use of liquids as conductive media carries a risk of leakage in many cases. Furthermore, it could be harmful when exposed to the human body in certain applications. To address these issues, a different sensing mechanism was proposed: highly stretchable optical sensors that could detect multiple modes of deformation. The method of operation involves a simple waveguide and its housing which are both made of silicone elastomer. The soft waveguide is coated with a thin gold reflective layer to encapsulate light propagating internally, with an light-emitting diode (LED) and a photodiode embedded at each end. When the sensor is stretched, compressed, or bent, micro-cracks within the reflective layer form and allow part of the light to escape, resulting in optical power losses in the light transmission. In this paper, we describe the design and fabrication of the proposed soft sensors. A prototype was created and characterized for pressure, strain, and curvature up to 350 kPa, 90%, and 0.12 mm−1, respectively, showing promising results of reasonable repeatability and linearity in certain ranges.

This paper presents a dual-user teleoperation scheme to perform a collaborative task using n-DOF nonlinear manipulators as masters and slave. Impedance controllers for the manipulators are implemented in order to achieve a desired dynamic behavior depending on the user's necessities. Furthermore, a sliding mode controller is introduced to cope with the time delay in the communication channels and the uncertainty in the slave. Since the slave teleoperator is in contact with a rigid environment, the slave controller requires a free of chattering control strategy, which makes first order sliding mode teleoperation control unsuitable. Then a higher order sliding mode based impedance controller is proposed to guarantee robust impedance tracking under constant, but unknown time delay. Therefore, a position scaling factor is incorporated to deal with the different workspaces among masters and slave. The validity of the proposed control scheme is demonstrated via experimentation on a 3-DOF dual-user teleoperation system. While the haptic devices Phantom Premium 1.0A and a Phantom Omni are used as masters, a virtual industrial manipulator Catalyst-5 is used as slave. The dual-user system is tested not only in presence of constant unknown time delay in each of the communication channels but also in free and constrained motion regime.

This paper presents a method that enables highspeed, steady flight in confined spaces for tethered quadrocopters. Thanks to the centripetal force exerted by the tether, high-speed trajectories along circles at different velocities, accelerations, and orientations in space can be flown. Various circular maneuvers are experimentally demonstrated, and tangential velocities of up to 15 m/s and centripetal accelerations of more than 13 g can be achieved in steady flight. The recorded data allows to characterize the flight behavior of quadrocopters at high airspeeds: As an example, an estimate of the actual thrust produced by the motors and of the aerodynamic drag acting on the vehicle is presented. An accompanying video shows tethered quadrocopters performing high-speed maneuvers.

We developed an optical high-speed image rotation controller and realized a high-speed roll camera that is able to cancel the rotational motion blur of a rotating target. This system is composed of a hollow motor, a Dove prism, and a high-speed camera and controls optical image rotation according to the target rotation by using high-speed image processing. This so-called optical lever formed of the Dove prism worked effectively also for a high-speed rotating target, and our prototype system shows that rotational motion blur was suppressed to 0.125 [°] at 1420 [r/min].

High-speed vision technology in which not just image capturing and recording but also image processing are executed simultaneously at high frame rates, exceeding video rates (30 Hz), has recently been considered an important technology for various applications, such as robotics, and man-machine interfaces. However, the image processing performed in conventional high-speed vision systems is mainly based on two-dimensional pattern recognition. In order to extend the possibilities of this technology, here we focus on real-time three-dimensional sensing at the speeds achievable by high-speed vision systems. Although a related approach for high-speed 3D sensing can achieve a frame rate of over 200 fps, there are disadvantages, including the need for multiple captured frames during sensing, a limited measurement range and low resolution. Our proposed real-time 3D sensing system consists of a projector and two cameras. By projecting a well-designed segmented pattern and using three-viewpoint epipolar constraints, the proposed system can obtain 3D points at high speed. The developed system robustly obtained a 3D shape at 450–500 fps in real time.

Reliable detection of obstacles at long range is crucial for the timely response to hazards by fast-moving safety-critical platforms like autonomous cars. We present a novel method for the joint detection and localization of distant obstacles using a stereo vision system on a moving platform. The approach is applicable to both static and moving obstacles and pushes the limits of detection performance as well as localization accuracy.

This paper develops a simple and low-cost method for 3D, high-rate vehicle state estimation, specially designed for free-flying Micro Aerial Vehicles (MAVs). We fuse observations from inertial measurement units and the recently appeared low-cost optical flow smart cameras. These smart cameras integrate a sonar altimeter, a triaxial gyrometer and an optical flow sensor, and directly provide metric ego-motion information in the form of body velocities and altitude. Compared to state-of-the-art visual-inertial odometry methods, we are able to drastically reduce the computational load in the main processor unit, and obtain an accurate estimation of the vehicle state at a high update rate of 100Hz. We thus extend the current use of these smart cameras from hovering purposes to odometry estimation. In order to propose a simple algorithmic solution, we investigate the performances of two Kalman filters, in the extended and error-state flavors, alongside a large number of algorithm variations, using simulations and real experiments with precise ground-truth. We observe that the marginal performance gain attained with these algorithm improvements does not pay for the effort of implementing them. We conclude that a classical EKF in its simplest form is sufficient for providing motion estimates that coherently exploit the available measurements.

We propose an eye robot system with master-slave control using operator-gaze tracking data, which can produce a high response level such as that of the human eyes. The operators can move a remote robot camera by only moving their eye (master) in front of an eye tracking apparatus and obtain visual information of the environment surrounding the robot's eye (slave). The use of ultrasonic motors has enabled the high responsiveness of the robot eye and the compactness of the robot body. Our graphical user interface (GUI) makes the operation intuitive. We evaluate the robot system from modeling the ultrasonic motor model to the usability of GUI. Finally, the proposed robot system's usability compares well with that of the joystick typically used to operate robots.

Manipulation planning involves planning the combined motion of objects in the environment as well as the robot motions to achieve them. In this paper, we explore a hierarchical approach to planning sequences of non-prehensile and prehensile actions. We subdivide the planning problem into three stages (object contacts, object poses and robot contacts) and thereby reduce the size of search space that is explored. We show that this approach is more efficient than earlier strategies that search in the combined robot-object configuration space directly.

The precise analysis of a patient's or an elderly person's walking pattern is very important for an effective intelligent active mobility assistance robot. This walking pattern can be described by a cyclic motion, which can be modeled using the consecutive gait phases. In this paper, we present a completely non-invasive framework for analyzing and recognizing a pathological human walking gait pattern. Our framework utilizes a laser range finder sensor to detect and track the human legs, and an appropriately synthesized Hidden Markov Model (HMM) for state estimation, and recognition of the gait patterns. We demonstrate the applicability of this setup using real data, collected from an ensemble of different elderly persons with a number of pathologies. The results presented in this paper demonstrate that the proposed human data analysis scheme has the potential to provide the necessary methodological (modeling, inference, and learning) framework for a cognitive behavior-based robot control system. More specifically, the proposed framework has the potential to be used for the classification of specific walking pathologies, which is needed for the development of a context-aware robot mobility assistant.

In order to fully exploit the capabilities of a robotic systems, it is necessary to consider the limitations and errors of actuators and sensors already during the motion planning phase. In this paper, a framework for path planning is introduced, that uses heuristic search to build up a search graph in belief space, an extension to the deterministic state space considering the uncertainty associated with this space. As sources of uncertainty actuator errors and map uncertainties are considered. We apply this framework to various scenarios for a non-holonomic vehicle and compare the resulting paths to heuristic state space planners and LQG-MP[1] with the help of simulations. As a result, paths generated with this framework could either not be found with worst-case assumptions or have a higher probability of being successfully executed compared to planners with more relaxed constraints.

This paper presents an original method for interactions' haptic rendering when treating hyperelastic materials. Such simulations are known to be difficult due to the non-linear behavior of hyperelastic bodies; furthermore, haptic constraints enjoin contact forces to be refreshed at least at 1000 updates per second. To enforce the stability of simulations of generic objects of any range of stiffness, this method relies on implicit time integration. Soft tissues dynamics is simulated in real time (20 to 100 Hz) using the Multiplicative Jacobian Energy Decomposition (MJED) method. An asynchronous preconditioner, updated at low rates (1 to 10 Hz), is used to obtain a close approximation of the mechanical coupling of interactions. Finally, the contact problem is linearized and, using a specific-loop, it is updated at typical haptic rates (around 1000 Hz) allowing this way new simulations of prompt stiff-contacts and providing a continuous haptic feedback as well.