IROS 2016 Papers

We present a framework for vision based localization for two or more multirotor aerial vehicles relative to each other. This collaborative localization technique is built upon a relative pose estimation strategy between two or more cameras with the capability of estimating accurate metric poses between each other even through fast motion and continually changing environments. Through synchronized feature detection and tracking with a robust outlier rejection process, classical multiple view geometry concepts have been utilized for obtaining scale-ambiguous relative poses, which are then refined through reconstruction and pose optimization to provide a metric estimate. Furthermore, we present the implementation details of this technique followed by a set of results which involves evaluation of the accuracy of the pose estimates through test cases in both simulated and real experiments. Test cases include keeping one camera stationary as the other is mounted on a quadrotor which is then flown through various types of trajectories. We also perform a quantitative comparison with a GPS/IMU localization technique to demonstrate the accuracy of our method.

We introduce in this paper visibility maps for robots of any shape, representing the reachability limit of the robot's motion and sensing in a 2D gridmap with obstacles. The brute-force approach to determine the optimal visibility map is computationally expensive, and prohibitive with dynamic obstacles. We contribute the Robot-Dependent Visibility Map (RDVM) as a close approximation to the optimal, and an effective algorithm to compute it. The RDVM is a function of the robot's shape, initial position, and sensor model. We first overview the computation of RDVM for the circular robot case in terms of the partial morphological closing operation and the optimal choice for the critical points position. We then present how the RDVM for any-shape robots is computed. In order to handle any robot shape, we introduce in the first step multiple layers that discretize the robot orientation. In the second step, our algorithm determines the frontiers of actuation, similarly to the case of the the circular robot case. We then derive the concept of critical points to the any-shape robot, as the points that maximize expected visibility inside unreachable regions. We compare our method with the ground-truth in a simulated map compiled to capture a variety of challenges of obstacle distribution and type, and discuss the accuracy of our approximation to the optimal visibility map.

Suturing is a challenging and highly dexterous task in minimally invasive surgery, even with the assistance of robotic surgical systems. In this work, we propose a simple yet versatile impedance virtual fixture framework, which can be applied on the master manipulator in a tele-operated robotic surgical system. With this framework, we further develop two types of virtual fixtures that assist with the needle passing and knot tying sub-tasks in suturing. The paper also presents the results of a 14-participant user study for both needle passing and knot tying sub-tasks, showing that virtual fixture assistance for novice users increases the needle passing exit point accuracy, reduces the number of errors (suture slip) in knot tying, and simultaneously decreases the task completion time and overall operator workload.

In this paper, a new method with impulse spectrum is presented as a more general way for vibration control of the robotic arm of flexible multiple links. An impulse spectrum is directly responsible for basic processions. The multiple ODE of the arm is analyzed as an LTI system fit for superposition due to its dynamically relative constant. The study shows that the impulse spectrum proposed to suppress vibration of the multiple links driven by a joint is dependent on the spectral functions of individual subsystem modes. An oscillatory matrix built with spectral functions of higher orders is derived for multi-degree optional design of the multilink, multi-mode spectrum robust against multiple parameters perturbation. Nonlinear programming is discussed to shorten a band spectrum. The band spectra are proposed, presented for wider robustness against variation of frequencies in the vicinity of a centered frequency, and for band robustness over grouped frequencies. Finally simulations are conducted to validate the theoretical work, and the satisfactory results are achieved.

We present a method to validate a cognitive model, based on the cognitive architecture ACT-R, in dynamic human-automation systems with discrete human input. We are inspired by the general problem of K-choice games as a proxy for many decision making applications in dynamical systems. We model the human as a Markovian controller based on gathered experimental data, that is, a non-deterministic control input with known likelihoods of control actions associated with certain configurations of the state-space. We use reachability analysis to predict the outcome of the resulting discrete-time stochastic hybrid system, in which the outcome is defined as a function of the system trajectory. We suggest that the resulting expected outcomes can be used to validate the cognitive model against actual human subject data. We apply our method to a two-choice game in which the human is tasked with maximizing net coverage of a robotic swarm that can operate under rendezvous or deployment dynamics. We validate the corresponding ACTR cognitive model generated with the data from eight human subjects. The novelty of this work is (1) a method to compute expected outcome in a hybrid dynamical system with a Markov chain model of the human's discrete choice, and (2) application of this method to validation of cognitive models with a database of actual human subject data.

Feature learning has become popular in robotics due to recent advances in machine learning. In this paper, we propose a novel method to utilize the high-dimensional features from these techniques as observations in Bayesian estimation problems in a real-time manner. We develop an approach that: 1) pre-processes the observations and maps them into a new space with both reduced dimensions and a linear relationship to the estimation states; and 2) estimates the uncertainty of resulting outputs using data perturbation. The result is that deep learning approaches can be combined with more traditional filtering approaches like the Kalman filter (KF) to achieve state-of-the-art real-time performance. We validate the method by presenting the first real-time application of underwater robot localization using an imaging sonar. The proposed technique shows similar localization accuracy to benchmark approaches while simultaneously achieving real-time performance.

Non-humanoid robots are becoming increasingly utilized for collaborative tasks that rely on each collaborator's ability to effectively convey their mental state while accurately estimating and interpreting their partner's knowledge, intent, and actions. During these tasks, it may be beneficial or even necessary for the human collaborator to assist the robot. Consequently, we explore the use of nonverbal signals to request help during a collaborative task. We focus on light and sound as they are commonly used communication channels across many domains. This paper analyzes the effectiveness of three nonverbal help signals that vary in urgency. Our results show that these signals significantly influence the human collaborator's and their perception of the collaboration.

We present a novel approach for generating sequences of whole-body poses with multi-contacts for humanoid robots, which is inspired by techniques from natural language processing. To this end, we propose a probabilistic n-gram language model learned from observation of human locomotion tasks. Human motion data is automatically segmented according to detected contacts of the body with the environment to provide support, that is, support poses, which are further subdivided with regard to whole-body configuration. These poses are subsequently used to train a language model, whose words are the poses, and whose sentences represent sequences of poses. Then, we propose a planning algorithm that, given the constraints imposed by a task, finds the sequence of transitions with the highest probability according to our language model. We have applied our approach to 140 motion capture recordings of locomotion tasks that involve using one or both hands for support. The evaluation demonstrates that our approach is able to generate complex sets of pose transitions, and shows promising results regarding its application to more complex tasks.

We address in this paper the ill-posed problem of initial alignment of pre-operative to intra-operative data for augmented reality during minimally invasive hepatic surgery. This problem consists of finding the rigid transformation that relates the scanning reference and the endoscopic camera pose, and the non-rigid transformation undergone by the liver w.r.t its scanned state. Most of the state-of-the-art methods assume a known initial registration. Here, we propose a method that permits to recover the deformation undergone by the liver while simultaneously finding the rotational and translational parts of the transformation. Our formulation considers the boundaries of the liver with its surrounding tissues as hard constraints directly encoded in an energy minimization process. We performed experiments on real in-vivo data of human hepatic surgery and synthetic data, and compared our method with related works.

All over the world, people find joy and amusement in playing hand-clapping games such as “Pat-a-cake” and “Slide.” Thus, as robots enter everyday human spaces and work together with people, we see potential for them to entertain, engage, and assist humans through cooperative clapping games. This paper explores how data recorded from a pair of commonly available inertial measurement units (IMUs) worn on a human's hands can contribute to the teaching of a hand-clapping robot. We identified representative hand-clapping activities, considered approaches to classify games, and conducted a study to record hand-clapping motion data. Analysis of data from fifteen participants indicates that support vector machines and Markov chain analysis can correctly classify 95.5% of the demonstrated hand-clapping motions (from ten discrete actions) and 92.3% of the hand-clapping game demonstrations recorded in the study. These results were calculated by withholding a participant's entire dataset for testing, so these results should represent general system behavior for new users. Overall, this research lays the groundwork for a simple and efficient method that people could use to demonstrate hand-clapping games to robots.

Fall detection systems have been proposed to prevent additional injuries following fall accidents. This paper introduces an easily learnable fall detection system based on the data of an individual patient in a hospital room. The improvement of low performance using a single accelerometer at wrists and the inconvenience of sensor attached to a waist in the conventional approach was concentrated on by integrating heart rate signals to the conventional acceleration approach and changing the sensor location from a waist to wrists. As for the optimal heart rate feature selection, we proposed a four-feature vector combination (root mean square of successive differences, standard deviation of successive differences, normal to normal 50, normal to normal 20) with correlation and mutual information analysis in addition to mean absolute deviation selected as an accelerometer feature. To easily acquire and train the patients' fall data, our system was based on unsupervised learning approaches using Gaussian mixture models for optimal classifiers with the optimal cluster number decided by cluster validation index of square error sum. A 10-fold cross validation was applied for a final performance evaluation where each threshold for separating fall state from non-fall state was automatically decided in several comparison groups, which were created on the basis of fusion timing and used sensors. As a result, despite sensors attached to the wrist, the wearable inconvenience of the conventional is overcome using the feature-level fused approach between heart rates and accelerations with the accuracy up to 98.39 %, which is closest to 99.34 % of the case using a single accelerometer located at the waist.

Robots typically possess sensors of different modalities, such as colour cameras, inertial measurement units, and 3D laser scanners. Often, solving a particular problem becomes easier when more than one modality is used. However, while there are undeniable benefits to combine sensors of different modalities the process tends to be complicated. Segmenting scenes observed by the robot into a discrete set of classes is a central requirement for autonomy as understanding the scene is the first step to reason about future situations. Scene segmentation is commonly performed using either image data or 3D point cloud data. In computer vision many successful methods for scene segmentation are based on conditional random fields (CRF) where the maximum a posteriori (MAP) solution to the segmentation can be obtained by inference. In this paper we devise a new CRF inference method for scene segmentation that incorporates global constraints, enforcing the sets of nodes are assigned the same class label. To do this efficiently, the CRF is formulated as a relaxed quadratic program whose MAP solution is found using a gradient-based optimisation approach. The proposed method is evaluated on images and 3D point cloud data gathered in urban environments where image data provides the appearance features needed by the CRF, while the 3D point cloud data provides global spatial constraints over sets of nodes. Comparisons with belief propagation, conventional quadratic programming relaxation, and higher order potential CRF show the benefits of the proposed method.

Powered transfemoral prostheses are robotic systems that aim to restore the mobility of transfemoral amputees by mimicking the functionalities of healthy human legs. The advantage of using a powered prosthetic device is the enhanced performance on various terrains. One of the most frequent terrain found during daily locomotion (other than flat ground) is the surface with slope. In this work, we introduce a framework to generate upslope walking gaits automatically utilizing an online algorithmic formulation. This approach is inspired from analyzing human gait characteristics during upslope walking. In particularly, it is found that the ankle and knee trajectories of upslope walking share a similar pattern with flat ground walking during the middle section (from 20% to 80%) of one step. This observation motivates us to propose an approach of blending the first portion of nominal flat ground gaits with a set of cubic splines to achieve upslope gaits. Importantly, parameters of these cubic splines are solved using an online optimization, which gives the users ability to traverse in different terrains without using any intention detection algorithm. For the last portion of a step, an impedance controller with low gains is considered upon the contact of prosthetic legs to the ground, which allows the users to step onto unknown terrains. The proposed framework is validated on a custom transfemoral prosthesis AMPRO II with showing automatic motion switches between flat ground and upslope walking.

Compressors and air hoses limit the mobility of mobile robots and the use of pneumatic actuators such that small pneumatic actuators, which can be driven without compressors, are strongly desired. We have been developing a new hose-free pneumatic actuator based on reversible gas/liquid chemical reactions. This novel pneumatic actuator neither requires compressors nor air hoses for its source of power because its pressure source is in the actuator itself. In this paper, we present a way to estimate gas generation speed from both theoretical and experimental aspects. We also present a new untethered soft robot, which has this actuator located internally. This robot has three hands and can move them up in 16 s and down in 35 s.

Autonomous robots and humans need to create a coherent 3D representation of their peripersonal space in order to interact with nearby objects. Recent studies in visual neuroscience suggest that the small coordinated head/eye movements that humans continually perform during fixation provides useful depth information. In this work, we mimic such a behavior on a humanoid robot and propose a computational model that extracts depth information without requiring the kinematic model of the robot. First, we show that, during fixational head/eye movements, proprioceptive cues and optic flow lie on a low dimensional subspace that is a function of the depth of the target. Then, we use the generative adaptive subspace self-organizing map (GASSOM) to learn these depth-dependent subspaces. The depth of the target is eventually decoded using a winner-take-all strategy. The proposed model is validated on a simulated model of the iCub robot.

In this paper, we describe an algorithm, based on the well-known Unscented Quaternion Estimator, to estimate external forces and torques acting on a quadrotor. This formulation uses a non-linear model for the quadrotor dynamics, naturally incorporates process and measurement noise, requires only a few parameters to be tuned manually, and uses singularity-free unit quaternions to represent attitude. We demonstrate in simulation that the proposed algorithm can outperform existing methods. We then highlight how our approach can be used to generate force and torque profiles from experimental data, and how this information can later be used for controller design. Finally, we show how the resulting controllers enable a quadrotor to stay in the wind field of a moving fan.

Safe and robust grasping of unknown objects is a major challenge in robotics, which has no general solution yet. A promising approach relies on haptic exploration, where active optimization strategies can be employed to reduce the number of exploration trials. One critical problem is that certain optimal grasps discoverd by the optimization procedure may be very sensitive to small deviations of the parameters from their nominal values: we call these unsafe grasps because small errors during motor execution may turn optimal grasps into bad grasps. To reduce the risk of grasp failure, safe grasps should be favoured. Therefore, we propose a new algorithm, unscented Bayesian optimization, that performs efficient optimization while considering uncertainty in the input space, leading to the discovery of safe optima. The results highlight how our method outperforms the classical Bayesian optimization both in synthetic problems and in realistic robot grasp simulations, finding robust and safe grasps after a few exploration trials.

This paper presents a new recursive information consensus filter for decentralized dynamic-state estimation. Local estimators are assumed to have access only to local information and no structure is assumed about the topology of the communication network, which need not be connected at all times. Iterative Covariance Intersection (ICI) is used to reach consensus over priors which might become correlated, while consensus over new information is handled using weights based on a Metropolis Hastings Markov Chain (MHMC). We establish bounds for estimation performance and show that our method produces unbiased conservative estimates that are better than CI. The performance of the proposed method is evaluated and compared with competing algorithms on an atmospheric dispersion problem.

Interaction control is a mature, but still, current area of research in robotics. Various approaches have been developed for passive and active force regulation and tracking, e.g. impedance control, direct force and hybrid force-position control. The latter method implements a force feedback outer loop on top of a position controller inner loop, where commands are issued with respect to a compliant frame. However, if the compliant frame undergoes a rigid transformation, e.g. in order to specify the same task relative to another reference frame, then commands can be rendered non-compliant with the task at hand. As a consequence, the robot can be damaged or hurt somebody, this is further aggravated by the rigidness of a position controller. In the present paper, we propose a unified impedance and hybrid force-position control scheme to address such issues. The unified controller benefits from the impedance control compliance while an explicit force value can be achieved. Furthermore, we augment the designed controller with a kinestatic filter, which ensures that the commanded pose and wrench are consistent with a given task model. We validate the designed system through experiments with a lightweight robot (LWR). The proposed approach finds applications in industrial settings where interactions with the environment are required in order to fulfill a task and the system must be robust w.r.t. input commands.