IROS 2016 Papers

In the application of physical human-robot interaction (pHRI), the collaboration between human and robot can significantly improve the production efficiency through combination of the human's flexible intelligence and the robot's consistent performance. In this application, however, it is an important concern to ensure the safety of the human and the robot. In the human guidance programming scenario, the operator plans a collision-free path for the robot end-effector, but the robot body might collide with an obstacle while being guided by the operator. In this paper, a novel on-line velocity based collision avoidance algorithm is developed to solve the problem in this particular scenario. The proposed algorithm gives an explicit solution to deal with both collision avoidance and human guidance command at the same time, which provides the operator a better and safer lead through programming experience. The real-time experiment is performed on FANUC LR Mate 200 iD/7L in three different obstacle scenarios.

Human-Robot Co-Working is an active research domain in the field of robotics and control theory. Most of pioneering contributions addressing Co-Working interaction control problems has focused on PID based control. On the other hand, there are, in modern control theory, several tools allowing to design and tune arbitrarily complex servo-controllers. Non-Smooth optimization is one of recently developed tools that addresses the problem of tuning structured controllers according to H∞, H2, and pole placement specifications. In this paper, the problem of force amplification is transcribed into high level H∞ specifications and is solved using Non-Smooth optimization. The resulting controller is tested on an industrial robot and shows similar performances as the state of the art while respecting some extra robustness certificates. Therefore, thanks to the abstraction level of the proposed technique, it becomes possible to tune interaction controllers with high number of tunable parameters according to complex performance/robustness specifications.

Spatial affordance can be defined as the functionality a space, or place, lends to human activity. Different places afford different activity possibilities - sleeping is mostly done in the bedroom, and cooking is mostly done in the kitchen. Semantic place labels like kitchen and bedroom, therefore, provide context with which a robot can better infer human activity. Real rooms, however, often defy simple place labels, as they can be multi-purpose, supporting many different types of human activity. The solution is to identify the spatial affordances associated with the current nexus of human activity - a microlevel place labeling. In this paper, we will demonstrate how to estimate these local spatial affordances by integrating a deep learning based place estimator with human pose estimation. The resulting affordances are then used to improve activity recognition using Bayesian belief network.

Human activity recognition plays an important role in personal assistive robot, being able to recognize human activity and perform corresponding assistive action is a great challenges for personal assistive robot. Human body is an articulated system of rigid segments that can be divided into five parts, but many existing methods always identify actions based on the motion trajectories of whole body. In this paper, taking into account the fact that most actions can be performed by a few limbs and the other limbs should not impact on the action recognition, we proposed an activity recognition method based on limb weights. The weight of each limb is composed of consistency weight and uniqueness weight, which are learned according to the similarity degree among different sequences for each specific action. The covariance descriptor, which is the concatenation of eigenvalues extracted from covariance matrices, is adopted to represent the motion trajectory of each limb. In order to distinguish action instances from each other in the feature sequences, a simple annotation method is used. Experimental results on the Cornell activity dataset and the Lab dataset show that the proposed method not only can outperform the state-of-the-art algorithms, but also is appropriate to recognize the actions whose non-core limbs' trajectories are different from each other.

Behavior Trees (BTs) is a way of organizing the switching structure of a control system, that was originally developed in the computer gaming industry but is now also being used in robotics. The Teleo-Reactive programs (TRs) is a highly cited reactive hierarchical robot control approach suggested by Nilsson and And-Or-Trees are trees used for heuristic problems solving. In this paper, we show that BTs generalize TRs as well as And-Or-Trees, even though the two concepts are quite different. And-Or-Trees are trees of conditions, and we show that they transform into a feedback execution plan when written as a BT. TRs are hierarchical control structures, and we show how every TR can be written as a BT. Furthermore, we show that so-called Universal TRs, guaranteeing that the goal will be reached, are a special case of so-called Finite Time Successful BTs. This implies that many designs and theoretical results developed for TRs can be applied to BTs.

This work presents a 10 kg tensegrity ball probe that can quickly and precisely deliver a 1 kg payload over a 1 km distance on the Moon by combining cable-driven rolling and thruster-based hopping. Previous research has shown that cable-driven rolling is effective for precise positioning, even in rough terrain. However, traveling large distances using thruster-based hopping, which is made feasible by the lightweight and compliant nature of the tensegrity structure, has not been explored. To evaluate the feasibility of a thruster-based tensegrity robot, a centrally-positioned cold gas thruster with nitrogen propellant was selected, and the system was simulated using the NASA Tensegrity Robotics Toolkit (NTRT) for four hopping profiles on hilly terrains. Optimizing energy efficiency and mechanical capabilities of the tensegrity robot, hopping profiles with a long flight distance per hop, followed by the higher accuracy rolling, are recommended. Simulations also show that thrust regulation can improve energy efficiency. Regulation of thrust magnitude can be achieved using a pressure regulator, but regulation of thrust orientation calls for additional control effort. In this paper, it is demonstrated that gimbal systems as well as shape-shifting control of the tensegrity structure have the potential to regulate thrust orientation. Finally, algorithms for localization and path planning that combine hopping and rolling for energy-efficient navigation are presented.

For a depalletizing robot to be practical, it has to be able to work skillfully in a limited space with a depalletizinging speed equal to or exceeding that of a human. Our new robot has the following advantages. A gantry robot and a telescopic arm mechanism are adopted so that the robot, although space-saving, has a large workspace. Using camera image and depth image, the robot recognizes the various package stacked complicatedly and decides the handling order of the package. The robot generates the most suitable handling path and performs handling control such that stacks do not collapse. The robot achieves fall prevention of the package and stable conveyance by supporting beneath the package with another arm, the conveyor arm. Moreover, in order to accelerate processing of packages, we thought it was important for the robot to be equipped with a wide end effector capable holding multiple packages simultaneously. Therefore we developed a depalletizing simulator and performed the optimal design of the end effector by using a simulator. This, we were able to realize a compact depalletizing robot with a depalletizing speed equal to that of a human.

Stereo cameras and dense stereo matching algorithms are core components for many robotic applications due to their abilities to directly obtain dense depth measurements and their robustness against changes in lighting conditions. However, the performance of dense depth estimation relies heavily on accurate stereo extrinsic calibration. In this work, we present a real-time markerless approach for obtaining high-precision stereo extrinsic calibration using a novel 5-DOF (degrees-of-freedom) and nonlinear optimization on a manifold, which captures the observability property of vision-only stereo calibration. Our method minimizes epipolar errors between spatial per-frame sparse natural features. It does not require temporal feature correspondences, making it not only invariant to dynamic scenes and illumination changes, but also able to run significantly faster than standard bundle adjustment-based approaches. We introduce a principled method to determine if the calibration converges to the required level of accuracy, and show through online experiments that our approach achieves a level of accuracy that is comparable to offline marker-based calibration methods. Our method refines stereo extrinsic to the accuracy that is sufficient for block matching-based dense disparity computation. It provides a cost-effective way to improve the reliability of stereo vision systems for long-term autonomy.

Robotic vision, unlike computer vision, typically involves processing a stream of images from a camera with time varying pose operating in an environment with time varying lighting conditions and moving objects. Repeating robotic vision experiments under identical conditions is often impossible, making it difficult to compare different algorithms. For machine learning applications a critical bottleneck is the limited amount of real world image data that can be captured and labelled for both training and testing purposes. In this paper we investigate the use of a photo-realistic simulation tool to address these challenges, in three specific domains: robust place recognition, visual SLAM and object recognition. For the first two problems we generate images from a complex 3D environment with systematically varying camera paths, camera viewpoints and lighting conditions. For the first time we are able to systematically characterise the performance of these algorithms as paths and lighting conditions change. In particular, we are able to systematically generate varying camera viewpoint datasets that would be difficult or impossible to generate in the real world. We also compare algorithm results for a camera in a real environment and a simulated camera in a simulation model of that real environment. Finally, for the object recognition domain, we generate labelled image data and characterise the viewpoint dependency of a current convolution neural network in performing object recognition. Together these results provide a multi-domain demonstration of the beneficial properties of using simulation to characterise and analyse a wide range of robotic vision algorithms.

One of the most important points in path tracking applications, is the capability of the robot to track the desired path as accurately as possible. Results presented in the literature in on-road contexts are convincing, but the case of off-road mobile robots introduces other issues, like bad grip conditions or non-flat ground. These issues can lead to a lack of accuracy of the tracking, even more when we consider high speed. The quality of the tracking depends on the modelling and the control laws used, but it depends also on the path to follow and on the soil conditions. If some small errors may be acceptable, it is sometimes mandatory to have a limited error, in particular when avoiding an obstacle or when moving in a narrow pathway. The aim of the algorithm proposed in this paper is to determine the maximum velocity of the robot during the tracking, for the lateral error to stay below a desired limit. In order to achieve this, a predictive approach taking into account for the evolution of a dynamic model, control laws and actuators properties is proposed. It permits to predict the forthcoming tracking error and consequently to estimate the maximum velocity that the robot can reach in accordance with the maximum deviation allowed.

While aerial vehicles have become more and more popular in the hobbyist sector as a platform for photography and cinematography, currently there is little utilization of aerial vehicles in an industrial environment. In this work, we describe our efforts to develop an aerial robotics platform with the ability to traverse a ceiling with high stability and precision using a top mounted omni wheel drive system and an AR-marker system. The marker system has been extended to incorporate the use of dual camera sensor fusion. The proposed system is specifically designed and planned to be suitable for a real world industrial assignment, namely the setting of painted ink-marker used as an orientation for drilling, measuring or maintenance tasks on the ceiling in an infrastructure inspection work environment. We show with test flights the feasibility of this approach and the ability of the developed platform to do high precision localization and positioning relative to an AR-marker in order to perform an ink-marker placement task.

This paper considers the problem of grasp pose detection in point clouds. We follow a general algorithmic structure that first generates a large set of 6-DOF grasp candidates and then classifies each of them as a good or a bad grasp. Our focus in this paper is on improving the second step by using depth sensor scans from large online datasets to train a convolutional neural network. We propose two new representations of grasp candidates, and we quantify the effect of using prior knowledge of two forms: instance or category knowledge of the object to be grasped, and pretraining the network on simulated depth data obtained from idealized CAD models. Our analysis shows that a more informative grasp candidate representation as well as pretraining and prior knowledge significantly improve grasp detection. We evaluate our approach on a Baxter Research Robot and demonstrate an average grasp success rate of 93% in dense clutter. This is a 20% improvement compared to our prior work.

This paper is devoted to the performance optimization of an aerial manipulation system composed of a Flettner-helicopter and 7 DoF manipulator. With experiments we demonstrate that the time delays in signal propagation between perception and actuation modules play an important role for the overall performance of an aerial manipulator system using visual servoing. We present an approach for estimation of the perception-action time delay and its active compensation based on the predicted motion of the manipulator end-effector.

In this paper we consider the problem of coordinating the motion of a group of Automated Guided Vehicles (AGVs) utilized in industrial environments for logistics operations. In particular, we consider a hierarchical coordination strategy, where the environment is partitioned into sectors: coordination on the top layer defines the sequence of sectors to be traveled, while coordination on the bottom layer deals with traffic management inside each sector. In this paper we introduce a novel partitioning algorithm, that defines the sectors in a dynamic manner, taking into account both the shape of the (generally non-convex) environment, and the current distribution of the AGVs. This is achieved exploiting a clustering algorithm, and subsequently defining the sectors based on the geodesic distance.

Object shape information is essential for robot manipulation tasks, in particular for grasp planning and collision-free motion planning. But in general a complete object model is not available, in particular when dealing with unknown objects. We propose a method for completing shapes that are only partially known, which is a common situation when a robot perceives a new object only from one direction. Our approach is based on the assumption that most objects used in service robotic setups have symmetries. We determine and rate symmetry plane candidates to estimate the hidden parts of the object. By finding possible supporting planes based on its immediate neighborhood, the search space for symmetry planes is restricted, and the bottom part of the object is added. Gaps along the sides in the direction of the view axis are closed by linear interpolation. We evaluate our approach with real-world experiments using the YCB object and model set [1].

Common design principles for low cost humanoid robots include a low center of mass height and a large support area for increased static stability. However, such principles limit the bipedal mobility of the robot due to the kinematic constraints involved. In this paper, we present an efficient locomotion controller that utilizes automatically calculated heel and toe lift motions to overcome the kinematic constraints. This helps with uneven terrain traversal by providing additional support, and also enables a dynamic heel-strike toe-off gait with a large stride length. We demonstrate the controller in physically realistic simulations, and on the THOR-RD full-sized humanoid robot and DARwIn-OP miniature humanoid robot.

In order to solve the problems of current hearing aid devices, we make use of environment sensor network, and propose a hearing support system, where individual target and anti-target sound sources in the environment can be selected, and spatial information of the target sound sources is reconstructed. The performance of the selective sound separation module was evaluated for different noise conditions. Results showed that signal-to-noise ratios of around 15dB could be achieved by the proposed system for a 65dB babble noise plus directional music noise condition. In the same noise condition, subjective intelligibility tests were conducted, and an improvement of 65 to 90% word intelligibility rates could be achieved by using the proposed hearing support system.

Lateral skin stretch is a promising technology for haptic display of information between an autonomous or semi-autonomous car and a driver. We present the design of a steering wheel with an embedded lateral skin stretch display and report on the results of tests (N=10) conducted in a driving vehicle in suburban traffic. Results are generally consistent with previous results utilizing skin stretch in stationary applications, but a slightly higher, and particularly a faster rate of stretch application is preferred for accurate detection of direction and approximate magnitude.

Among most existing gait rehabilitation robots, it is difficult to find adequate devices for gait rehabilitation of chronic stroke patients who can already stand and move but still need to rehabilitate the affected lower limb through simple, compact, and easy-to use devices. This paper presents a novel haptic based gait rehabilitation system (HGRS) which has the potential to provide over-ground gait training regimens for post-stroke ambulatory subjects. It consists of a portable cane for kinesthetic sensing and a wearable vibrotactor array for tactile biofeedback. Contact of user with the handle provides light grip force, it serves the purpose of balance assurance and increased muscle activity through light touch concept and vibrotactors contribute in enhancing the gait modification through afferent signal of vibration. Walking trials conducted with stroke patients indicate increased muscle activation and balance, and improved temporal symmetry with use of HGRS. HGRS is capable of assisting physical therapists in training individuals with stroke suffering from gait abnormalities. In addition, it is easy to use and low-cost which makes it reachable to a vast domain of subjects suffering from gait abnormalities.