IROS 2017 Papers

The present paper exploits set invariance theory to address the problem of robot implicit force control in presence of stiffness uncertainty in the interaction model. A numerical approach is introduced to compute the invariance function for constraints with arbitrary relative degree. The method is then applied to robot force control, enhancing force regulation performance, in terms of steady state error and convergence speed, despite model mismatch and measurement noise. Its effectiveness is experimentally validated and compared to previous results of set invariance control on a hybrid force/position task performed with a 6 degrees of freedom industrial robot equipped with a force/torque sensor.

Tactile surface sensors (TSSs) are often utilized for contact management in human robot interaction scenarios. To provide added value in these applications, online robot programming approaches may exploit TSSs as gesture input devices. To this end, we introduce an invariant, compact gesture representation which facilitates robust and efficient online gesture recognition even for very small training data sets. The proposed two-stage recognition approach permits reliable gesture classification as well as convenient parameter extraction in the presence of typical disturbances affecting TSSs attached to manipulator links. Our experimental results demonstrate the remarkable recognition performance of the proposed approach using a set of 16 gestures and data of up to 31 subjects.

In recent years, applications of visual tracking algorithms has seen a substantial growth with deployments in intelligent robots such as drones for human tracking. The algorithms for such tasks has to be efficient in terms of computational cost while been robust, accurate and fast. Object tracking algorithms based on handcrafted heuristics and constraints are widely used in uav applications. The handcrafted heuristics are mostly implemented for task-oriented applications which limits the extensions in uav's capability beyond the predefined functions. This paper considers the challenges of tracking and landing an autonomous uav on a speed high moving target, and presents a visual tracking algorithm that integrates correlation filters with deep comparison network for real-time tracking with state-of-the-art accuracy. The method first tracks the target upto translation using an online learnt model via local search technique. The changes in scale is estimated by a deep comparison network (DCN) instead of the commonly used pyramidal approach. In a single network evaluation, DCN can estimate the changes in scale as well as compensate the drifting of the tracker by refining the object region estimated by the correlation filters. The network is end-to-end trained which attempts to learn a powerful matching function for object localization using a known template. Generally, the integrated framework can be viewed as coarse-to-fine level motion estimation. Moreover, the framework can redetect the lost target without a need for a separate detector.

This paper presents a fast and robust multiple object tracking algorithm based on an RGB-D version of the MeanShift tracking algorithm and exploiting RGB-D camera networks when multiple RGB-D sensors are available. The original Mean-Shift algorithm has been improved in three ways. First, a color-depth Joint Probability Density Function is proposed for taking into account both depth and color information. Secondly, we propose an occlusion detection mechanism which can handle long-term occlusions even when objects move fast and unpredictably. Finally, when multiple views are available, we combine the tracking outcomes from all the RGB-D sensors in our network to deal with the identity confusion problem and enhance the overall tracking performance. Experimental results demonstrate that the proposed scheme is robust, realtime and has yielded a marked improvement with respect to the state-of-the-art in terms of tracking quality. As a further contribution, we released our work as open-source in order to provide the best benefit to the wide Computer Vision and Robotics community.

In this paper, we propose a robust on-the-fly estimator initialization algorithm to provide high-quality initial states for monocular visual-inertial systems (VINS). Due to the non-linearity of VINS, a poor initialization can severely impact the performance of either filtering-based or graph-based methods. Our approach starts with a vision-only structure from motion (SfM) to build the up-to-scale structure of camera poses and feature positions. By loosely aligning this structure with pre-integrated IMU measurements, our approach recovers the metric scale, velocity, gravity vector, and gyroscope bias, which are treated as initial values to bootstrap the nonlinear tightly-coupled optimization framework. We highlight that our approach can perform on-the-fly initialization in various scenarios without using any prior information about system states and movement. The performance of the proposed approach is verified through the public UAV dataset and real-time onboard experiment. We make our implementation open source, which is the initialization part integrated in the VINS-Mono1.

This paper aims to leverage magnetic information from a Magneto-Inertial Measurement Unit - an IMU sensor augmented with an array of magnetometers, called MIMU hereafter - in a vision/inertial navigation system (VINS). This ego-motion estimation problem is formulated as an optimization over a sliding window fusing data from the MIMU with features tracked in a monocular camera image stream. The novelty of our approach lies in the formulation of preintegrated magnetic measurements that are computed from successive measurements of the local variations of the magnetic field, in the line of the preintegration of IMU data introduced in [1]. The resulting magnetic error terms participate to the minimized cost function along with the classical reprojection and IMU error terms. Our experiments show the benefits of this fusion. On the one hand, the added magnetic information from the MIMU allows to complement the VINS in cases where vision does not provide useful information during an extended period of time; on the other hand, vision does extend the operational domain of the navigation system compared to a pure MIMU solution, in particular for the outdoor portions of the trajectory.

In this work, we present a real-time robust edge-based visual odometry framework for RGBD sensors (REVO). Even though our method is independent of the edge detection algorithm, we show that the use of state-of-the-art machine-learned edges gives significant improvements in terms of robustness and accuracy compared to standard edge detection methods. In contrast to approaches that heavily rely on the photo-consistency assumption, edges are less influenced by lighting changes and the sparse edge representation offers a larger convergence basin while the pose estimates are also very fast to compute. Further, we introduce a measure for tracking quality, which we use to determine when to insert a new key frame. We show the feasibility of our system on real-world datasets and extensively evaluate on standard benchmark sequences to demonstrate the performance in a wide variety of scenes and camera motions. Our framework runs in real-time on the CPU of a laptop computer and is available online.

In this paper, we investigate the best-of-n distributed decision problem in robot swarms. In this context, we compare the weighted voter model [25] with a three-valued model that incorporates an intermediate belief state meaning either `uncertain' or `indifferent'. We focus particularly on the trade-off between speed of convergence to a shared belief, and robustness to the presence of unreliable individuals in the population. By means of both simulation and embodied experiments in real robot swarms of 400 Kilobots, we show that the three-valued model is much more robust than the weighted voter model, but with decreased speed of convergence.

Ropeways must be stopped when there is strong cross-wind because the ensuing oscillations are a safety risk. In this paper, we developed a controller for an actuated mass onboard the gondola aiming to increase the gondola's damping, thus reducing the oscillations and permitting operation to continue. We modeled the gondola as a pendulum, acknowledged the limitation on the actuated mass' displacement, and included input and parametric uncertainty. We synthesized the controller through a two-step process. First, we used numerical optimization to compute a static full state feedback controller specifically designed to reduce the effects that cross-wind gusts and random disturbances have on the gondola. For this purpose, we included a wind model in the cost function. Then, we used this controller in a model to synthesize an H∞ controller with reduced actuator requirements. Analysis of the linearized system showed that the proposed H∞ controller robustly stabilized the system despite the modeled uncertainty. Furthermore, it made the peak magnitude of the gondola's oscillations in the median of 1005 simulations 51.4 percent less than that of the uncontrolled system, and 19.3 percent less than that of the system using a passive dynamic vibration absorber.

This paper investigates the robust control of an underactuated brachiating robot. Inspired by the pendulumlike movements in gibbons' arboreal locomotion, the controllers are designed to synchronize the brachiator with a virtual oscillator. Two schemes are proposed: a model-dependent feedback linearization scheme and a sliding-mode scheme that is independent of the system model. The simulation results illustrate that the proposed schemes are robust to the arbitrary initial configurations of the brachiator and the limitation in the motor torque at the elbow joint. Furthermore, both controllers enable the underactuated robot to brachiate along a structural member with an upward slope.

In this work a constraint-based control approach is proposed in order to perform a cap rotation task with a dual-arm robot. The method relies on the introduction of a robust specification for the constraint on the interaction force arising during the task, accounting for robot-environment contact model uncertainties, in addition to force measurement noise and surface uncertainties. Experiments have been performed on an ABB dual-arm prototype robot to validate the proposed approach in a cap assembly task, employing a model-based sensorless observer of the interaction forces.

When several Multirotor Micro Aerial Vehicles (MAVs) share the same airspace, reliable and robust collision avoidance is required. In this paper we address the problem of multi-MAV reactive collision avoidance. We employ a model-based controller to simultaneously track a reference trajectory and avoid collisions. Moreover, to achieve a higher degree of robustness, our method also accounts for the uncertainty of the state estimator and of the position and velocity of the other agents. The proposed approach is decentralized, does not require a collision-free reference trajectory and accounts for the full MAV dynamics. We validated our approach in simulation and experimentally with two MAV.

This paper considers the question of measuring attitude using magnetometer-equipped Inertial Measurement Units (IMUs) when in the presence of magnetic disturbance. Specifically, for use in micro-observational class underwater vehicles. Such vehicles' small size and mobility suit activities in coral reefs but often face significant magnetic interference due to high metal debris content, especially in artificial reefs. Their small size also means motor thrusters are situated closer to the magnetometers, producing additional interference. An Adaptive Indirect Kalman Filter is proposed that uses IMU and motor speed measurements to deal with two main sources of disturbances: magnetic fields from foreign objects, and from the vehicle's own motors rotating in slow speed. The method has been evaluated using a custom underwater vehicle made in HKUST, and results show that the proposed method is able to obtain accurate orientation representations even when approaching objects with strong magnetic fields or when the motors are producing interference by rotating in slow speeds.

Modern automation demands mobile robots to be robustly localized in complex scenarios. Current localization systems typically use maps that require to be built and interpreted by experienced operators, increasing deployment costs as well as reducing the adaptability of robots to rearrangements in the environment. In contrast, architectural floor plans can be easily understood by non-expert users and typically represent only the non-rearrangeable parts of buildings. In this paper we propose a system for robot localization in architectural CAD drawings. Our method employs a simultaneous localization and mapping approach to online augment the floor plan with a map represented as a pose-graph with LiDAR measurements. Whenever the environment is accurately mapped in the vicinity of the robot, we use the graph to perform relative localization. We thoroughly evaluate our system in challenging real-world scenarios. Experiments demonstrate that our method is able to robustly track the robot pose even when the floor plan shows major discrepancies from the real-world. We show that our system achieves sub-centimeter accuracy and is suitable for real-time application.

To ensure system integrity, robots need to proactively avoid any unwanted physical perturbation that may cause damage to the underlying hardware. In this paper, we investigate a machine learning approach that allows robots to anticipate impending physical perturbations from perceptual cues. In contrast to other approaches that require knowledge about sources of perturbation to be encoded before deployment, our method is based on experiential learning. Robots learn to associate visual cues with subsequent physical perturbations and contacts. In turn, these extracted visual cues are then used to predict potential future perturbations acting on the robot. To this end, we introduce a novel deep network architecture which combines multiple sub-networks for dealing with robot dynamics and perceptual input from the environment. We present a self-supervised approach for training the system that does not require any labeling of training data. Extensive experiments in a human-robot interaction task show that a robot can learn to predict physical contact by a human interaction partner without any prior information or labeling.

We discuss the architecture and software engineering principles of the Robotics Library (RL). Driven by requirements of robot systems, research projects, industrial applications, and education, we identify relevant design requirements and present an approach to manage hardware and real-time, provide a user-friendly, object-oriented interface to powerful kinematics and dynamics calculations, and support various platforms. After over ten years of development that started in 2004 and evaluating many variants of the architecture, we discuss the design choices for the components of the library in its current version.

Robot assisted surgery has become increasingly popular with the rapid development of more sophisticated robotic systems. Nevertheless, there are still many tedious surgical tasks that remain to be performed manually by surgeons with high supervisions. One example is to tie a surgical knot during the surgery, which includes the processes of needle threading, suture looping, suture tail grasping and suture pulling. In this research, we present a new method for robotic suture looping process. This method is based on a spatial trajectory planning for the surgical instruments, where challenges such as suture slippage and potential collisions between the instruments could be eliminated. In contrast to conventional looping process, a dynamic control scheme is proposed to optimize the time and workspace required to complete the process. Vision information is incorporated in order to enable closed loop control of the robotic system with Linear Quadratic (LQ) controller. A series of experiments were conducted in order to examine the performance of the proposed control scheme and the results confirmed that surgical knots can be successfully tied robotically, offering new insights to the field of automated robotassisted surgery.

As robotic systems become more popular in household environments, the complexity of required tasks also increases. In this work we focus on a domestic chore deemed dull by a majority of the population, the task of ironing. The presented algorithm improves on the limited number of previous works by joining 3D perception with force/torque sensing, with emphasis on finding a practical solution with a feasible implementation in a domestic setting. Our algorithm obtains a point cloud representation of the working environment. From this point cloud, the garment is segmented and a custom Wrinkleness Local Descriptor (WiLD) is computed to determine the location of the present wrinkles. Using this descriptor, the most suitable ironing path is computed and, based on it, the manipulation algorithm performs the force-controlled ironing operation. Experiments have been performed with a humanoid robot platform, proving that our algorithm is able to detect successfully wrinkles present in garments and iteratively reduce the wrinkleness using an unmodified iron.

Deep learning has significantly advanced computer vision and natural language processing. While there have been some successes in robotics using deep learning, it has not been widely adopted. In this paper, we present a novel robotic grasp detection system that predicts the best grasping pose of a parallel-plate robotic gripper for novel objects using the RGB-D image of the scene. The proposed model uses a deep convolutional neural network to extract features from the scene and then uses a shallow convolutional neural network to predict the grasp configuration for the object of interest. Our multi-modal model achieved an accuracy of 89.21% on the standard Cornell Grasp Dataset and runs at real-time speeds. This redefines the state-of-the-art for robotic grasp detection.