IROS 2017 Papers

The use of immersive Virtual Reality (VR) for studying Human-Robot Interaction (HRI) offers many benefits, including decreased cost and risk as well as increased experimental control and repeatability. Previous work has shown that people reliably underestimate distances in VR; however, the effect of this underestimation on gesture recognition has not been characterized. This work contributes to the validation of immersive VR as a platform for HRI investigation and simulation for training in industry. A matched pair of studies compared the location preferences of human participants when viewing gestures generated by a robot in both virtual and physical environments. Participants were asked to select up to three optimal locations within a bounded region at which they perceived the robot's gesture to be the clearest. We found that the use of VR did increase the preferred proxemic distance (χ2 (1) = 18.046, p <; 0.001) by approximately 642 ± 96mm. The difference in viewing angle between the virtual and physical environments was not significant, with a 95% confidence interval limiting the difference within -8.6° to +7.9°. Observations relating gesture features to optimal viewing locations are also presented.

Wire detection is a key capability for safe navigation of autonomous aerial vehicles and is a challenging problem as wires are generally only a few pixels wide, can appear at any orientation and location, and are hard to distinguish from other similar looking lines and edges. We leverage the recent advances in deep learning by treating wire detection as a semantic segmentation task, and investigate the effectiveness of convolutional neural networks for the same. To find an optimal model in terms of detection accuracy and real time performance on a portable GPU, we perform a grid search over a finite space of architectures. Further, to combat the issue of unavailability of a large public dataset with annotations, we render synthetic wires using a ray tracing engine, and overlay them on 67K images from flight videos available on the internet. We use this synthetic dataset for pretraining our models before finetuning on real data, and show that synthetic data alone can lead to pretty accurate detections qualitatively as well. We also verify if providing explicit information about local evidence of wiry-ness in the form of edge and line detection results from a traditional computer vision method, as additional channels to the network input, makes the task easier or not. We evaluate our best models from the grid search on a publicly available dataset and show that they outperform previous work using traditional computer vision and various deep net baselines of FCNs, SegNet and E-Net, on both standard edge detection metrics and inference speed. Our top models run at more than 3Hz on the NVIDIA Jetson TX2 with input resolution of 480×640, with an Average Precision score of 0.73 on our test split of the USF dataset.

We consider the problem of developing robots that navigate like pedestrians on sidewalks through city centers for performing various tasks including delivery and surveillance. One particular challenge for such robots is crossing streets without pedestrian traffic lights. To solve this task the robot has to decide based on its sensory input if the road is clear. In this work, we propose a novel multi-modal learning approach for the problem of autonomous street crossing. Our approach solely relies on laser and radar data and learns a classifier based on Random Forests to predict when it is safe to cross the road. We present extensive experimental evaluations using real-world data collected from multiple street crossing situations which demonstrate that our approach yields a safe and accurate street crossing behavior and generalizes well over different types of situations. A comparison to alternative methods demonstrates the advantages of our approach.

Current research interests for humanoid robots include cost, static stability, and human-likeness. With the help of well-developed locomotion controllers, human-like gait has been realized on humanoid robots with higher energy efficiency and increased motion speed. However, such results are based on the walking function of bipedal robots, running function has been less touched or investigated. Different from walking, running could cause more injuries when human cope with the impact from the ground. In this paper, we define a new foot strike pattern that describes the instinctive human running manner and the performance of a group of adults is collected and analysed for comparison of different foot strike patterns. This helps with lower limbs' injury prevention by proving that most of the impact transient is reduced in the new foot strike pattern for both the shod and the barefoot conditions. Therefore, the newly defined foot strike pattern can be considered as an alternative option to reduce the injury of running activities. Furthermore, it would be possible to apply the foot strike pattern mechanism in humanoid robots, not only to extend the service life from the collision and vibration damage of the components, but also to make robots more human-like.

This paper is about enabling robots to improve their perceptual performance through repeated use in their operating environment, creating local expert detectors fitted to the places through which a robot moves. We leverage the concept of `experiences' in visual perception for robotics, accounting for bias in the data a robot sees by fitting object detector models to a particular `place'. The key question we seek to answer in this paper is simply: how do we define a place? We build bespoke pedestrian detector models for autonomous driving, highlighting the necessary trade off between generalisation and model capacity as we vary the extent of the `place' we fit to. We demonstrate a sizeable performance gain over a current state-of-the-art detector when using computationally lightweight bespoke place-fitted detector models.

The paper proposes a method to assess the feasible torque bandwidth for electrically driven torque controllable actuators over its entire torque amplitude range. The method solely relies on the knowledge of hardware parameters and thereby determines the physically feasible torque control bandwidth at a given torque amplitude, independent of a controller. The method yields torque-frequency diagrams that are suitable to benchmark torque controllers, formulate actuator design specifications and compare as well as select actuators for a specific torque control application. The paper exemplifies the method on a WALK-MAN leg actuator with locked actuator output and the more practical case of a varying load inertia.

This paper describes a octo-rotor unmanned aerial vehicle (UAV) with one degree of freedom (DoF) manipulator for bridge inspection. The UAV has an ability to contact bridge pier to conduct hammering test. To control contact force applied by the UAV to the bridge pier, one DoF is enough for the manipulator implemented on the UAV. A preliminary experiment shows that the force is proportional to the pitch angle of the UAV when it contacts the bridge pier. A bridge pier contact experiment shows that our developed UAV can control contact force between its end-effector and the bridge pier; mean error of the contact force is 4.27 ± 3.64 [N] against 20[N] desired force.

While humans are highly capable of recovering from external disturbances and uncertainties that result in large tracking errors, humanoid robots have yet to reliably mimic this level of robustness. Essential to this is the ability to combine traditional “ankle strategy” balancing with step timing and location adjustment techniques. In doing so, the robot is able to step quickly to the necessary location to continue walking. In this work, we present both a new swing speed up algorithm to adjust the step timing, allowing the robot to set the foot down more quickly to recover from errors in the direction of the current capture point dynamics, and a new algorithm to adjust the desired footstep, expanding the base of support to utilize the center of pressure (CoP)-based ankle strategy for balance. We then utilize the desired centroidal moment pivot (CMP) to calculate the momentum rate of change for our inverse-dynamics based whole-body controller. We present simulation and experimental results using this work, and discuss performance limitations and potential improvements.

Micro Aerial Vehicles (MAVs) that operate in unstructured, unexplored environments require fast and flexible local planning, which can replan when new parts of the map are explored. Trajectory optimization methods fulfill these needs, but require obstacle distance information, which can be given by Euclidean Signed Distance Fields (ESDFs). We propose a method to incrementally build ESDFs from Truncated Signed Distance Fields (TSDFs), a common implicit surface representation used in computer graphics and vision. TSDFs are fast to build and smooth out sensor noise over many observations, and are designed to produce surface meshes. We show that we can build TSDFs faster than Octomaps, and that it is more accurate to build ESDFs out of TSDFs than occupancy maps. Our complete system, called voxblox, is available as open source and runs in real-time on a single CPU core. We validate our approach on-board an MAV, by using our system with a trajectory optimization local planner, entirely on-board and in real-time.

Autonomous mobile robots continuously perceive the world, plan or replan to achieve objectives, and execute the selected actions. Videos of autonomous robots are often naturally used to aid in replaying and demonstrating robot performance. However, plain videos contain no information about the ongoing internals of the robots. In this work, we contribute an approach to automate the overlay of visual annotations on videos of robots' execution to capture information underlying their reasoning. We concretely focus our presentation on the complex robot soccer domain, where the high speed of the robots' execution results from action planning for collaboration and response to the adversary.

Manipulation tasks requiring high precision are difficult for reasons such as imprecise calibration and perceptual inaccuracies. We present a method for visual task outcome verification that provides an assessment of the task status as well as information for the robot to improve this status. The final status of the task is assessed as success, failure or in progress. We propose a deep learning strategy to learn the task with a small number of training episodes and without requiring the robot. A probabilistic, appearance-based pose estimation method is used to learn the demonstrated task. For real-data efficiency, synthetic training images are created around the trajectory of the demonstrated task. We show that our method can estimate the task status with high accuracy in several instances of different tasks, and demonstrate the accuracy of a high-precision task on a real robot.

In this paper, we propose a complete scheme to control a mirror, using a visual servoing scheme, using lines as a set of visual features. Considering the equations of the projection of the reflection of a lines on a mirror, this paper introduces the theoretical background that allows to control the mirror using visual information. Experiments using a mirror mounted on the end-effector of a 6 d.o.f robot validate the proposed approach.

In this paper, we propose a strategy for the control of soft robots with visual tracking and simulation-based predictor. A kinematic model of soft robots is obtained thanks to the Finite Element Method (FEM) computed in real-time. The FEM allows to obtain a prediction of the Jacobian matrix of the robot. This allows a first control of the robot, in the actuator space. Then, a second control strategy based on the feedback of infrared cameras is developed to obtain a correction of the effector position. The robust stability of this closed-loop system is obtained based on Lyapunov stability theory. Otherwise, to deal with the problem of image features (the marker points placed on the end effector of soft robot) loss, a switched control strategy is proposed to combine both the open-loop controller and the closed-loop controller. Finally, experiments on a parallel soft robot driven by four cables are conducted and show the effectiveness of these methods for the real-time control of soft robots.

For robotic systems to interact with or learn from the actions of surrounding humans, it is important that they can accurately interpret the intention driving human motor actions. Making such interpretations, however, requires the ability to perceive the relevant feature(s) from the observed human behavior. With visual sensing alone, robots are typically limited to perceiving only the human's overt motion in the form of joint angles and positions. Ideally, robots designed to interface with humans would also be able to infer information as to how the human is controlling itself from that overt motion. In this study, we investigated if and how humans might be able to visually sense changes in limb mechanical impedance of others. Results indicated that humans can visually perceive changes in joint stiffness from the motion of a two-link planar arm, suggesting that humans can extract information regarding how humans control limb impedance from kinematic information. These findings have important implications for applications where robots must interpret the motor actions of humans, such as during robot imitation learning and human-robot physical interaction.

In this paper, we propose a system for autonomous vehicle following without a line of sight. From monocular camera images, the leading vehicle extracts scene descriptors which it transmits to the following vehicle by means of vehicle-to-vehicle (V2V) communication. The follower is able to recognize the scenes using its own camera and follow autonomously. A particle filter framework is employed for jump-free localization on the driven path of the leading vehicle. We compare the performance of different place features for accurate localization on a custom application-oriented dataset and evaluate methods to reduce the feature size for low-bandwidth V2V communication, while maintaining and even improving the recognition performance. Real-world results demonstrate the applicability of our system.

Visual inertial odometry (VIO) gained lots of interest recently for efficient and accurate ego-motion estimation of robots and automobiles. With a monocular camera and an inertial measurement unit (IMU) rigidly attached, VIO aims to estimate the 3D pose trajectory of the device in a global metric space. We propose a novel visual inertial odometry algorithm which directly optimizes the camera poses with noisy IMU data and visual feature locations. Instead of running separate filters for IMU and visual data, we put them into a unified non-linear optimization framework in which the perspective reprojection costs of visual features and the motion costs on the acceleration and angular velocity from the IMU and pose trajectory are jointly optimized. The proposed system is tested on the EuRoC dataset for quantitative comparison with the state-of-the-art in visual-inertial odometry and on the mobile phone data as a real-world application. The proposed algorithm is conceptually very clear and simple, achieves good accuracy, and can be easily implemented using publicly available non-linear optimization toolkits.

In this paper, we propose a visual homing method by mismatch removal and robust interpolation of sparse motion flows. First, a smoothness prior is proposed and verified by using synthetic and real panoramic images. Then a mismatch removal method is developed for panoramic images based on such smoothness prior. Finally, an interpolation function for the motion flow is obtained simultaneously as a byproduct of mismatch removal, and the focus-of-expansion or focus-of-contraction is used to determine homing directions. The proposed visual homing can be used alone. Also the mismatch removal method can be used as a pre-processing method, and integrated with any other visual homing method which depends on precious keypoints matching. The effectiveness of our method is demonstrated by a panoramic dataset for visual homing.

Feedback control problems pertaining to the control of tensegrity robotic systems are detailed in this paper. The unique problems that arise due to the positivity of the string tensions required to maintain the static stability and desirable stiffness of the structural system are shown to bring about interesting opportunities to optimize for the redundancy in the actuation process. The static stability consideration, coupled with the nonlinear dynamics and the sensor models introduce additional algebraic constraints in implementation of both kinematic and model based dynamic controllers for tensegrity systems. Approaches to develop kinematic and dynamic control techniques are detailed in this paper. A bench top experimental setup consisting of a simple tensegrity system is utilized to demonstrate the efficacy of the output feedback control approach developed in the paper. Near real time image measurements are utilized to drive the output error used in the control scheme.

This paper addresses recursive markerless estimation of a robot's end-effector using visual observations from its cameras. The problem is formulated into the Bayesian framework and addressed using Sequential Monte Carlo (SMC) filtering. We use a 3D rendering engine and Computer Aided Design (CAD) schematics of the robot to virtually create images from the robot's camera viewpoints. These images are then used to extract information and estimate the pose of the end-effector. To this aim, we developed a particle filter for estimating the position and orientation of the robot's end-effector using the Histogram of Oriented Gradient (HOG) descriptors to capture robust characteristic features of shapes in both cameras and rendered images. We implemented the algorithm on the iCub humanoid robot and employed it in a closed-loop reaching scenario. We demonstrate that the tracking is robust to clutter, allows compensating for errors in the robot kinematics and servoing the arm in closed loop using vision.