ICRA 2015 Papers

Myoelectric control has seen decades of research as a potential interface between human and machines. High-density surface electromyography (HDsEMG) non-invasively provides a rich set of signals representing underlying muscle contractions and, at a higher level, human motion intent. Many pattern recognition techniques have been proposed to predict motions based on these signals. However, control schemes incorporating pattern recognition struggle with long-term reliability due to signal stochasticity and transient changes. This study proposes an alternative approach for HDsEMG-based interfaces using concepts of motor skill learning and muscle synergies to address long-term reliability. Muscle synergy-inspired decomposition reduces HDsEMG into control inputs robust to small electrode displacements. The novel control scheme provides simultaneous and proportional control, and is learned by the subject simply by interacting with the device. In a multiple-day experiment, subjects learned to control a virtual 7-DoF myoelectric interface, displaying performance learning curves consistent with motor skill learning. On a separate day, subjects intuitively transferred this learning to demonstrate precision tasks with a 7-DoF robot arm, without requiring any recalibration. These results suggest that the proposed method may be a practical alternative to pattern recognition-based control for long-term use of myoelectric interfaces.

We present an unsupervised framework for simultaneous appearance-based object discovery, detection, tracking and reconstruction using RGBD cameras and a robot manipulator. The system performs dense 3D simultaneous localization and mapping concurrently with unsupervised object discovery. Putative objects that are spatially and visually coherent are manipulated by the robot to gain additional motion-cues. The robot uses appearance alone, followed by structure and motion cues, to jointly discover, verify, learn and improve models of objects. Induced motion segmentation reinforces learned models which are represented implicitly as 2D and 3D level sets to capture both shape and appearance. We compare three different approaches for appearance-based object discovery and find that a novel form of spatio-temporal super-pixels gives the highest quality candidate object models in terms of precision and recall. Live experiments with a Baxter robot demonstrate a holistic pipeline capable of automatic discovery, verification, detection, tracking and reconstruction of unknown objects.

Simultaneous localization and mapping with infinite planes is attractive because of the reduced complexity with respect to both sparse point-based and dense volumetric methods. We show how to include infinite planes into a least-squares formulation for mapping, using a homogeneous plane parametrization with a corresponding minimal representation for the optimization. Because it is a minimal representation, it is suitable for use with Gauss-Newton, Powell's Dog Leg and incremental solvers such as iSAM. We also introduce a relative plane formulation that improves convergence. We evaluate our proposed approach on simulated data to show its advantages over alternative solutions. We also introduce a simple mapping system and present experimental results, showing real-time mapping of select indoor environments with a hand-held RGB-D sensor.

In an asteroid exploration and sample return mission, accurate estimation of the shape and motion of the target asteroid is essential for selecting a touchdown site and navigating a spacecraft during touchdown operation. In this work, we present an automatic estimation method for the shape and motion of an asteroid, which is planned to be tested in future exploration missions including Japanese Hayabusa-2 [1]. Our task is to estimate the shape and rotation axis of the asteroid, as well as positions of the spacecraft from optical images. The proposed method is based on the expectation conditional-maximization (ECM) framework that consists of an auxiliary particle filter and nonlinear optimization techniques. One of our technical contributions is the estimation of the direction of rotation axis of the asteroid from monocular camera images, which are taken by the moving spacecraft. We conducted two experiments with synthetic data and an asteroid mock-up to show the validity of the proposed method and to present the numerical accuracy.

In this paper, we extend our previous external wrench estimation scheme for flying robots with an aerodynamic model such that we are able to simultaneously estimate aerodynamic and contact forces online. This information can be used to identify the metric wind velocity vector via model inversion. Noticeably, we are still able to accurately sense collision forces at the same time. Discrimination between the two is achieved by identifying the natural contact frequency characteristics for both “interaction cases”. This information is then used to design suitable filters that are able to separate the aerodynamic from the collision forces for subsequent use. Now, the flying system is able to correctly respond to typical contact forces and does not accidentally “hallucinate” contacts due to a misinterpretation of wind disturbances. Overall, this paper generalizes our previous results towards significantly more complex environments.

There is growing need for software tools that can accurately simulate the complex dynamics of modern robots. While a number of candidates exist, the field is fragmented. It is difficult to select the best tool for a given project, or to predict how much effort will be needed and what the ultimate simulation performance will be. Here we introduce new quantitative measures of simulation performance, focusing on the numerical challenges that are typical for robotics as opposed to multi-body dynamics and gaming. We then present extensive simulation results, obtained within a new software framework for instantiating the same model in multiple engines and running side-by-side comparisons. Overall we find that each engine performs best on the type of system it was designed and optimized for: MuJoCo wins the robotics-related tests, while the gaming engines win the gaming-related tests without a clear leader among them. The simulations are illustrated in the accompanying movie.

Needle placement errors can mitigate the effectiveness of the diagnosis or the therapy, sometimes with catastrophic outcomes. Previous design of a simplified model for needle deflection estimation was motivated by the clinical constraints of ARCS (Abdomino-pelvic Robotic-driven slightly flexible needle insertion performed in CT/MRI-guided Scenario). We present in this work, the validation results for the needle deflection prediction model. Its robustness is evaluated under an unknown context such as a different robotic platform, facing uncertainties conditions not conceived previously in the model's confection. In addition, the work presents the development and validation experiments of an adaptive path planner that uses the model as predictor's strategy. It provides pre-operative planning assistance, as well as intra-operative decision-making support. The experiments results showed average error around 1mm for the pre-operative planning and the intra-operative replanning approach showed to be very robust to correct the initial predictions, showing average error smaller than 1 mm.

This paper presents an algorithm for aerial vehicle trajectory generation based on the optimal Rapidly-exploring Random Tree (RRT*). The trajectory generation for the aerial vehicle is a complex path planning problem since the vehicle flies in a heterogeneous environment. The vehicle must also avoid some obstacles or inaccessible zones such as buildings, mountains and even radar detection zones depending on the mission. The RRT* algorithm is used as a basis to find near-optimal solutions for this problem. The shortest Dubins' path in heterogeneous environment is used to compute a distance and a trajectory between two vehicle states. Simulated results show the capability of the algorithm to find a feasible near-optimal trajectory in terms of path length that anticipates future flight conditions, such as the decrease in maneuverability in high altitude. The results also show the advantages over the numerical methods in avoiding obstacles.

Outdoor robotic systems rely on perception modules that must be robust to variations in environmental conditions. In particular, vision-based perception systems are affected by illumination variations caused by occlusions. We propose an approach to calculate the lighting distribution of outdoor scenes. The new approach enables us to compensate for shadows and therefore obtain images which are invariant to the sun position and scene geometry, while also retaining the dimensionality of the original data. The method combines images with geometric information provided by range sensors to infer shadows. We select a pair of points on a shadow boundary from a single material and estimate a terrestrial sunlight-skylight ratio. Individual scaling factors are then calculated for all points based on their orientation and incident illumination sources. The result is a coloured point cloud that is independent of illumination variation due to occlusions and geometry. To demonstrate the effectiveness and generalisation of the approach, we present evaluations using two datasets with different cameras. The first uses a hyperspectral sensor that allows us to analyse the results for a large number of wavelengths, while the second dataset uses a standard RGB camera. The approach is shown to consistently provide good illumination compensation in both scenarios.

Traditional visual servoing is able to navigate a robotic system between two views of the same object. However, it is not designed to servo between views of different objects. In this paper, we consider a novel problem of servoing any instance (exemplar) of an object category to a desired pose (view) and propose a strategy to accomplish the task. We use features that semantically encode the locations of object parts and define the servoing error as the difference between positions of corresponding parts in the image space. Our controller is based on the linear combination of 3D models, such that the resulting model interpolates between the given and desired instances. We conducted our experiments on five different object categories in simulation framework and show that our approach achieves the desired pose with smooth trajectory. Furthermore, we show the performance gain achieved by using a linear combination of models (instances) vis a vis a controller that switches across models during servoing in terms of trajectory's length, smoothness and error in camera pose and image features.

In this paper, we propose a novel approach for generating generic object candidates for object discovery and recognition in continuous monocular video. Such candidates have recently become a popular alternative to exhaustive window-based search as basis for classification. Contrary to previous approaches, we address the candidate generation problem at the level of entire video sequences instead of at the single image level. We propose a processing pipeline that starts from individual region candidates and tracks them over time. This enables us to group candidates for similar objects and to automatically filter out inconsistent regions. For generating the per-frame candidates, we introduce a novel multi-scale saliency approach that achieves a higher per-frame recall with fewer candidates than current state-of-the-art methods. Taken together, those two components result in a significant reduction of the number of object candidates compared to frame level methods, while keeping a consistently high recall.

When a person interacts with an environment using a tool, he/she receives tactile information in the form of fingerpad skin deformation. Different interaction forces and torques on the tool cause different skin deformation patterns on the fingerpads. We designed a 6-degree-of-freedom tactile device that creates similar skin deformation patterns on the fingerpads. The device communicates force and torque information by translating and rotating skin deformation tactors relative to the fingerpads. An experiment was conducted to determine participants' ability to use skin deformation tactile cues to perform a peg-in-hole insertion task. Results show that participants can use the tactile cues to reduce interaction force and torque, and they use the tactile force cues to reduce interaction force more than they use the tactile torque cues to reduce interaction torque. Rendering force and torque cues simultaneously causes device saturation and degrades user performance. These results suggest that additional training may help participants use the skin deformation torque cues, and motivate a tactile device design that decouples force and torque skin deformation rendering to minimize device saturation. Fingerpad skin deformation is a promising form of tactile feedback to convey force and torque information in teleoperation systems such as robot-assisted surgery, where force feedback may be undesirable due to stability and safety concerns.

This paper presents a method of augmenting streaming point cloud data with pretouch proximity sensor information for the purposes of teleoperated grasping of transparent targets. When using commercial RGB-Depth (RGB-D) cameras, material properties can significantly affect depth measurements. In particular, transparent objects are difficult to perceive with RGB images and commercially available depth sensors. Geometric information of such objects needs to be gathered with additional sensors, and in many scenarios, it is of interest to gather this information without physical contact. In this work, a non-contact pretouch sensor fixed to the robot end effector is used to sense and explore physical geometries previously unobserved. Thus, the point cloud representation of an unknown, transparent grasp target, can be enhanced through telerobotic exploration in real-time. Furthermore, real-time haptic rendering algorithms and haptic virtual fixtures used in combination with the augmented streaming point clouds assist the teleoperator in collision avoidance during exploration. Theoretical analyses are performed to design virtual fixtures suitable for pretouch sensing, and experiments show the effectiveness of this method to gather geometry data without collision and eventually to successfully grasp a transparent object.

Semantic understanding of environments is an important problem in robotics in general and intelligent autonomous systems in particular. In this paper, we propose a semantic segmentation algorithm which effectively fuses information from images and 3D point clouds. The proposed method incorporates information from multiple scales in an intuitive and effective manner. A late-fusion architecture is proposed to maximally leverage the training data in each modality. Finally, a pairwise Conditional Random Field (CRF) is used as a post-processing step to enforce spatial consistency in the structured prediction. The proposed algorithm is evaluated on the publicly available KITTI dataset [1] [2], augmented with additional pixel and point-wise semantic labels for building, sky, road, vegetation, sidewalk, car, pedestrian, cyclist, sign/pole, and fence regions. A per-pixel accuracy of 89.3% and average class accuracy of 65.4% is achieved, well above current state-of-the-art [3].

We consider the problem of deploying a swarm of mobile robots into an unknown environment for attaining complete sensor coverage of the environment. The robots have limited and noisy sensing capabilities and no metric or global information available to them. Using tools from algebraic topology, we formally describe the sensor coverage as a simplicial complex, deploy robots through the complex using bearing-based local controllers, and attain coverage while identifying and removing sensor redundancies. Despite the highly limited sensing capabilities and complete lack of global localization and metric information, we demonstrate that the proposed algorithm is complete, always terminates in a finite-sized environment, is guaranteed to attain complete coverage and is robust to sensor failures. The algorithm presented in this paper was demonstrated through simulation and proves to effectively cover and explore unknown indoor environments.

This paper investigates the integration of multiple time-of-flight (ToF) depth sensors for object reconstruction. The advantage of such a sensor network is in the increased viewing coverage and addressing the problem of object self-occlusion. However, in utilizing a network of depth sensors, calibration and interference between the sensors can be one of the key challenges of their effective utilization. In comparison with infrared depth sensors, integration of time-of-flight sensors offer some new challenges. This paper presents experimental studies of practical factors that can affect such integration and proposes a guideline which can be used to avoid interferences. Using the proposed set-up, the paper also presents a method which can be used for reconstructing objects in such a sensor network.

We present the design and implementation of an aircraft collision avoidance algorithm for Unmanned Aerial Vehicles (UAVs). Automatic Dependent Surveillance-Broadcast (ADS-B) is used to detect aircraft. The UAV needs to fly through pre-assigned waypoints while avoiding collisions with other aircraft. The aircraft are indifferent to the UAV. A collision with aircraft are detected by simulating the UAV's trajectory along the path of assigned waypoints using its closed-loop dynamics. A sampling based algorithm is used for collision avoidance path planning. A second collision check is performed on the generated path with the updated UAV and aircraft's states. The path will be re-planned if it leads to a collision. The algorithm was validated in Software-In-the-Loop Simulation (SITL). ADS-B data obtained from commercial aircraft flying over the Phoenix Skyharbor airport were used for simulating the collisions. The paper shows that the algorithm enables the UAV to avoid multiple aircraft with different approaching angles and speeds.

Medical robots have been widely used to assist surgeons to carry out dexterous surgical tasks via various ways. Most of the tasks require surgeon's operation directly or indirectly. Certain level of autonomy in robotic surgery could not only free the surgeon from some tedious repetitive tasks, but also utilize the advantages of robot: high dexterity and accuracy. This paper presents a semi-autonomous neurosurgical procedure of brain tumor ablation using RAVEN Surgical Robot and stereo visual feedback. By integrating with the behavior tree framework, the whole surgical task is modeled flexibly and intelligently as nodes and leaves of a behavior tree. This paper provides three contributions mainly: (1) describing the brain tumor ablation as an ideal candidate for autonomous robotic surgery, (2) modeling and implementing the semi-autonomous surgical task using behavior tree framework, and (3) designing an experimental simulated ablation task for feasibility study and robot performance analysis.

The problem of image based localization has a long history both in robotics and computer vision and shares many similarities with image based retrieval problem. Existing techniques use either local features or (semi)-global image signatures in the context of topological mapping or loop closure detection. Difficulties of the location recognition problem are often affected by large appearance and viewpoint variation between the query view and reference dataset and presence of non-discriminative features due to vegetation, sky and road. In this work we show that semantic segmentation labeling of man-made structures can inform the traditional bag-of-visual words models to obtain proper feature weighting and improve the overall location recognition accuracy. We also demonstrate additional capability of identifying individual buildings and estimating their extent in images, providing the essential building block for semantic localization. Towards this end we introduce a new challenging outdoors urban dataset exhibiting large variations in appearance and viewpoint.