ICRA 2018 Papers

This paper presents trajectory following control experiments of a miniature spherical rolling and spinning robot mechanism on three different types of outdoor surfaces. The research is inspired from the efficient locomotory rolling patterns of various insects in unstructured environment. A nonlinear adaptive sliding mode (ASMC) feedback method maintains the robot stability and robustness in the presence of parameter uncertainties and external disturbances. The proposed trajectory following control policy is developed, implemented and tested for the miniature spherical robot on three different types of irregular surfaces in outdoors. Trajectory following accuracy, roll angle stability and wheel velocity response are three parameters measured to evaluate robot performance. ASMC controller is compared with an integral sliding (ISMC) controller. Experimental results show that proposed control policy is able to manage an accurate trajectory following amidst robust control of a rolling and spinning robot on three types of irregular surface in practical outdoor conditions.

Robots, either made from hard or soft materials, are being used increasingly for unknown object manipulation tasks in unstructured environments. To hold an object firmly and do the required task, the orientation of the object with respect to the robotic hand gripper is one of the necessary key information. Humans can sense the orientation of an object based on vision, kinesthetic sensation, or touch sensation. Similarly, robots shall also be able to estimate grasped object orientation just based on tactile sensing without knowing manipulator kinesthetic or kinematics. Existing sensors are mostly based on vision or rigid inertial measurement units (such as accelerometers, gyroscopes, magnetometers), which require additional setup and are typically not compliant with the arbitrary surfaces of the unknown obj ects. This paper discusses a new approach for object orientation estimation from a multilayered tactile sensor based on conductive silver ink and conductive fabric. Fabric based sensors are flexible and can confer to both hard and soft surfaces. Experimental results show that the tactile sensor developed is able to estimate the tilt angle without any information about manipulator kinematics.

Autonomous underwater vehicles (AUVs) rely on a variety of sensors - acoustic, inertial and visual - for intelligent decision making. Due to its non-intrusive, passive nature and high information content, vision is an attractive sensing modality, particularly at shallower depths. However, factors such as light refraction and absorption, suspended particles in the water, and color distortion affect the quality of visual data, resulting in noisy and distorted images. AUVs that rely on visual sensing thus face difficult challenges and consequently exhibit poor performance on vision-driven tasks. This paper proposes a method to improve the quality of visual underwater scenes using Generative Adversarial Networks (GANs), with the goal of improving input to vision-driven behaviors further down the autonomy pipeline. Furthermore, we show how recently proposed methods are able to generate a dataset for the purpose of such underwater image restoration. For any visually-guided underwater robots, this improvement can result in increased safety and reliability through robust visual perception. To that effect, we present quantitative and qualitative data which demonstrates that images corrected through the proposed approach generate more visually appealing images, and also provide increased accuracy for a diver tracking algorithm.

Humans prefer to use voice commands to guide their peer companions in daily assistive tasks. In human perspective, they expect same behavior from assistive robot companions as well. The paper presents an approach towards using such voice instructions based on uncertain and qualitative information to describe object placements. Consider a case in which a set of objects has to be arranged on a table in a particular spatial area. In such a situation, humans will prefer to use a single command regarding overall arrangement rather than repeating the same command for each and every object placement. In such scenarios, people will be comfortable with commands which are simple and with non-technical words. Most of such commands include uncertain spatial terms such as “Left”, “Middle”, “Right” and uncertain qualitative terms such as “Together”, “Little separately”, “Separately” to describe the arrangement. For example “Keep all objects together in the middle of the table” and “Keep objects separately on the right side of the table” can be considered. But in some situations, these commands will not give a direct idea about the placement location of the objects. For instance, “Keep the middle of the table free” can be cited. Therefore, the robot must be able to understand precisely such information in commands before executing them. The experiments are conducted in a simulated domestic environment. Results of the experiment are presented and discussed.

Robotics teleoperation has been extensively studied and considered in the past in several task scenarios where direct human intervention is not possible due to the hazardous environments. In such applications, both communication degradation and reduced perception of the remote environment are practical issues that can challenge the human operator while controlling the robot and attempting to physically interact within the remote workspace. To address this challenge, we introduce a novel shared-autonomy Tele-Interaction control approach that blends the motion commands from the pilot (master side) with locally (slave side) executed autonomous motion and impedance modulators. This enables a remote robot to handle and autonomously avoid physical obstacles during manoeuvring, reduce interaction forces during contacts, and finally accommodate different payload conditions while at the same time operating with a “default” low impedance setting. We implemented and experimentally validated the proposed method both on simulation and on a real robot platform called CENTAURO. A series of tasks, such as maneuvering through the physical constraints of the remote environment in an autonomous manner, pushing and lifting heavy objects with autonomous impedance regulation and colliding with the rigid geometry of the remote environment were executed. The obtained results demonstrate the effectiveness of the shared-autonomy control principles that eventually aim to reduce the level of attention and stress of human pilot while manoeuvring the slave robot, and at the same time to enhance the robustness of the robot during physical interactions even if accidentally occurred.

A reliable, real time, multi-sensor fusion functionality is crucial for localization of actively controlled capsule endoscopy robots, which are an emerging, minimally invasive diagnostic and therapeutic technology for the gastrointestinal (GI) tract. In this study, we propose a novel multi-sensor fusion approach based on a particle filter that incorporates an online estimation of sensor reliability and a non-linear kinematic model learned by a recurrent neural network. Our method sequentially estimates the true robot pose from noisy pose observations delivered by multiple sensors. We experimentally test the method using 5 degree-of-freedom (5-DoF) absolute pose measurement by a magnetic localization system and a 6-DoF relative pose measurement by visual odometry. In addition, the proposed method is capable of detecting and handling sensor failures by ignoring corrupted data, providing the robustness expected of a medical device. Detailed analyses and evaluations are presented using ex vivo experiments on a porcine stomach model, proving that our system achieves high translational and rotational accuracies for different types of endoscopic capsule robot trajectories.

In the last decade, researchers and medical device companies have made major advances towards transforming passive capsule endoscopes into active medical robots. One of the major challenges is to endow capsule robots with accurate perception of the environment inside the human body, which will provide necessary information and enable improved medical procedures. We extend the success of deep learning approaches from various research fields to the problem of sensor fusion for endoscopic capsule robots in the case of asynchronous and asymmetric sensor data without any need of calibration between sensors. The results performed on real pig stomach datasets show that our method achieves high precision for both translational and rotational movements and contains various advantages over traditional sensor fusion techniques.

In this paper we propose a novel approach to tracking by detection that can exploit both cameras as well as LIDAR data to produce very accurate 3D trajectories. Towards this goal, we formulate the problem as a linear program that can be solved exactly, and learn convolutional networks for detection as well as matching in an end-to-end manner. We evaluate our model in the challenging KITTI dataset and show very competitive results.

We present research using the latest reinforcement learning algorithm for end-to-end driving without any mediated perception (object recognition, scene understanding). The newly proposed reward and learning strategies lead together to faster convergence and more robust driving using only RGB image from a forward facing camera. An Asynchronous Actor Critic (A3C) framework is used to learn the car control in a physically and graphically realistic rally game, with the agents evolving simultaneously on tracks with a variety of road structures (turns, hills), graphics (seasons, location) and physics (road adherence). A thorough evaluation is conducted and generalization is proven on unseen tracks and using legal speed limits. Open loop tests on real sequences of images show some domain adaption capability of our method.

Deep networks trained on demonstrations of human driving have learned to follow roads and avoid obstacles. However, driving policies trained via imitation learning cannot be controlled at test time. A vehicle trained end-to-end to imitate an expert cannot be guided to take a specific turn at an upcoming intersection. This limits the utility of such systems. We propose to condition imitation learning on high-level command input. At test time, the learned driving policy functions as a chauffeur that handles sensorimotor coordination but continues to respond to navigational commands. We evaluate different architectures for conditional imitation learning in vision-based driving. We conduct experiments in realistic three-dimensional simulations of urban driving and on a 1/5 scale robotic truck that is trained to drive in a residential area. Both systems drive based on visual input yet remain responsive to high-level navigational commands.

Learning-based methods have demonstrated clear advantages in controlling robot tasks, such as the information fusion abilities, strong robustness, and high accuracy. Meanwhile, the on-board systems of robots have limited computation and energy resources, which are contradictory with state-of-the-art learning approaches. They are either too lightweight to solve complex problems or too heavyweight to be used for mobile applications. On the other hand, training spiking neural networks (SNNs) with biological plausibility has great potentials of performing fast computation and energy efficiency. However, the lack of effective learning rules for SNNs impedes their wide usage in mobile robot applications. This paper addresses the problem by introducing an end to end learning approach of spiking neural networks for a lane keeping vehicle. We consider the reward-modulated spike-timing-dependent-plasticity (R-STDP) as a promising solution in training SNNs, since it combines the advantages of both reinforcement learning and the well-known STDP. We test our approach in three scenarios that a Pioneer robot is controlled to keep lanes based on an SNN. Specifically, the lane information is encoded by the event data from a neuromorphic vision sensor. The SNN is constructed using R-STDP synapses in an all-to-all fashion. We demonstrate the advantages of our approach in terms of the lateral localization accuracy by comparing with other state-of-the-art learning algorithms based on SNNs.

Dynamic Camera Clusters (DCCs) are multi-camera systems where one or more cameras are mounted on actuated mechanisms such as a gimbal. Existing methods for DCC calibration rely on joint angle measurements to resolve the time-varying transformation between the dynamic and static camera. This information is usually provided by motor encoders, however, joint angle measurements are not always readily available on off-the-shelf mechanisms. In this paper, we present an encoderless approach for DCC calibration which simultaneously estimates the kinematic parameters of the transformation chain as well as the unknown joint angles. We also demonstrate the integration of an encoderless gimbal mechanism with a state-of-the art VIO algorithm, and show the extensions required in order to perform simultaneous online estimation of the joint angles and vehicle localization state. The proposed calibration approach is validated both in simulation and on a physical DCC composed of a 2-DOF gimbal mounted on a UAV. Finally, we show the experimental results of the calibrated mechanism integrated into the OKVIS VIO package, and demonstrate successful online joint angle estimation while maintaining localization accuracy that is comparable to a standard static multi-camera configuration.

We report system and algorithmic development for a sensing suite comprising multiple sensors for both surface and subsurface transportation infrastructure inspection focusing on multi-modal mapping for inspection. The sensing suite contains a camera, a ground penetrating radar (GPR), and a wheel encoder. We design the sensing suite and propose a data collection scheme using customized artificial landmarks (ALs). We use ALs to synchronize two types data streams: camera images that are temporally evenly-spaced and GPR/encoder data that are spatially evenly-spaced. We also employ pose graph optimization with synchronization as penalty functions to further refine synchronization and perform data fusion for 3D reconstruction. We have implemented the system and tested it in physical experiments. The results show that our system successfully fuses three sensory data and product metric 3D reconstruction. The sensor fusion approach reduces the end-to-end distance error from 7.45cm to 3.10cm.

Micro air vehicles exemplified by quadrotors generate downward thrust in their body fixed frame and may only maneuver spatially by changing their orientation. As a result of this underactuation they are fundamentally incapable of simultaneously regulating orientation and position. Furthermore, their feasible maneuvers are limited to spatial trajectories with continuously differentiable acceleration. We present a coaxial helicopter which emulates full actuation over forces and torques (six degrees of freedom) using only two actuators. The orientation of the thrust vector from each rotor is governed by the drive motor by exciting a cyclic flapping response in special articulated blades. The useful separation of orientation and translation dynamics is demonstrated in flight experiments by tracking spatial trajectories while maintaining flat body attitude as well as tracking desired orientations near hover while station keeping.

Surface electromyography is currently the sensing modality of choice for control of biosignal-driven prostheses and exoskeletons; however, the sensor's noisy and aggregate nature inhibits collection of distinguishable signal streams to robustly manipulate multiple device degrees of freedom (DoF). We here explore 2D B-mode ultrasound as an alternative source of muscle activation data (namely, muscle deformation) that can be more precisely localized, allowing for the theoretical collection of multiple naturally-varying signals that could be used to control high-DoF assistive devices. We here present a proof-of-concept study showing a) the observability of muscle deformation via ultrasound, and b) novel descriptions of the spatially-varying nature of the signal. These analyses are accomplished through the study of nine volumetric scans of the biceps brachii under varied elbow angle and loading conditions, collected and spatially localized using an ultrasound scanner and motion capture. We here establish the feasibility of measuring several force-associated deformation signals (including muscle cross-sectional area and thickness) via real-time ultrasound scanning and quantify the spatial variation of these signals. Additionally, we propose future applications for both our signal characterizations and the generated muscle volume data set, including better design of assistive device sensor locations and validation of existing muscle deformation models.

Acoustically expressed emotions can make communication with a robot more efficient. Detecting emotions like anger could provide a clue for the robot indicating unsafe/undesired situations. Recently, several deep neural network-based models have been proposed which establish new state-of-the-art results in affective state evaluation. These models typically start processing at the end of each utterance, which not only requires a mechanism to detect the end of an utterance but also makes it difficult to use them in a real-time communication scenario, e.g. human-robot interaction. We propose the EmoRL model that triggers an emotion classification as soon as it gains enough confidence while listening to a person speaking. As a result, we minimize the need for segmenting the audio signal for classification and achieve lower latency as the audio signal is processed incrementally. The method is competitive with the accuracy of a strong baseline model, while allowing much earlier prediction.

The low-cost manufacturing and maintenance of prostheses is of vital importance to their successful deployment in developing countries. Low-cost prosthesis actuation is generally achieved by combining pre-programmed control strategies, with surface-electromyographic measurements taken from the residual limb. In a standard setting, these signals are measured with disposable gel electrodes. However, this limit on electrode reuse requires that prosthesis users have a stable supply of electrodes. Alternatively, the textile electrodes sewn from conductive thread are studied in the context of hand gesture recognition to consider their future use with low-cost prostheses. In this paper, it is demonstrated that textile electrodes can be applied for gesture recognition. To do so, surface electromyography (sEMG) experiments are run in South Africa on three amputees where they were asked to perform gestures with their phantom limb (i.e., the missing limb segment). A gesture recognition method is implemented, and the classification accuracy with data recorded from textile electrodes is compared to that from gel electrodes. Further analysis examining the relationship between classifier performance and physiological parameters are performed. Results show that textile electrodes can be used to perform accurate gesture recognition, and are comparable to disposable gel electrodes. This demonstrates that low-cost sensory systems are not barrier to myoelectric control in developing countries.

The concept of continuous-time trajectory representation has brought increased accuracy and efficiency to multi-modal sensor fusion in modern SLAM. However, regardless of these advantages, its offline property caused by the requirement of global batch optimization is critically hindering its relevance for real-time and life-long applications. In this paper, we present a dense map-centric SLAM method based on a continuous-time trajectory to cope with this problem. The proposed system locally functions in a similar fashion to conventional Continuous-Time SLAM (CT-SLAM). However, it removes the need for global trajectory optimization by introducing map deformation. The computational complexity of the proposed approach for loop closure does not depend on the operation time, but only on the size of the space it explored before the loop closure. It is therefore more suitable for long term operation compared to the conventional CT-SLAM. Furthermore, the proposed method reduces uncertainty in the reconstructed dense map by using probabilistic surface element (surfel) fusion. We demonstrate that the proposed method produces globally consistent maps without global batch trajectory optimization, and effectively reduces LiDAR noise by surfel fusion.

Magnetically-driven micrometer to millimeter-scale robotic devices have recently shown great capabilities for remote applications in medical procedures, in microfluidic tools and in microfactories. Significant effort recently has been on the creation of mobile or stationary devices with multiple independently-controllable degrees of freedom (DOF) for multiagent or complex mechanism motions. In most applications of magnetic microrobots, however, the relatively large distance from the field generation source and the microscale devices results in controlling magnetic field signals which are applied homogeneously over all agents. While some progress has been made in this area allowing up to six independent DOF to be individually commanded, there has been no rigorous effort in determining the maximum achievable number of DOF for systems with homogeneous magnetic field input. In this work, we show that this maximum is eight and we introduce the theoretical basis for this conclusion, relying on the number of independent usable components in a magnetic field at a point. In order to verify the claim experimentally, we develop a simple demonstration mechanism with 8 DOF designed specifically to show independent actuation. Using this mechanism with 500 μm magnetic elements, we demonstrate eight independent motions of 0.6 mm with 8.6 % coupling using an eight coil system. These results will enable the creation of richer outputs in future microrobotic devices.