IROS 2018 Papers

This paper presents a novel regrasp control policy that makes use of tactile sensing to plan local grasp adjustments. Our approach determines regrasp actions by virtually searching for local transformations of tactile measurements that improve the quality of the grasp. First, we construct a tactile-based grasp quality metric using a deep convolutional neural network trained on over 2800 grasps. The quality of each grasp, a continuous value between 0 and 1, is determined experimentally by measuring its resistance to external perturbations. Second, we simulate the tactile imprints associated with robot motions relative to the initial grasp by performing rigid-body transformations of the given tactile measurements. The newly generated tactile imprints are evaluated with the learned grasp quality network and the regrasp action is chosen to maximize the grasp quality. Results show that the grasp quality network can predict the outcome of grasps with an average accuracy of 85% on known objects and 75% on novel objects. The regrasp control policy improves the success rate of grasp actions by an average relative increase of 70% on a test set of 8 objects. We provide a video summarizing our approach at https://youtu.be/gjn7DmfpwDk.

Here we propose and investigate a novel vibrissal tactile sensor - the Tac Whisker array - based on modifying a 3D-printed optical cutaneous (fingertip) tactile sensor - the TacTip. Two versions are considered: a static Tac Whisker array analogous to immotile tactile vibrissae (e.g. rodent microvib-rissae) and a dynamic Tac Whisker array analogous to motile tactile vibrissae (e.g. rodent macrovibrissae). Performance is assessed on an active object localization task. The whisking motion of the dynamic Tac Whisker leads to millimetre-scale location perception, whereas perception with the static Tac Whisker array is relatively poor when making dabbing contacts. The dynamic sensor output is dominated by a self-generated motion signal, which can be compensated by comparing to a reference signal. Overall, the Tac Whisker arrays give a new class of tactile whiskered robots that benefit from being relatively inexpensive and customizable. Furthermore, the biomimetic basis for the Tac Whiskers fits well with building an embodied model of the rodent sensory system for investigating animal perception.

Temporal object detection has attracted significant attention, but most popular methods can not leverage the rich temporal information in video or robotic vision. Although many different algorithms have been developed for video detection task, real-time online approaches are frequently deficient. In this paper, based on attention mechanism and convolutional long short-term memory (ConvLSTM), we propose a temporal single-shot detector (TSSD)for robotic vision. Distinct from previous methods, we aim to temporally integrate pyramidal feature hierarchy using ConvLSTM, and design a novel structure including a high-level ConvLSTM unit as well as a low-level one (HL-LSTM)for multi-scale feature maps. Moreover, we develop a creative temporal analysis unit, namely, ConvLSTM-based attention and attention-based ConvLSTM (A&CL), in which the ConvLSTM-based attention is specially tailored for background suppression and scale suppression while the attention-based ConvLSTM temporally integrates attention-aware features. Finally, our method is evaluated on ImageNet VID dataset. Extensive comparisons on detection performance confirm the superiority of the proposed approach, and the developed TSSD achieves a considerably enhanced accuracy vs. speed trade-off, i.e., 64.8% mAP vs. 27 FPS.

This paper describes a system identification method and the development of a closed-loop controller for a Hydraulically Amplified Self-healing Electrostatic (HASEL) actuator. Our efforts focus on developing a reliable and consistent way to identify system models for these soft robotic actuators using high-speed videography based motion tracking. Utilizing a mass-spring-damper model we are able to accurately capture the behavior of a HASEL actuator. We use the resulting plant model to design a Proportional-Integral controller that demonstrates improved closed-loop tracking and steady-state error performance.

We present an algorithm for rapidly learning neural network policies for robotics systems. The algorithm follows the model-based reinforcement learning paradigm and improves upon existing algorithms: PILeO and a sample-based version of PILeo with neural network dynamics (Deep-PILeO). To improve convergence, we propose a model-based algorithm that uses fixed random numbers and clips gradients during optimization. We propose training a neural network dynamics model using variational dropout with truncated Log-Normal noise. These improvements enable data-efficient synthesis of complex neural network policies. We test our approach on a variety of benchmark tasks, demonstrating data-efficiency that is competitive with that of PILeO, while being able to optimize complex neural network controllers. Finally, we assess the performance of the algorithm for learning motor controllers for a six legged autonomous underwater vehicle. This demonstrates the potential of the algorithm for scaling up the dimensionality and dataset sizes, in more complex tasks.

We propose a novel interaction strategy for a human-swarm communication when a human operator guides a formation of quadrotors with impedance control and receives vibrotactile feedback. The presented approach takes into account the human hand velocity and changes the formation shape and dynamics accordingly using impedance interlinks simulated between quadrotors, which helps to achieve a life-like swarm behavior. Experimental results with Crazyflie 2.0 quadrotor platform validate the proposed control algorithm. The tactile patterns representing dynamics of the swarm (extension or contraction) are proposed. The user feels the state of the swarm at his fingertips and receives valuable information to improve the controllability of the complex life-like formation. The user study revealed the patterns with high recognition rates. Subjects stated that tactile sensation improves the ability to guide the drone formation and makes the human-swarm communication much more interactive. The proposed technology can potentially have a strong impact on the human-swarm interaction, providing a new level of intuitiveness and immersion into the swarm navigation.

Mobile manipulation tasks are one of the key challenges in the field of search and rescue (SAR) robotics requiring robots with flexible locomotion and manipulation abilities. Since the tasks are mostly unknown in advance, the robot has to adapt to a wide variety of terrains and workspaces during a mission. The centaur-like robot Centauro has a hybrid legged-wheeled base and an anthropomorphic upper body to carry out complex tasks in environments too dangerous for humans. Due to its high number of degrees of freedom, controlling the robot with direct teleoperation approaches is challenging and exhausting. Supervised autonomy approaches are promising to increase quality and speed of control while keeping the flexibility to solve unknown tasks. We developed a set of operator assistance functionalities with different levels of autonomy to control the robot for challenging locomotion and manipulation tasks. The integrated system was evaluated in disaster response scenarios and showed promising performance.

Recent progress in 3D sensor devices and in semantic mapping allows to build very rich HD 3D maps very useful for autonomous navigation and localization. However, these maps are particularly huge and require important memory capabilities as well computational resources. In this paper, we propose a new method for summarizing a 3D map (Mesh)as a set of compact spheres in order to facilitate its use by systems with limited resources (smartphones, robots, UAVs,...). This vision-based summarizing process is applied in a fully automatic way using jointly photometric, geometric and semantic information of the studied environment. The main contribution of this research is to provide a very compact map that maximizes the significance of its content while maintaining the full visibility of the environment. Experimental results in summarizing large-scale 3D map demonstrate the feasibility of our approach and evaluate the performance of the algorithm.

For VSLAM (Visual Simultaneous Localization and Mapping), localization is a challenging task, especially for some challenging situations: textureless frames, motion blur, etc. To build a robust exploration and localization system in a given space, a submap-based VSLAM system is proposed in this paper. Our system uses a submap back-end and a visual front-end. The main advantage of our system is its robustness with respect to tracking failure, a common problem in current VSLAM algorithms. The robustness of our system is compared with the state-of-the-art in terms of average tracking percentage. The precision of our system is also evaluated in terms of ATE (absolute trajectory error) RMSE (root mean square error) comparing the state-of-the-art. The ability of our system in solving the “kidnapped” problem is demonstrated. Our system can improve the robustness of visual localization in challenging situations.

The model produced by 3D reconstruction algorithm is usually represented by voxels. The management of these voxels is usually divided into two categories: ordered and unordered methods. The ordered method holds too many empty voxels to maintain data order which leads to a low storage efficiency. On the contrary, the unordered method keeps massive index data to only store nonempty voxels. In this paper, we design a new data management method for real-time indoor 3D reconstruction, called Structured Skip List (SSL). The SSL can be treated as a semi-ordered method, because the advantages of both the ordered and unordered methods are taken into account: 1) it only holds nonempty voxels similar to the unordered method; 2) the structured information is introduced to reduce the storage space of index data. By these designs, the SSL has a better performance on storage efficiency. To handle the data collision in voxel allocation, a hash allocation list (HAL) is proposed. The length of each Skip List is kept balanced by fusing the IMU (Inertial Measurement Unit) information for a high operation efficiency. The storage efficiency analysis of different data management methods is shown in this paper. What's more, exhaustive investigation is carried out on several datasets with these methods. The experimental result demonstrates that our design can achieve a high storage efficiency with little time loss compared to the state-of-the-art methods.

This paper proposes an integration of multi-body and actuator control for multi-contact balancing for robots with highly elastic joints. Inspired by the structure preserving control concept for series-elastic fixed-base robots, the presented approach aims to minimize the control effort by keeping the system structure intact. Balancing on multiple contacts requires to solve the force distribution problem. In locomotion, contacts change quickly, requiring a swift redistribution of contact forces. This is a challenge for elastic robots as the actuator dynamics and limits prevent instantaneous changes of contact forces. The proposed dynamically consistent force distribution is implemented as a model predictive controller which resolves redundancy while complying with contact force and actuator constraints.

This paper describes a system to map a ground-plane, and to subsequently use the map for localization of a mobile robot. The robot has a downward-facing camera, and works on a variety of ground textures including general texture like tarmac, man-made designs like carpet, and rectilinear textures like indoor tiles or outdoor slabs. Such textures provide a basis for measuring relative motion (i.e. computer mouse functionality). But the goal here is the more challenging one of absolute localization. The paper describes a complete working pipeline to build a globally consistent map of a given ground-plane and subsequently to localize within this map at real-time. Two algorithms are described. The first is a feature-based approach which is general to any ground plane texture. The second algorithm takes advantage of the extra constraints available for common rectilinear textures like indoor tiling, paving slabs, and laid brickwork. Quantitative and qualitative experimental results are shown for mapping and localization on a variety of ground-planes.

In challenging environments where human intervention is expensive, robust and persistent autonomy is a key requirement. AI Planners can efficiently construct plans to achieve this long-term autonomous behaviour. However, in plans which are expected to last over days, or even weeks, the size of the state-space becomes too large for current planners to solve as a single problem. These problems are well-suited to decomposition and abstraction planning techniques. We present a novel approach in the context of persistent autonomy in autonomous underwater vehicles, in which tasks are complex and diverse and plans cannot be precomputed. Our approach performs a decomposition into a two-level hierarchical structure, which dynamically constructs planning problems at the upper level of the hierarchy using solution plans from the lower level. Solution plans are then executed and monitored simultaneously at both levels. We evaluate the approach, showing that compared to strictly top-down hierarchical decompositions, our approach leads to more robust solution plans of higher quality.

In this paper, we present a multi-link articulated wheeled in-pipe robot that can drive the wheel and roll joint by using only a single actuator installed in each link. The proposed mechanism enables the robot to move forward or backward and helically in pipes owing to rotation of the drive wheel and twisting of the body. These two movements are generated by a miter-geared differential mechanism installed in each joint, and the magnitudes of these movements depend on the load applied to the wheels and roll joints. However, controlling of two outputs independently and aligning the rotation of the roll joints as desired are extremely challenging. Therefore, in this study, we switch those two movements by driving the rear wheels and the front wheels of the robot alternately. In addition, a stopper is used to constrain the roll joint movement. By calculating the angle of elevation of the robot's helical movement in the pipe by using a kinematic model, we can design a stopper to precisely adjust the roll angle. We verified that the robot can twist using the differential mechanism, and we validated experimentally the effectiveness of the stopper.

In this work, we present a Stochastic Gradient Ascent (SGA) algorithm for multi-vehicle information gathering that accounts for limitations on a vehicle's control authority caused by external forces. By representing vehicle paths using a novel action space representation, rather than a state space representation, we remove the need to perform feasibility calculations on the vehicle's path. Our algorithm uses a stochastic optimization scheme by sampling perturbed action sequences around the current best known sequence to estimate the gradient of a state space information function with respect to the action sequence. Additionally, we use sequential greedy allocation to plan for multiple vehicles. Results are shown using a Navy Coastal Ocean Model (NCOM) for the Gulf of Mexico (GoM). SGA shows improvement in the amount of information gained over a greedy baseline. Additionally, we compare to Monte Carlo Tree Search (MCTS) Method, which is able to gather competitive amounts of information but is more computationally intensive than our approach.

In this paper we propose a particle filter localization approach, based on stereo visual odometry (VO) and semantic information from indoor environments, for mini-aerial robots. The prediction stage of the particle filter is performed using the 3D pose of the aerial robot estimated by the stereo VO algorithm. This predicted 3D pose is updated using inertial as well as semantic measurements. The algorithm processes semantic measurements in two phases; firstly, a pre-trained deep learning (DL) based object detector is used for real time object detections in the RGB spectrum. Secondly, from the corresponding 3D point clouds of the detected objects, we segment their dominant horizontal plane and estimate their relative position, also augmenting a prior map with new detections. The augmented map is then used in order to obtain a drift free pose estimate of the aerial robot. We validate our approach in several real flight experiments where we compare it against ground truth and a state of the art visual SLAM approach.

As simultaneous localization and mapping (SLAM) techniques have flourished with the advent of 3D Light Detection and Ranging (LiDAR) sensors, accurate 3D maps are readily available. Many researchers turn their attention to localization in a previously acquired 3D map. In this paper, we propose a novel and lightweight camera-only visual positioning algorithm that involves localization within prior 3D LiDAR maps. We aim to achieve the consumer level global positioning system (GPS) accuracy using vision within the urban environment, where GPS signal is unreliable. Via exploiting a stereo camera, depth from the stereo disparity map is matched with 3D LiDAR maps. A full six degree of freedom (DOF) camera pose is estimated via minimizing depth residual. Powered by visual tracking that provides a good initial guess for the localization, the proposed depth residual is successfully applied for camera pose estimation. Our method runs online, as the average localization error is comparable to ones resulting from state-of-the-art approaches. We validate the proposed method as a stand-alone localizer using KITTI dataset and as a module in the SLAM framework using our own dataset.

This paper analyzes and implements two novel turning strategies for underactuated legged robots that leverage contact of an active tail against terrain. The first strategy produces a sustained turn with a tail dragging against the ground during forward locomotion. The second strategy produces a rapid point turn by impacting the tail against the ground. LoadRoACH, a 55 g palm-sized legged robot, is developed to carry the active tail payload used in turning experiments. A steady-state turning model predicts the achievable turn speed of the robot on carpet, and open-loop turning experiments characterize the performance of the two tail contact turning strategies. Tail drag turning provides comparable turning maneuverability to differential drive turning gaits on carpet and gravel surfaces. Tail impact turning can produce rapid point turns on carpet, tarp, and gravel, but has a large variability in turn angle and time to recover from the turn. Finally, tail drag and tail impact turning control methods are implemented in an aggressive closed-loop corner steering maneuver.

This paper proposes a novel steerable locomotion controller for six-strut tensegrity robots. Tensegrity robots are lightweight and have many promising features such as robustness, shape-shifting capabilities, and deployability, making them good candidates for exploration and scouting of remote areas. Despite these advantages, tensegrity robots are challenging to control due to their large number of degrees of freedom, nonlinear dynamics, and intrinsic compliance. Recently, many step-wise motion controllers have been employed to simplify the locomotion problem, thanks to the discrete nature of the tensegrity structure. In this paper we present a novel locomotion controller which will steer the direction of motion of a six-strut tensegrity robot when used in conjunction with any preexisting step-wise controller. We validated our controller on the SUPERball v2 robot, showing straight and curved trajectories, and an example of navigation around obstacles. Our method is computationally inexpensive, only requires knowledge about the current base triangle (e.g, via accelerometer data), and can be generalized to any six-strut tensegrity robot which can perform step-wise locomotion.