IROS 2020 Papers

Although research indicates that telepresence robots offer a more socially telepresent alternative to conventional forms of remote communication, the lack of touch-based interactions presents challenges for both remote and local users. In order to address these challenges, we have designed and implemented a robotic manipulator emulating a human arm. However, contact interactions like handshakes with a robotic manipulator may feel awkward and unnatural to local users. In this work, we present the design of a wearable haptic measurement glove (HMG) and use it to collect force and inertial data on handshakes in human-human and humanrobot interactions in the interest of developing intelligent shared control algorithms for natural, human-like contact interactions in human-robot interactions.

In this paper we present the LoCO AUV, a Low-Cost, Open Autonomous Underwater Vehicle. LoCO is a general-purpose, single-person-deployable, vision-guided AUV, rated to a depth of 100 meters. We discuss the open and expandable design of this underwater robot, as well as the design of a simulator in Gazebo. Additionally, we explore the platform's preliminary local motion control and state estimation abilities, which enable it to perform maneuvers autonomously. In order to demonstrate its usefulness for a variety of tasks, we implement a variety of our previously presented human-robot interaction capabilities on LoCO, including gestural control, diver following, and robot communication via motion. Finally, we discuss the practical concerns of deployment and our experiences in using this robot in pools, lakes, and the ocean. All design details, instructions on assembly, and code will be released under a permissive, open-source license.

Grasping and manipulating objects is an integral part of many robotic systems. Both soft and rigid grippers have been investigated for manipulating objects in a multitude of different roles. Rigid grippers can hold heavy objects and apply large amounts of force, while soft grippers can conform to the size and shape of objects as well as protect fragile objects from excess stress. However, grippers that possess the qualities of both rigid and soft grippers are under-explored. In this paper, we present a novel gripper with two distinct properties: 1) it can vary its stiffness to become either a soft gripper that can conform its shape to fit complex objects or a rigid gripper that can hold a large weight; 2) when the gripper is soft, it has two stable states (i.e., bistable): open and closed: allowing it to be closed without an actuator but through contact force with a target object. The variable stiffness is accomplished by heating a shape memory polymer (SMP) material through its glass transition temperature. The bi-stability is achieved by shaping the gripper's energy landscape through two elastic elements. This paper details the design and fabrication process of this gripper, as well as quantifies the influence of temperature variations on this gripper. The capability of the gripper is experimentally verified by grasping different objects with various shapes and weights. We expect such a gripper to be suitable for many applications that traditionally require either a rigid or a soft gripper.

With a growing number of applications in the world for UAVs, there is a clear limitation regarding the need for extended battery life. With the current flight times, many users would benefit greatly with an innovative option of field charging these devices. The objective of this project is to investigate feasibility of inductively harvesting energy from a power line cable for applications such as charging a UAV drone. Research investigates a dual hook perching device that securely attaches to a power cable and aligns an inductive core with the cable for harvesting energy from its electro-magnetic field. Modeling and analysis of the core highlights critical design parameters, leading to evaluation of circular, semi-cylindrical, and u-shaped prototypes designed to interface with a 1" power cable. Underactuated two jaw manipulators at each end of the coil are proposed for grasping the cable and aligning it with the charging coil, ultimately providing a firm grasp and perch. An open source hexacopter drone was used in this study to integrate with the charging novelty. The results provided can be used as a starting point to study the reliability of this method of charging and to further investigate perching abilities of UAVs.

This paper presents the design and control of a novel quadrotor with a variable geometry to physically interact with cluttered environments and fly through narrow gaps and passageways. This compliant quadrotor with passive morphing capabilities is designed using torsional springs at every arm hinge to allow for rotation driven by external forces. We derive the dynamic model of this variable geometry quadrotor (SQUEEZE), and develop an adaptive controller for trajectory tracking. The corresponding Lyapunov stability proof of attitude tracking is also presented. Further, an admittance controller is designed to account for changes in yaw due to physical interactions with the environment. Finally, the proposed design is validated in flight tests with two setups: a small gap and a passageway. The experimental results demonstrate the unique capability of the SQUEEZE in navigating through constrained narrow spaces.

Most state-of-the-art bipedal robots are designed to be anthropomorphic and therefore possess legs with knees. Whilst this facilitates more human-like locomotion, there are implementation issues that make walking with straight or near-straight legs difficult. Most bipedal robots have to move with a constant bend in the legs to avoid singularities at the knee joints, and to keep the centre of mass at a constant height for control purposes. Furthermore, having a knee on the leg increases the design complexity as well as the weight of the leg, hindering the robot’s performance in agile behaviours such as running and jumping.We present SLIDER, an ultra-lightweight, low-cost bipedal walking robot with a novel knee-less leg design. This non-anthropomorphic straight-legged design reduces the weight of the legs significantly whilst keeping the same functionality as anthropomorphic legs. Simulation results show that SLIDER’s low-inertia legs contribute to less vertical motion in the center of mass (CoM) than anthropomorphic robots during walking, indicating that SLIDER’s model is closer to the widely used Inverted Pendulum (IP) model. Finally, stable walking on flat terrain is demonstrated both in simulation and in the physical world, and feedback control is implemented to address challenges with the physical robot.

The ability to perform in-hand manipulation still remains an unsolved problem; having this capability would allow robots to perform sophisticated tasks requiring repositioning and reorienting of grasped objects. In this work, we present a novel non-anthropomorphic robot grasper with the ability to manipulate objects by means of active surfaces at the fingertips. Active surfaces are achieved by spherical rolling fingertips with two degrees of freedom (DoF) - a pivoting motion for surface reorientation - and a continuous rolling motion for moving the object. A further DoF is in the base of each finger, allowing the fingers to grasp objects over a range of size and shapes. Instantaneous kinematics was derived and objects were successfully manipulated both with a custom handcrafted control scheme as well as one learned through imitation learning, in simulation and experimentally on the hardware.

In this paper, we explore the possibility of achieving a more accurate depth estimation by fusing monocular images and Radar points using a deep neural network. We give a comprehensive study of the fusion between RGB images and Radar measurements from different aspects and proposed a working solution based on the observations. We find that the noise existing in Radar measurements is one of the main key reasons that prevents one from applying the existing fusion methods developed for LiDAR data and images to the new fusion problem between Radar data and images. The experiments are conducted on the nuScenes dataset, which is one of the first datasets which features Camera, Radar, and LiDAR recordings in diverse scenes and weather conditions. Extensive experiments demonstrate that our method outperforms existing fusion methods. We also provide detailed ablation studies to show the effectiveness of each component in our method.

Modern high-definition LIDAR is expensive for commercial autonomous driving vehicles and small indoor robots. An affordable solution to this problem is fusion of planar LIDAR with RGB images to provide a similar level of perception capability. Even though state-of-the-art methods provide approaches to predict depth information from limited sensor input, they are usually a simple concatenation of sparse LIDAR features and dense RGB features through an end-to-end fusion architecture. In this paper, we introduce an inductive late-fusion block which better fuses different sensor modalities inspired by a probability model. The proposed demonstration and aggregation network propagates the mixed context and depth features to the prediction network and serves as a prior knowledge of the depth completion. This late-fusion block uses the dense context features to guide the depth prediction based on demonstrations by sparse depth features. In addition to evaluating the proposed method on benchmark depth completion datasets including NYUDepthV2 and KITTI, we also test the proposed method on a simulated planar LIDAR dataset. Our method shows promising results compared to previous approaches on both the benchmark datasets and simulated dataset with various 3D densities.

With the rapidly developing unmanned aerial vehicles, the requirements of generating maps efficiently and quickly are increasing. To realize online mapping, we develop a real-time dense mapping framework named DenseFusion which can incrementally generates dense geo-referenced 3D point cloud, digital orthophoto map (DOM) and digital surface model (DSM) from sequential aerial images with optional GPS information. The proposed method works in real-time on standard CPUs even for processing high resolution images. Based on the advanced monocular SLAM, our system first estimates appropriate camera poses and extracts effective keyframes, and next constructs virtual stereo-pair from consecutive frame to generate pruned dense 3D point clouds; then a novel realtime DSM fusion method is proposed which can incrementally process dense point cloud. Finally, a high efficiency visualization system is developed to adopt dynamic levels of detail (LoD) method, which makes it render dense point cloud and DSM smoothly. The performance of the proposed method is evaluated through qualitative and quantitative experiments. The results indicate that compared to traditional structure from motion based approaches, the presented framework is able to output both large-scale high-quality DOM and DSM in real-time with low computational cost.

We develop an online probabilistic metric-semantic mapping approach for autonomous robots relying on streaming RGB-D observations. We cast this problem as a Bayesian inference task, requiring encoding both the geometric surfaces and semantic labels (e.g., chair, table, wall) of the unknown environment. We propose an online Gaussian Process (GP) training and inference approach, which avoids the complexity of GP classification by regressing a truncated signed distance function representation of the regions occupied by different semantic classes. Online regression is enabled through sparse GP approximation, compressing the training data to a finite set of inducing points, and through spatial domain partitioning into an Octree data structure with overlapping leaves. Our experiments demonstrate the effectiveness of this technique for large-scale probabilistic metric-semantic mapping of 3D environments. A distinguishing feature of our approach is that the generated maps contain full continuous distributional information about the geometric surfaces and semantic labels, making them appropriate for uncertainty-aware planning.

This article introduces a decentralized multi-robot algorithm for Simultaneous Localization And Mapping (SLAM) inspired from previous work on collaborative mapping [1]. This method makes robots jointly build and exchange i) a collection of 3D dense locally consistent submaps, based on a Truncated Signed Distance Field (TSDF) representation of the environment, and ii) a pose-graph representation which encodes the relative pose constraints between the TSDF submaps and the trajectory keyframes, derived from the odometry, inter-robot observations and loop closures. Such loop closures are spotted by aligning and fusing the TSDF submaps. The performances of this method have been evaluated on multi-robot scenarios built from the EuRoC dataset [2].

Underwater roll rotation is a basic but essential maneuver that allows many biological swimmers to achieve high maneuverability and complex locomotion patterns. In particular, sea mammals (e.g., sea otter) with flexible vertebra structures have a unique mechanism to efficiently achieve roll rotation, not propelled mainly by inter-digital webbing or fin, but by bending and twisting their body.In this work, we attempt to implement and effectively control the roll rotation by mimicking this kind of efficient biomorphic roll mechanism on our two degrees of freedom (DOF) soft modular swimming robot. The robot also allows the achievement of other common maneuvers, such as pitch/yaw rotation and linear swimming patterns. The proposed 2DOF soft swimming robot platform includes an underactuated, cable-driven design that mimics the flexible cascaded skeletal structure of soft spine tissue and hard spine bone seen in many fish species. The cable-driven actuation mechanism is oriented laterally for forwarding motion and steering in a 3D plane. The robot can perform a steady and controllable roll rotation with a maximum angular speed of 41.6 deg/s. A hypothesis explaining this novel roll rotation mechanism is set forth, and the phenomenon is systematically studied at different frequencies and phase lag gait conditions. Preliminary results show a linear relationship between roll angular velocity and frequency within a specific range. Additionally, the roll rotation can be controlled independently in some special conditions. These abilities form the foundation for future research on 3D underwater locomotion with adaptive, controllable maneuvering capabilities.

In this paper, we propose a real-time deep learning approach for determining the 6D relative pose of Autonomous Underwater Vehicles (AUV) from a single image. A team of autonomous robots localizing themselves in a communication-constrained underwater environment is essential for many applications such as underwater exploration, mapping, multi-robot convoying, and other multi-robot tasks. Due to the profound difficulty of collecting ground truth images with accurate 6D poses underwater, this work utilizes rendered images from the Unreal Game Engine simulation for training. An image-to-image translation network is employed to bridge the gap between the rendered and the real images producing synthetic images for training. The proposed method predicts the 6D pose of an AUV from a single image as 2D image keypoints representing 8 corners of the 3D model of the AUV, and then the 6D pose in the camera coordinates is determined using RANSAC-based PnP. Experimental results in real-world underwater environments (swimming pool and ocean) with different cameras demonstrate the robustness and accuracy of the proposed technique in terms of translation error and orientation error over the state-of-the-art methods. The code is publicly available.

We present a novel algorithm (DeepMNavigate) for global multi-agent navigation in dense scenarios using deep reinforcement learning (DRL). Our approach uses local and global information for each robot from motion information maps. We use a three-layer CNN that takes these maps as input to generate a suitable action to drive each robot to its goal position. Our approach is general, learns an optimal policy using a multi-scenario, multi-state training algorithm, and can directly handle raw sensor measurements for local observations. We demonstrate the performance on dense, complex benchmarks with narrow passages and environments with tens of agents. We highlight the algorithm’s benefits over prior learning methods and geometric decentralized algorithms in complex scenarios.

Scene flow is the dense 3D reconstruction of motion and geometry of a scene. Most state-of-the-art methods use a pair of stereo images as input for full scene reconstruction. These methods depend a lot on the quality of the RGB images and perform poorly in regions with reflective objects, shadows, ill-conditioned light environment and so on. LiDAR measurements are much less sensitive to the aforementioned conditions but LiDAR features are in general unsuitable for matching tasks due to their sparse nature. Hence, using both LiDAR and RGB can potentially overcome the individual disadvantages of each sensor by mutual improvement and yield robust features which can improve the matching process. In this paper, we present DeepLiDARFlow, a novel deep learning architecture which fuses high level RGB and LiDAR features at multiple scales in a monocular setup to predict dense scene flow. Its performance is much better in the critical regions where image-only and LiDAR-only methods are inaccurate. We verify our DeepLiDARFlow using the established data sets KITTI and FlyingThings3D and we show strong robustness compared to several state-of-the-art methods which used other input modalities. The code of our paper is available at https://github.com/dfki-av/DeepLiDARFlow.

Tactile sensing is inherently contact based. To use tactile data, robots need to make contact with the surface of an object. This is inefficient in applications where an agent needs to make a decision between multiple alternatives that depend the physical properties of the contact location. We propose a method to get tactile data in a non-invasive manner. The proposed method estimates the output of a tactile sensor from the depth data of the surface of the object based on past experiences. An experience dataset is built by allowing the robot to interact with various objects, collecting tactile data and the corresponding object surface depth data. We use the experience dataset to train a neural network to estimate the tactile output from depth data alone. We use GelSight tactile sensors, an image-based sensor, to generate images that capture detailed surface features at the contact location. We train a network with a dataset containing 578 tactile-image to depth- map correspondences. Given a depth-map of the surface of an object, the network outputs an estimate of the response of the tactile sensor, should it make a contact with the object. We evaluate the method with structural similarity index matrix (SSIM), a similarity metric between two images commonly used in image processing community. We present experimental results that show the proposed method outperforms a baseline that uses random images with statistical significance getting an SSIM score of 0.84 ± 0.0056 and 0.80 ± 0.0036, respectively.

Connector insertion and many other tasks commonly found in modern manufacturing settings involve complex contact dynamics and friction. Since it is difficult to capture related physical effects with first-order modeling, traditional control methods often result in brittle and inaccurate controllers, which have to be manually tuned. Reinforcement learning (RL) methods have been demonstrated to be capable of learning controllers in such environments from autonomous interaction with the environment, but running RL algorithms in the real world poses sample efficiency and safety challenges. Moreover, in practical real-world settings, we cannot assume access to perfect state information or dense reward signals. In this paper, we consider a variety of difficult industrial insertion tasks with visual inputs and different natural reward specifications, namely sparse rewards and goal images. We show that methods that combine RL with prior information, such as classical controllers or demonstrations, can solve these tasks from a reasonable amount of real-world interaction.

Coverage path planning is a well-studied problem in robotics in which a robot must plan a path that passes through every point in a given area repeatedly, usually with a uniform frequency. To address the scenario in which some points need to be visited more frequently than others, this problem has been extended to non-uniform coverage planning. This paper considers the variant of non-uniform coverage in which the robot does not know the distribution of relevant events beforehand and must nevertheless learn to maximize the rate of detecting events of interest. This continual area sweeping problem has been previously formalized in a way that makes strong assumptions about the environment, and to date only a greedy approach has been proposed. We generalize the continual area sweeping formulation to include fewer environmental constraints, and propose a novel approach based on reinforcement learning in a Semi-Markov Decision Process. This approach is evaluated in an abstract simulation and in a high fidelity Gazebo simulation. These evaluations show significant improvement upon the existing approach in general settings, which is especially relevant in the growing area of service robotics. We also present a video demonstration on a real service robot.