IROS 2020 论文列表

Tumbling locomotion allows for small robots to traverse comparatively rough terrain, however, their motion is complex and difficult to control. Existing tumbling robot control methods involve manual control or the assumption of at terrain. Reinforcement learning allows for the exploration and exploitation of diverse environments. By utilizing reinforcement learning with domain randomization, a robust control policy can be learned in simulation then transferred to the real world. In this paper, we demonstrate autonomous setpoint navigation with a tumbling robot prototype on at and non- at terrain. The flexibility of this system improves the viability of nontraditional robots for navigational tasks.

In this paper, we propose a novel sim-to-real framework to solve bolting tasks with tight tolerance and complex contact geometry which are hard to be modeled. The sim-to-real has desirable features in terms of cost and safety, however, that of the assembly task is rare due to the lack of simulator, which can robustly render multi-contact assembly. We implement the sim-to-real transfer of nut tightening policy which is adaptive to uncertain bolt positions. This can be realized through developing a novel contact model, which is fast and robust to complex assembly geometry, and novel hierarchical controller with reinforcement learning (RL), which can perform the tasks with a narrow and complicated path. The fast and robust contact model is achieved by utilizing configuration space abstraction and passive midpoint integrator (PMI), which render the simulator robust even in a high stiffness contact condition. And we use sampling-based motion planning to construct a path library and design linear quadratic tracking controller as a low-level controller to be compliant and avoid local optima. Additionally, we use the RL agent as a high-level controller to make it possible to adapt to the bolt position uncertainty, thereby realizing sim-to-real. Experiments are performed to verify our proposed sim-to-real framework.

Thin and imperceptible soft skins that can detect internal deformations as well as external forces, can go a long way to address perception and control challenges in soft robots. However, decoupling proprioceptive and exteroceptive stimuli is a challenging task. In this paper, we present a silicone-based, capacitive E-skin for exteroception and proprioception (SCEEP). This soft and stretchable sensor can perceive stretch as along with touch at 100 different points via its 100 tactels. In this paper, we present a novel algorithm that decouples global strain from local indentations due to external forces. The soft skin is 10.1cm in length and 10cm in width and can be used to accurately measure the global strain of up to 25% with an error of under 3%; while at the same time, can determine the amplitude and position of local indentations. This is a step towards a fully soft electronic skin that can act as a proprioceptive sensor to measure internal states while measuring external forces.

In this paper, we introduce a new large-scale sidewalk dataset called SideGuide that could potentially help impaired people. Unlike most previous datasets, which are focused on road environments, we paid attention to sidewalks, where understanding the environment could provide the potential for improved walking of humans, especially impaired people. Concretely, we interviewed impaired people and carefully selected target objects from the interviewees' feedback (objects they encounter on sidewalks). We then acquired two different types of data: crowd-sourced data and stereo data. We labeled target objects at instance-level (i.e., bounding box and polygon mask) and generated a ground-truth disparity map for the stereo data. SideGuide consists of 350K images with bounding box annotation, 100K images with a polygon mask, and 180K stereo pairs with the ground-truth disparity. We analyzed our dataset by performing baseline analysis for object detection, instance segmentation, and stereo matching tasks. In addition, we developed a prototype that recognizes the target objects and measures distances, which could potentially assist people with disabilities. The prototype suggests the possibility of practical application of our dataset in real life.

CCD camera-based tactile sensors provide high-resolution information about the deformation of soft and elastic interfaces. However, they have poor scalibility as it is difficult to sense a large surface area without increasing the distance between the camera and the interface or using multiple processing chips. For example, using such tactile sensors for a whole robotic arm is not yet possible. In this work, we demonstrate a data driven method that can reconstruct the high-resolution information about deformation of the soft interface while keeping the space requirements and power consumption relatively low. Our modified tactile sensor incorporates two independent sensing techniques, one low- and one high-resolution, and we learn to map to the latter from the former. As a low-resolution sensor, we use liquid-filled channels that transmit the information from the location of the tactile interaction to a rigid display, where the liquid displacements are tracked by a CCD camera. Simultaneously, the same interaction is measured by tracking the markers on the bottom of the sensor using a second CCD camera. After data collection, we train two different machine learning models to reconstruct the time series of the high-resolution sensor. By training a convolutional autoencoder (CAE) and attaching it to the recurrent neural network (RNN), we demonstrate the reconstruction of high-resolution video frames using only the time series of the low-resolution sensor.

To fuse information from a 3D Light Detection and Ranging (LiDAR) sensor and a camera, the extrinsic transformation between the sensor coordinate systems needs to be known. Therefore, an extrinsic calibration must be performed, which is usually based on features extracted from sensor data. Naturally, sensor errors can affect the feature extraction process, and thus distort the calibration result. Unlike previous works, which do not consider the uncertainties of the sensors, we propose a set-membership approach that takes all sensor errors into account. Since the actual error distribution of off- the-shelf sensors is often unknown, we assume to only know bounds (or intervals) enclosing the sensor errors and accordingly introduce novel error models for both sensors. Next, we introduce interval-based approaches to extract corresponding features from images and point clouds. Due to the unknown but bounded sensor errors, we cannot determine the features exactly, but compute intervals guaranteed to enclose them. Subsequently, these feature intervals enable us to formulate a Constraint Satisfaction Problem (CSP). Finally, the CSP is solved to find a set of solutions that is guaranteed to contain the true solution and simultaneously reflects the accuracy of the calibration. Experiments using simulated and real data validate our approach and show its advantages over existing methods.

In this paper, we present a sequential motion planning and control method for generating somersaults on bipedal robots. The somersault (backflip or frontflip) is considered as a coupling between an axile hopping motion and a rotational motion about the center of mass of the robot; these are encoded by a hopping Spring-loaded Inverted Pendulum (SLIP) model and the rotation of a Flywheel, respectively. We thus present the Flywheel SLIP model for generating the desired motion on the ground phase. In the flight phase, we present a momentum transmission method to adjust the orientation of the lower body based on the conservation of the centroidal momentum. The generated motion plans are realized on the full-dimensional robot via momentum-included task space control. Finally, the proposed method is implemented on a modified version of the bipedal robot Cassie in simulation wherein multiple somersault motions are generated.

Human beings and animals are capable of recognizing places from a previous journey when viewing them under different environmental conditions (e.g., illuminations and weathers). This paper seeks to provide robots with a human-like place recognition ability using a new point cloud feature learning method. This is a challenging problem due to the difficulty of extracting invariant local descriptors from the same place under various orientation differences and dynamic obstacles. In this paper, we propose a novel lightweight 3D place recognition method, SeqSphereVLAD, which is capable of recognizing places from a previous trajectory regardless of the viewpoint and the temporary observation differences. The major contributions of our method lie in two modules: (1) the spherical convolution feature extraction module, which produces orientation-invariant local place descriptors, and (2) the coarse-to-fine sequence matching module, which ensures both accurate loop-closure detection and real-time performance. Despite the apparent simplicity, our proposed approach outperform the state-of-the-arts for place recognition under datasets that combine orientation and context differences. Compared with the arts, our method can achieve above 95% average recall for the best match with only 18% inference time of PointNet-based place recognition methods.

So far, multiple LCD monitors and joysticks have been used for remote operation of excavators while it has low work efficiency. This is because it is difficult for the operator to recognize the state of the excavator and its surrounding environment. We have developed a semi-autonomous control system which consists of autonomy (attractor based dynamical system) and human action (admittance control). On the other hand, excavation tasks require the task selection. In this paper, we propose a nonlinear dynamical system with attractor with stagnation and bifurcation. The stagnation of the attractor is designed as a negative divergence vector field that converges to a point on the trajectory. A stagnation is placed at a bifurcation point of the trajectory, and the operator selects the next task by adding a force to the leader system.

There has been a considerable amount of recent work on high-speed micro-aerial vehicle flight in unknown and unstructured environments. Generally these approaches either use active sensing or fly slowly enough to ensure a safe braking distance with the relatively short sensing range of passive sensors. The former generally requires carrying large and heavy LIDARs and the latter only allows flight far away from the dynamic limits of the vehicle. One of the significant challenges for high-speed flight is the computational demand of trajectory planning at sufficiently high rates and length scales required in outdoor environments. We tackle both problems in this work by leveraging semantic information derived from an RGB camera on-board the vehicle. We first describe how to use semantic information to increase the effective range of perception on certain environment classes. Second, we present a sparse representation of the environment that is sufficiently lightweight for long distance path planning. We show how our approach outperforms more traditional metric planners which seek the shortest path, demonstrate the semantic planner's capabilities in a set of simulated and excessive real-world autonomous quadrotor flights in an urban environment.

In this paper, we present the first large-scale dataset for semantic Segmentation of Underwater IMagery (SUIM). It contains over 1500 images with pixel annotations for eight object categories: fish (vertebrates), reefs (invertebrates), aquatic plants, wrecks/ruins, human divers, robots, and sea-floor. The images have been rigorously collected during oceanic explorations and human-robot collaborative experiments, and annotated by human participants. We also present a comprehensive benchmark evaluation of several state-of-the-art semantic segmentation approaches based on standard performance metrics. Additionally, we present SUIM-Net, a fully-convolutional deep residual model that balances the trade-off between performance and computational efficiency. It offers competitive performance while ensuring fast end-to-end inference, which is essential for its use in the autonomy pipeline by visually-guided underwater robots. In particular, we demonstrate its usability benefits for visual servoing, saliency prediction, and detailed scene understanding. With a variety of use cases, the proposed model and benchmark dataset open up promising opportunities for future research in underwater robot vision.

Due to the difficulty in generating the effective descriptors which are robust to occlusion and viewpoint changes, place recognition for 3D point cloud remains an open issue. Unlike most of the existing methods that focus on extracting local, global, and statistical features of raw point clouds, our method aims at the semantic level that can be superior in terms of robustness to environmental changes. Inspired by the perspective of humans, who recognize scenes through identifying semantic objects and capturing their relations, this paper presents a novel semantic graph based approach for place recognition. First, we propose a novel semantic graph representation for the point cloud scenes by reserving the semantic and topological information of the raw point cloud. Thus, place recognition is modeled as a graph matching problem. Then we design a fast and effective graph similarity network to compute the similarity. Exhaustive evaluations on the KITTI dataset show that our approach is robust to the occlusion as well as viewpoint changes and outperforms the state-of-the-art methods with a large margin. Our code is available at: https://github.com/kxhit/SG_PR.

A physical selfie stick extends the user's reach, enabling the acquisition of personal photos that include more of the background scene. Similarly, a quadcopter can capture photos from vantage points unattainable by the user; but teleoperating a quadcopter to good viewpoints is a difficult task. This paper presents a natural interface for quadcopter photography, the SelfieDroneStick that allows the user to guide the quadcopter to the optimal vantage point based on the phone's sensors. Users specify the composition of their desired long-range selfies using their smartphone, and the quadcopter autonomously flies to a sequence of vantage points from where the desired shots can be taken. The robot controller is trained from a combination of real-world images and simulated flight data. This paper describes two key innovations required to deploy deep reinforcement learning models on a real robot: 1) an abstract state representation for transferring learning from simulation to the hardware platform, and 2) reward shaping and staging paradigms for training the controller. Both of these improvements were found to be essential in learning a robot controller from simulation that transfers successfully to the real robot.

This study proposes a self-supervised method for detecting scene changes from an image pair. For mobile cameras such as drive recorders, to alleviate the camera viewpoints' difference, image alignment and change detection must be optimized simultaneously because they depend on each other. Moreover, lighting condition makes the scene change detection more difficult because it widely varies in images taken at different times. To solve these challenges, we propose a self-supervised simultaneous alignment and change detection net-work (SACD-Net). The proposed network is robust specifically in differences of camera viewpoints and lighting conditions to simultaneously estimate warping parameters and multi-scale change probability maps while change regions are not taken into account of calculation of the feature consistency and semantic losses. Based on comparative analysis between our self-supervised and the previous supervised models as well as ablation study of the losses of SACD-Net, the results show the effectiveness of the proposed method using a synthetic dataset and our new real dataset.

Self-supervised learning for visual object tracking possesses valuable advantages compared to supervised learning, such as the non-necessity of laborious human annotations and online training. In this work, we exploit an end-to-end Siamese network in a cycle-consistent self-supervised framework for object tracking. Self-supervision can be performed by taking advantage of the cycle consistency in the forward and backward tracking. To better leverage the end-to-end learning of deep networks, we propose to integrate a Siamese region proposal and mask regression network in our tracking framework so that a fast and more accurate tracker can be learned without the annotation of each frame. The experiments on the VOT dataset for visual object tracking and on the DAVIS dataset for video object segmentation propagation show that our method outperforms prior approaches on both tasks.

Detecting sound source objects within visual observation is important for autonomous robots to comprehend surrounding environments. Since sounding objects have a large variety with different appearances in our living environments, labeling all sounding objects is impossible in practice. This calls for self-supervised learning which does not require manual labeling. Most of conventional self-supervised learning uses monaural audio signals and images and cannot distinguish sound source objects having similar appearances due to poor spatial information in audio signals. To solve this problem, this paper presents a self-supervised training method using 360° images and multichannel audio signals. By incorporating with the spatial information in multichannel audio signals, our method trains deep neural networks (DNNs) to distinguish multiple sound source objects. Our system for localizing sound source objects in the image is composed of audio and visual DNNs. The visual DNN is trained to localize sound source candidates within an input image. The audio DNN verifies whether each candidate actually produces sound or not. These DNNs are jointly trained in a self-supervised manner based on a probabilistic spatial audio model. Experimental results with simulated data showed that the DNNs trained by our method localized multiple speakers. We also demonstrate that the visual DNN detected objects including talking visitors and specific exhibits from real data recorded in a science museum.

In this paper, we report a self-sensing soft tactile actuator based on Dielectric elastomer actuator (DEA) for wearable haptic interface. DEAs are one of electroactive polymer actuators, which are reported to have large area strain and fast response speed. A soft tactile actuator is constructed of a multi-layered DEA membrane layer, a passive membrane layer, and an inner circular pillar. The soft actuator was optimized by varying the geometry, and the force and displacement tests were conducted under a frequency range of 0 to 30 Hz. The selected actuator produces an output force up to 0.9 N, with a displacement of 1.43 mm. To provide accurate physical force feedback to the user, the actuator is integrated with a 1.1 mm thick film-type soft force sensor that enables feedback control. Under the pressure, touch layer contacts with the core, and the light inside the core scatters to the touch layer. A fabricated soft force sensor can measure the force in a range of 0 to 1.25 N under various frequency ranges. Our wearable prototype exhibits high output force of 0.9 N, as well as flexibility, conformity, and light-weight structure (3.2 g).

In this paper, we present a novel description for the configuration space of adaptive modular robots with a triangular structure based on extended binary trees. In general, binary trees can serve as a representation of kinematic trees with a maximum of two immediate descendants per element. Kinematic loops are incorporated in the tree structure by an ingenious extension of the binary tree indices. The introduction of equivalence classes then allows a unique mathematical description of specific configurations of the robot system. Subsequently, we show how the extended binary tree can serve as a systematic tool for reconfiguration planning, allowing to solve the self-reconfiguration problem for modular robots with a triangular structure, which has as yet no general solution. Reconfiguration is performed by populating the binary tree indices of a desired target configuration in an ascending manner, moving modules along the surface of the robot. We demonstrate the planning algorithm on a simple example and conclude by outlining a way to translate the individual reconfiguration steps to specific module movement commands.

We used cells, which are the units that make up a living body, as building blocks to design a biomachine hybrid system and develop a tactile sensor that uses living cells as sensor receptors. We fabricated a novel cell tactile sensor with the electrodes formed using printed electronics technology. This sensor comprises elastic electrodes mounted on a soft material to acquire tactile information; similar to a conventional cell tactile sensor, it acquires signals through mechanical stimulation. Further, self-organization of cells can be induced, and logical processing such as selective responses to stimuli can be performed directly by the physical system, without any coding using programming languages. The proposed novel cell tactile sensor that uses printed electrodes is small enough to mount on robots. Interestingly, we confirmed the self-healing properties of the proposed sensor after cells were injured mechanically.