IROS 2020 Papers

In recent years, many excellent SLAM methods based on cameras, especially the camera-IMU fusion (VIO), have emerged, which has greatly improved the accuracy and robustness of SLAM. However, we find through experiments that most of the existing VIO methods perform well on drones or drone datasets, but for ground robots on complex terrain, they cannot continuously provide accurate and robust localization results. Some researchers have proposed methods for ground robots, but most of them have limited applications due to the assumption of plane motion. Therefore, this paper proposes GR-SLAM for the localization of ground robots on complex terrain, which can fuse camera, IMU, and encoder data in a tightly coupled scheme to provide accurate and robust state estimation for robots. First, an odometer increment model is proposed, which can fuse the encoder and IMU data to calculate the robot pose increment on manifold, and calculate the frame constraints through the pre-integrated increment. Then we propose an evaluation algorithm for multi-sensor measurements, which can detect abnormal data and adjust its optimization weight. Finally, we implement a complete factor graph optimization framework based on sliding window, which can tightly couple camera, IMU, and encoder data to perform state estimation. Extensive experiments are conducted based on a real ground robot and the results show that GR-SLAM can provide accurate and robust state estimation for ground robots.

Sampling-based planning has become a de facto standard for complex robots given its superior ability to rapidly explore high-dimensional configuration spaces. Most existing optimal sampling-based planning algorithms are sequential in nature and cannot take advantage of wide parallelism available on modern computer hardware. Further, tight synchronization of exploration and exploitation phases in these algorithms limits sample throughput and planner performance. Policy Iteration RRT# (PI-RRT#) exposes fine-grained parallelism during the exploitation phase, but this parallelism has not yet been evaluated using a concrete implementation. We first present a novel GPU implementation of PI-RRT#'s exploitation phase and discuss data structure considerations to maximize parallel performance. Our implementation achieves 3-4× speedup over a serial PI-RRT# implementation for a 77.9% decrease in overall planning time on average. As a second contribution, we introduce the Batched-Extension RRT# algorithm, which loosens the synchronization present in PI-RRT# to realize independent 12.97× and 12.54× speedups under serial and parallel exploitation, respectively.

Autonomous vehicles are typically developed and trained to work under certain system and environmental conditions defined at design time and can fail or perform poorly if unforeseen conditions such as disturbances or changes in model dynamics appear at runtime. In this work, we present a fast online planning, learning, and recovery approach for safe autonomous operations under unknown runtime disturbances. Our approach estimates the behavior of the system with an unknown model and provides safe plans at runtime under previously unseen disturbances by leveraging Gaussian Process regression theory in which a model is continuously trained and adapted using data collected during the autonomous operation. A recovery procedure is event-triggered any time a safety constraint is violated to guarantee safety and enable learning and replanning. The proposed framework is applied and validated both in simulation and experiment on an unmanned aerial vehicle (UAV) delivery case study in which the UAV is tasked to carry an a priori unknown payload to a goal location in a cluttered/constrained environment.

This paper presents a 3D lidar SLAM system based on improved regionalized Gaussian process (GP) map reconstruction to provide both low-drift state estimation and mapping in real-time for robotics applications. We utilize spatial GP regression to model the environment. This tool enables us to recover surfaces including those in sparsely scanned areas and obtain uniform samples with uncertainty. Those properties facilitate robust data association and map updating in our scan-to-map registration scheme, especially when working with sparse range data. Compared with previous GP-SLAM, this work overcomes the prohibitive computational complexity of GP and redesigns the registration strategy to meet the accuracy requirements in 3D scenarios. For large-scale tasks, a two-thread framework is employed to suppress the drift further. Aerial and ground-based experiments demonstrate that our method allows robust odometry and precise mapping in real-time. It also outperforms the state-of-the-art lidar SLAM systems in our tests with light-weight sensors.

Detecting loop closures in 3D Light Detection and Ranging (LiDAR) data is a challenging task since point-level methods always suffer from instability. This paper presents a semantic-level approach named GOSMatch to perform reliable place recognition. Our method leverages novel descriptors, which are generated from the spatial relationship between semantics, to perform frame description and data association. We also propose a coarse-to-fine strategy to efficiently search for loop closures. Besides, GOSMatch can give an accurate 6-DOF initial pose estimation once a loop closure is confirmed. Extensive experiments have been conducted on the KITTI odometry dataset and the results show that GOSMatch can achieve robust loop closure detection performance and outperform existing methods.

We propose a novel approach to handling the ambiguity in elevation angle associated with the observations of a forward looking multi-beam imaging sonar, and the challenges it poses for performing an accurate 3D reconstruction. We utilize a pair of sonars with orthogonal axes of uncertainty to independently observe the same points in the environment from two different perspectives, and associate these observations. Using these concurrent observations, we can create a dense, fully defined point cloud at every time-step to aid in reconstructing the 3D geometry of underwater scenes. We will evaluate our method in the context of the current state of the art, for which strong assumptions on object geometry limit applicability to generalized 3D scenes. We will discuss results from laboratory tests that quantitatively benchmark our algorithm's reconstruction capabilities, and results from a real-world, tidal river basin which qualitatively demonstrate our ability to reconstruct a cluttered field of underwater objects.

Synergy provides a practical approach for expressing various postures of a multi-fingered hand. However, a conventional synergy defined for reproducing grasping postures cannot perform in-hand manipulation, e.g., tasks that involve simultaneously grasping and manipulating an object. Locking the position of particular fingers of a multi-fingered hand is essential for in-hand manipulation tasks either to hold an object or to fix unnecessary fingers. When using conventional synergy based control to manipulate an object, which requires locking some fingers, the coordination of joints is heavily restricted, decreasing the dexterity of the hand. We propose a functionally divided manipulation synergy (FDMS) method, which provides a synergy-based control to achieves both dimensionality reduction and in-hand manipulation. In FDMS, first, we define the function of each finger of the hand as either "manipulation" or "fixed." Then, we apply synergy control only to the fingers having the manipulation function, so that dexterous manipulations can be realized with a few control inputs. Furthermore, we propose the Synergy Switching Framework as a method for applying a finely defined FDMS to sequential task changes. The effectiveness of our method is experimentally verified.

This paper focuses on pose registration of different object instances from the same category. This is required in online object mapping because object instances detected at test time usually differ from the training instances. Our approach transforms instances of the same category to a normalized canonical coordinate frame and uses metric learning to train fully convolutional geometric features. The resulting model is able to generate pairs of matching points between the instances, allowing category-level registration. Evaluation on both synthetic and real-world data shows that our method provides robust features, leading to accurate alignment of instances with different shapes.

The growing consumer demand for large volumes of high quality fruit has generated an increasing need for auto-mated fruit quality control during production. Optical methods have been proved successful in a few cases, but with limitations related to the variability of fruit colors and lighting conditions during tests. Tactile sensing provides a valuable alternative, although it comes with the need of a physical interaction that could damage the fruit. To overcome these limitations, we propose the usage of a recently developed soft tactile sensor for non-invasive fruit quality control. The ability of the sensor to detect very small forces and to finely analyze surfaces allows the collection of relevant information about the fruit by performing a very delicate physical interaction, that does not cause any damage. We report experiments in which such information is used to determine whether apples and strawberries are ripe or senescent. We test different configurations of the sensor and different classification algorithms, achieving very good accuracy for both apples (96%) and strawberries (83%).

We propose an integrated approach to active exploration by exploiting the Cartographer method as the base SLAM module for submap creation and performing efficient frontier detection in the geometrically co-aligned submaps induced by graph optimization. We also carry out analysis on the reachability of frontiers and their clusters to ensure that the detected frontier can be reached by robot. Our method is tested on a mobile robot in real indoor scene to demonstrate the effectiveness and efficiency of our approach.

Planar structures are common in man-made environments. Their addition to monocular SLAM algorithms is of relevance in order to achieve more complete and higher- level scene representations. Also, the additional constraints they introduce might reduce the estimation errors in certain situations. In this paper we present a novel formulation to incorporate plane landmarks and planar constraints to feature- based monocular SLAM. Specifically, we enforce in-plane points to lie exactly in the plane they belong to, propagating such information to the rest of the states. Our formulation, differently from the state of the art, allows us to incorporate general planes, independently of depth information or CNN segmentation being available (although we could also use them). We evaluate our method in several sequences of public databases, showing accurate plane estimations and pose accuracy on par with state- of-the-art point-only monocular SLAM.

The Everyday Activities Science and Engineering (EASE) Collaborative Research Consortium’s mission to enhance the performance of cognition-enabled robots establishes its foundation in the EASE Human Activities Data Analysis Pipeline. Through collection of diverse human activity information resources, enrichment with contextually relevant annotations, and subsequent multimodal analysis of the combined data sources, the pipeline described will provide a rich resource for robot planning researchers, through incorporation in the OpenEASE cloud platform.

Mechanical systems are typically composed of a number of contacting surfaces that move against each other. Such surfaces are subject to friction forces. These dissipate part of the actuation energy and cause an undesired effect on the overall system functioning. Therefore, a suitable model of friction is needed to elide its action. The choice of such a model is not always straightforward, as it is influenced by the system properties and dynamics. In this paper, we show the identification of different friction models and evaluate their prediction capability on an experimental dataset. Despite being state-of-the-art models, some modifications were introduced to improve their performance. A pneumatic gripper was used to collect the data for the models evaluation. Two experimental setups were mounted to execute the experiments: information from two pressure sensors, a load cell and a position sensor was employed for the identification. During the experiments, the gripper was actuated at different constant velocities. Results indicate that all the identified models offer a proper prediction of the real friction force.

This paper proposes a novel modular selfreconfigurable robot (MSRR) "FreeBOT", which can be connected freely at any point on other robots. FreeBOT is mainly composed of two parts: a spherical ferromagnetic shell and an internal magnet. The connection between the modules is genderless and instant, since the internal magnet can freely attract other FreeBOT spherical ferromagnetic shells, and not need to be precisely aligned with the specified connector. This connection method has fewer physical constraints, so the FreeBOT system can be extended to more configurations to meet more functional requirements. FreeBOT can accomplish multiple tasks although it only has two motors: module independent movement, connector management and system reconfiguration. FreeBOT can move independently on the plane, and even climb on ferromagnetic walls; a group of FreeBOTs can traverse complex terrain. Numerous experiments have been conducted to test its function, which shows that the FreeBOT system has great potential to realize a freeform robotic system.

In this paper, we propose a formal language to specify robot skills, i.e. the elementary behaviours or functions provided by the robot platform in order to perform an autonomous mission. The advantage of the language we propose is that it integrates a wide range of elements that allows to define and provide automatic translation both to operational models, used online to control the skill execution, and descriptive models, allowing to reason about the expected skill execution, and then apply automated planning or model-checking taking skill models into account.

One-shot imitation is the vision of robot programming from a single demonstration, rather than by tedious construction of computer code. We present a practical method for realizing one-shot imitation for manipulation tasks, exploiting modern learning-based optical flow to perform real-time visual servoing. Our approach, which we call FlowControl, continuously tracks a demonstration video, using a specified foreground mask to attend to an object of interest. Using RGB-D observations, FlowControl requires no 3D object models, and is easy to set up. FlowControl inherits great robustness to visual appearance from decades of work in optical flow. We exhibit FlowControl on a range of problems, including ones requiring very precise motions, and ones requiring the ability to generalize.

The conceptual design and flight controller of a novel kind of quadcopter are presented. This design is capable of morphing the shape of the UAV during flight to achieve position and attitude control. We consider a dynamic center of gravity (CoG) which causes continuous variation in a moment of inertia (MoI) parameters of the UAV. These dynamic structural parameters play a vital role in the stability and control of the system. The length of quadcopter arms is a variable parameter, and it is actuated using attitude feedback-based control law. The MoI parameters are computed in real-time and incorporated in the equations of motion of the system. The UAV utilizes the angular motion of propellers and variable quadcopter arm lengths for position and navigation control. The movement space of the CoG is a design parameter and it is bounded by actuator limitations and stability requirements of the system. A detailed information on equations of motion, flight controller design and possible applications of this system are provided. Further, the proposed shape-changing UAV system is evaluated by comparative numerical simulations for way point navigation mission and complex trajectory tracking.

A flexible endoscope introduces more dexterity to the image capturing in endoscopic surgery. However, manual control or automatic control based on instrument tracking does not handle the misorientation between the endoscopic video and the surgeon. We propose an automatic flexible endoscope control method that tracks the surgeon's head with respect to the object in the surgical scene. The robotic flexible endoscope is actuated so that it captures the surgical scene from the same perspective as the surgeon. The surgeon wears a head-mounted display to observe the endoscopic video. The frustum of the flexible endoscope is rendered as an augmented reality overlay to provide surgical guidance. We developed the prototype, FlexiVision, integrating a 6-DOF robotic flexible endoscope based on the da Vinci Research Kit and Microsoft HoloLens. We evaluated the proposed automatic control method via a lesion observation task, and evaluated the AR surgical guidance in a lesion targeting task. The multi-user study results demonstrated that, for both tasks, FlexiVision significantly reduced the completion time (by 59% and 58%), number of errors (by 75% and 95%) and subjective task load level. With FlexiVision, the flexible endoscope could act as the surgeon's eyes teleported into the abdominal cavity of the patient.

Traditional approaches to quadruped control frequently employ simplified, hand-derived models. This significantly reduces the capability of the robot since its effective kinematic range is curtailed. In addition, kinodynamic constraints are often non-differentiable and difficult to implement in an optimisation approach. In this work, these challenges are addressed by framing quadruped control as optimisation in a structured latent space. A deep generative model captures a statistical representation of feasible joint configurations, whilst complex dynamic and terminal constraints are expressed via high-level, semantic indicators and represented by learned classifiers operating upon the latent space. As a consequence, complex constraints are rendered differentiable and evaluated an order of magnitude faster than analytical approaches. We validate the feasibility of locomotion trajectories optimised using our approach both in simulation and on a real-world ANY-mal quadruped. Our results demonstrate that this approach is capable of generating smooth and realisable trajectories. To the best of our knowledge, this is the first time latent space control has been successfully applied to a complex, real robot platform.