IROS 2022 论文列表

Simultaneous localization and mapping (SLAM) plays a vital role in mapping unknown spaces and aiding autonomous navigation. Virtually all state-of-the-art solutions today for 2D SLAM are designed for dense and accurate sensors such as laser range-finders (LiDARs). However, these sensors are not suitable for resource-limited nano robots, which become increasingly capable and ubiquitous nowadays, and these robots tend to mount economical and low-power sensors that can only provide sparse and noisy measurements. This introduces a challenging problem called SLAM with sparse sensing. This work addresses the problem by adopting the form of the state-of-the-art graph-based SLAM pipeline with a novel frontend and an improvement for loop closing in the backend, both of which are designed to work with sparse and uncertain range data. Experiments show that the maps constructed by our algorithm have superior quality compared to prior works on sparse sensing. Furthermore, our method is capable of running in real-time on a modern PC with an average processing time of 1/100th the input interval time.

Self-body awareness refers to the recognition of one's body as one's own and consists of two senses: “sense of body ownership” and “sense of agency.” In telexistence/telepresence robot operation, time delays in the robot's motion degrade self-body awareness of the robot body. We investigated how self-body recognition can be affected in a telexistence robot operation in a VR space when the robot is presented with a real robot arm that simulates a real robot with a delay and a virtual robot arm, or several virtual robot arms, with a delay less than that of the real robot. These experimental conditions include a ‘Predictive Display,’ which is well known as a time delay countermeasure. The results suggest that virtual robot arms presented consecutively with less delay than a real robot arm do not induce a sense of body ownership to the real robot arm, but they enhance the sense of agency to the real robot arm, and that sense of agency is stronger when the task requires precision.

In this work, we developed a mathematical model and a simulation platform for a fish-inspired robotic template, namely Magnetic, Modular, Undulatory Robot (\mu \text{Bot})(\mu \text{Bot}). Through this platform, we systematically explored the effects of robot design and fluid parameters on swimming performance via reinforcement learning. The mathematical model was composed of two interacting subsystems, the robotic dynamic model and the hydrodynamic model. The hydrodynamic model consisted of the reactive components (added-mass force and pressure forces) and the resistive components (drag and friction forces). These components were nondimensionalized for deriving key “control parameters” of the robot-fluid interaction. The \mu\text{Bots}\mu\text{Bots} were actuated via magnetic actuators controlled with harmonic voltage signals, which were optimized via EM-based Policy Hyper Parameter Exploration (EPHE) to maximize forward swimming speed. By varying the control parameters, a total of 36 cases with different robot template variations (Number of Actuators (NoA) and stiffness) and hydrodynamic parameters were simulated and optimized via EPHE. Results showed that the wavelength of the optimized gaits (i.e., backward traveling wave along the body) was independent of template variations and hydrodynamic parameters. Higher NoA yielded higher speed but lower speed per body length, suggesting a diminishing gain from added actuators. Body and caudal-fin dynamics were dominated by the interaction among fluid added-mass, spring, and actuation torque, with negligible contribution from fluid resistive drag. In contrast, thrust was dominated by the pressure force acting on the caudal fin, as steady swimming resulted from a balance between resistive force and pressure force, with minor contributions from added-mass force and body drag forces. Therefore, added-mass force only indirectly affected the thrust generation and forward swimming speed via the caudal fin dynamics.

Semantic object mapping in uncertain, perceptually degraded environments during long-range multi-robot autonomous exploration tasks such as search-and-rescue is important and challenging. During such missions, high recall is desirable to avoid missing true target objects and high precision is also critical to avoid wasting valuable operational time on false positives. Given recent advancements in visual perception algorithms, the former is largely solvable autonomously, but the latter is difficult to address without the supervision of a human operator. However, operational constraints such as mission time, computational requirements and mesh network bandwidth can make the operator's task infeasible unless properly managed. We propose the Early Recall, Late Precision (EaRLaP) semantic object mapping pipeline to solve this problem. EaRLaP was used by Team CoSTAR in DARPA Subterranean Challenge, where it successfully detected all the artifacts encountered by the team of robots. We will discuss these results and the performance of the EaRLaP on various datasets.

This paper provides the EVOPS dataset for plane segmentation from 3D data, both from RGBD images and LiDAR point clouds. We have designed two annotation methodologies (RGBD and LiDAR) running on well-known and widely-used datasets for SLAM evaluation and we have provided a complete set of benchmarking tools including point, planes and segmentation metrics. The data includes a total number of 10k RGBD and 7K LiDAR frames over different selected scenes which consist of high quality segmented planes. The experiments report quality of SOTA methods for RGBD plane segmentation on our annotated data. We also have provided learnable baseline for plane segmentation in LiDAR point clouds. All labeled data and benchmark tools used have been made publicly available https://evops.netlify.app/.

Augmented reality (AR) defines a new information-delivery paradigm by overlaying computer-generated information on the perception of the real world. AR-integrated robot has become an appealing concept in terms of enhanced human-robot interaction. Despite intensive research on AR, existing indoor location-based AR systems are vulnerable to attacks and can hardly meet the security and privacy requirements in practice. The problem of designing a secure AR framework to ensure the efficiency and privacy of location-based AR has not been sufficiently studied. In this paper, we holistically study this problem and propose EPAR, an efficient and privacy-aware AR framework for indoor location-based services. EPAR distinguishes itself from the existing work by being the first to address the issues of AR delivery in terms of system scalability, accuracy, privacy, and efficiency. First, an effective indoor location cloaking scheme is presented to safeguard user's privacy while improving system scalability and accuracy. Then, a novel privacy-aware localization scheme is proposed to hierarchically localize the user with privacy concerns. Finally, for the AR content delivery, a new authenticated data structure is tailored to save the data transmission cost and improve system efficiency. We implement EPAR and conduct extensive experiments in real-world scenarios. Evaluation results demonstrate the effectiveness of our EPAR system.

Ultrasound (US) imaging is a common but physically demanding task in the medical field, and sonographers may need to put in considerable physical effort for producing high-quality US images. During physical human-robot interaction on US imaging, robot compliance is a critical feature that can ensure human user safety while automatic force regulation ability can help to improve task performance. However, higher robot compliance may mean lower force regulation accuracy, and vice versa. Especially, the contact/non-contact status transition can largely affect the control system stability. In this paper, a novel electromyography (EMG)-based hybrid impedance-force control system is developed for US imaging task. The proposed control system incorporates the robot compliance and force regulation ability via a hybrid controller while the EMG channel enables the user to online modulate the trade-off between the two features as necessary. Two experiments are conducted to examine the hybrid controller and show the necessity of involving an EMG-based modulator. A proof-of-concept study on US imaging is performed with implementing the proposed EMG-based control system, and the effectiveness is demonstrated. The proposed control system is promising to ensure robot's stability and patient's safety, thus obtain high-quality US images, while monitoring and reducing sonographer's fatigue. Furthermore, it can be easily adapted to other physically demanding tasks in the field of medicine.

In interacting with stiff environments through teleoperated systems, time delays cause a mismatch between haptic feedback and the expected feedback by the operator. This mismatch causes artefacts in the feedback, which decrease transparency, but so does filtering these artefacts. Through modelling of operator stiffness and the expected feedback force with EMG, the artifacts can be selectively filtered without loss of transparency. We developed several feedback modulation techniques to bring the feedback force closer to the expected force: 1) the average between the modelled operator force and the feedback force, 2) a low pass filter and 3) a scaling modulation. To control for overdamping, a transparency check is included. We show that the averaging approach yields significantly better contacts than unmodulated feedback. None of the modulation algorithms differ significantly from the unmodulated feedback in transparency.

Highly accurate multi-person pose estimation at a high framerate is a fundamental problem in autonomous driving. Solving the problem could aid in preventing pedestrian-car accidents. The present study tackles this problem by proposing a new model composed of a feature pyramid and an original head to a general backbone. The original head is built using lightweight CNNs and directly estimates multi-person pose coordinates. This configuration avoids the complex post-processing and two-stage estimation adopted by other models and allows for a lightweight model. Our model can be trained end-to-end and performed in real-time on a resource-limited platform (low-cost edge device) during inference. Experimental results using the COCO and CrowdPose datasets showed that our model can achieve a higher framerate (approx. 20 frames/sec with NVIDIA Jetson AGX Xavier) than other state-of-the-art models while maintaining sufficient accuracy for practical use.

Robotic tactile sensing provides a method of recognizing objects and their properties where vision fails. Prior work on tactile perception in robotic manipulation has frequently focused on exploratory procedures (EPs). However, the also-human-inspired technique of in-hand-manipulation can glean rich data in a fraction of the time of EPs. We propose a simple 3-DOF robotic hand design, optimized for object rolling tasks via a variable-width palm and associated control system. This system dynamically adjusts the distance between the finger bases in response to object behavior. Compared to fixed finger bases, this technique significantly increases the area of the object that is exposed to finger-mounted tactile arrays during a single rolling motion (an increase of over 60% was observed for a cylinder with a 30-millimeter diameter). In addition, this paper presents a feature extraction algorithm for the collected spatiotemporal dataset, which focuses on object corner identification, analysis, and compact representation. This technique drastically reduces the dimensionality of each data sample from \boldsymbol{10\times 1500}\boldsymbol{10\times 1500} time series data to 80 features, which was further reduced by Principal Component Analysis (PCA) to 22 components. An ensemble subspace k-nearest neighbors (KNN) classification model was trained with 90 observations on rolling three different geometric objects, resulting in a three-fold cross-validation accuracy of 95.6% for object shape recognition.

Detection and segmentation of moving obstacles, along with prediction of the future occupancy states of the local environment, are essential for autonomous vehicles to proactively make safe and informed decisions. In this paper, we propose a framework that integrates the two capabilities together using deep neural network architectures. Our method first detects and segments moving objects in the scene, and uses this information to predict the spatiotemporal evolution of the environment around autonomous vehicles. to address the problem of direct integration of both static-dynamic object segmentation and environment prediction models, we propose using occupancy-based environment representations across the whole framework. Our method is validated on the real-world Waymo Open Dataset and demonstrates higher prediction accuracy than baseline methods.

This paper presents an attention-based, deep learning framework that converts robot camera frames with dynamic content into static frames to more easily apply simultaneous localization and mapping (SLAM) algorithms. The vast majority of SLAM methods have difficulty in the presence of dynamic objects appearing in the environment and occluding the area being captured by the camera. Despite past attempts to deal with dynamic objects, challenges remain to reconstruct large, occluded areas with complex backgrounds. Our proposed Dynamic-GAN framework employs a generative adversarial network to remove dynamic objects from a scene and inpaint a static image free of dynamic objects. The Dynamic-GAN framework utilizes spatial-temporal transformers, and a novel spatial-temporal loss function. The evaluation of Dynamic-GAN was comprehensively conducted both quantitatively and qualitatively by testing it on benchmark datasets, and on a mobile robot in indoor navigation environments. As people appeared dynamically in close proximity to the robot, results showed that large, feature-rich occluded areas can be accurately reconstructed with our attention-based deep learning framework for dynamic object removal. Through experiments we demonstrate that our proposed algorithm has up to 25% better performance on average as compared to the standard benchmark algorithms.

When navigating to a goal in an uncertain environment, a robot must simultaneously navigate the exploration-exploitation tradeoff: should it aim to gain information and reduce uncertainty, or should it simply brave the unknown? We formalize this as the Bayesian dynamic motion planning problem, and we analyze how several strategies from the literature balance these concerns via determinization and planning. Within the framework of determinization in the face of uncertainty, we shift the burden of exploration to determinization rather than planning. Dynamic Replanning with Posterior Sampling (DRPS) is very efficient: each iteration consists of a single posterior update and a shortest path query. Relative to comparative baselines across seven datasets of 2D planning problems, DRPS has a higher percentage of success, traverses lower or comparable total distances, and accelerates total planning time by 4–7×. Across a dataset of larger 7D Baxter manipulator planning problems, DRPS reduces total distance by 40% and total planning time by 18×.

Dialogue State Tracking (DST) is an important part in task-oriented dialog system, whose target is to infer the current dialog states and user intentions according to the dialog history information. To this end, we have achieved improvements to the existing work and proposed a dynamic network model suitable for multi-domain dialog, which can explicitly use domain information and better cope with zero-shot tasks. The model is composed of three modules: an encoder, a decoder and a slot classifier. The encoder module introduces a mixed-separate framework so that it can obtain the feature information of each domain on the premise of extracting the shared information between all domains. The experimental results show that the model achieves joint accuracy of 48.38% for the five domains of MultiWOZ, which is superior to existing models. Besides, by simulating the zero-shot scenario, the knowledge transferability of the model has also been well proven. Finally, in order to verify the effectiveness of the robot simulation system, this paper also uses the robot simulation technology to simulate the common tasks of helping users complete the task of taking items in the home environment service.

Enabling robots to perform complex dynamic tasks such as picking up an object in one sweeping motion or pushing off a wall to quickly turn a corner is a challenging problem. The dynamic interactions implicit in these tasks are critical towards the successful execution of such tasks. Graph neural networks (GNNs) provide a principled way of learning the dynamics of interactive systems but can suffer from scaling issues as the number of interactions increases. Furthermore, the problem of using learned GNN-based models for optimal control is insufficiently explored. In this work, we present a method for efficiently learning the dynamics of interacting systems by simultaneously learning a dynamic graph structure and a stable and locally linear forward model of the system. The dynamic graph structure encodes evolving contact modes along a trajectory by making probabilistic predictions over the edges of the graph. Additionally, we introduce a temporal dependence in the learned graph structure which allows us to incorporate contact measurement updates during execution thus enabling more accurate forward predictions. The learned stable and locally linear dynamics enable the use of optimal control algorithms such as iLQR for long-horizon planning and control for complex interactive tasks. Through experiments in simulation and in the real world, we evaluate the performance of our method by using the learned interaction dynamics for control and demonstrate generalization to more objects and interactions not seen during training. We introduce a control scheme that takes advantage of contact measurement updates and hence is robust to prediction inaccuracies during execution.

Free-space-oriented roadmaps typically generate a series of convex geometric primitives, which constitute the safe region for motion planning. However, a static environment is assumed for this kind of roadmap. This assumption makes it unable to deal with dynamic obstacles and limits its applications. In this paper, we present a dynamic free-space roadmap, which provides feasible spaces and a navigation graph for safe quadrotor motion planning. Our roadmap is constructed by continuously seeding and extracting free regions in the environment. In order to adapt our map to environments with dynamic obstacles, we incrementally decompose the polyhedra intersecting with obstacles into obstacle-free regions, while the graph is also updated by our well-designed mechanism. Extensive simulations and real-world experiments demonstrate that our method is practically applicable and efficient.

Large-scale environmental sensing with a finite number of mobile sensors is a challenging task that requires tremendous resources and time. This is especially true when the environmental features of interest are spatiotemporally changing with unknown or partially known dynamics. Fortunately, these dynamic features often evolve in a low-dimensional space, making it possible to capture their dynamics sufficiently well with a finite number of sensor measurements. This paper investigates the problem of dynamic compressed sensing of an unsteady, periodic flow field with a mobile sensor. We take advantage of the inherently low dimensionality of the under-lying flow dynamics to reduce number of critical waypoints for sensor trajectory. The optimal set of sensing waypoints are identified by an iterative compressed sensing algorithm that optimizes the flow reconstruction based on the proper orthogonal decomposition modes. An optimal sampling trajectory is then found to traverse these waypoints while minimizing the energy consumption, time, and flow reconstruction error. Simulation results in a double-gyre flow field is presented to demonstrate the efficacy of the proposed algorithms. Experimental results with an indoor quadcopter are presented to show the tracking feasibility of the resulting trajectory.

This study proposes a dual-camera system for indoor high magnification surveillance which is capable of achieving always-in-focus and non-delay gaze control based on high-speed vision. The users are enabled to move the mouse freely on the wide-view screen while observing its in-focal zoom-in monitoring video in real-time. The proposed system consists of a wide-angle camera for wide-view and a Galvano mirror-enabled ultra-fast pan-tilt-zoom (PTZ) camera for zoom-in view. To achieve always-in-focus, a high-speed focus scanning system is proposed that is comprised of a high-speed camera, a parfocal zoom lens, and a gear mechanism. Through continuously reciprocating rotational motion of the focusing ring driven by the servo motor, the high-speed camera captures sets of images with varying focal distances. Moreover, we proposed a most-in-focus (MIF) frame extraction algorithm to select the sharpest images as output. The experimental results are obtained to confirm the effectiveness of our system.

Concrete infrastructures provide the means to connect cities and transport people and goods. They require regular inspection to assess their current conditions. Aerial work platforms and underbridge platforms or scaffolding are the common equipment used for inspection of elevated infrastructure. These methods often cost more to operate and maintain, are time-consuming, and raise risks for the inspector. One interesting field of research for UAVs that can be used for infrastructure inspection is aerial perching. A perching UAV can be loaded with an inspection apparatus foregoing the need for costly equipment and risks involved in the inspection. Many have presented aerial perching for various applications and not as much for applications related to concrete infrastructure inspection. This study investigates a perching UAV that can perform perching on both smooth and rough concrete surfaces. This paper presents an unmanned aerial system that utilizes a suction-based perching mechanism with a pneumatic supply tethered from the ground. The proposed perching mechanism provides a reliable and high payload capacity needed for non-destructive testing of the infrastructure. The paper introduces the concept, presents the design and proof of concept, and validates the idea through actual bridge experiments.