IROS 2021 论文列表

Soft, tip-extending, pneumatic "vine robots" that grow via eversion are well suited for navigating cluttered environments. Two key mechanisms that add to the robot’s functionality are a tip-mounted retraction device that allows the growth process to be reversed, and a tip-mounted camera that enables vision. However, previous designs used rigid, relatively heavy electromechanical retraction devices and external camera mounts, which reduce some advantages of these robots. These designs prevent the robot from squeezing through tight gaps, make it challenging to lift the robot tip against gravity, and require the robot to drag components against the environment. To address these limitations, we present a soft, pneumatically driven retraction device and an internal camera mount that are both lightweight and smaller than the diameter of the robot. The retraction device is composed of a soft, extending pneumatic actuator and a pair of soft clamping actuators that work together in an inch-worming motion. The camera mount sits inside the robot body and is kept at the tip of the robot by two low-friction interlocking components. We present characterizations of our retraction device and demonstrations that the robot can grow and retract through turns, tight gaps, and sticky environments while transmitting live video from the tip. Our designs advance the ability of everting vine robots to navigate difficult terrain while collecting data.

Timely fault detection and identification (FDI) of soft manipulators are critical in the design of surgical systems to improve reliability. However, due to the intrinsic compliance of soft manipulators, their end effectors vibrate during the dynamic control process, which introduces noise into the measured signals and makes FDI of soft manipulators challenging. This paper proposes a novel method to accomplish these tasks based on Long Short Term Memory (LSTM) recurrent neural network. Based on LSTM network, a new Attention-based Noise Compensation (ANC) module is proposed to enable the network to filter the noise merged with signals input in a self-supervision manner. Moreover, weighted cross entropy loss is introduced to balance the normal and faulty samples in the training set. Of the 9930 samples presented to the model, 9489 are correctly diagnosed in less than 1.0 second, which implies that the method can learn the spatial and temporal dependence of the signals and distinguish the healthy modes from the faulty ones. Finally, we compare the ANC-based method with the vanilla LSTM method and the state-of-art Bruin et al. method. From the comparison, we conclude that the ANC-based method proposed in this paper not only shortens the time cost of the FDI process but also suppresses the sensitivity of diagnosis results to noise. Source code, pre-trained models and dataset are available on https://github.com/IRMVLab/ANC-LSTM-fault-detection.

Due to their inherent compliance, soft robots are more versatile than rigid linked robots when they interact with their environment, such as object manipulation or biomimetic motion, and are considered to be the key element in introducing robots to everyday environments. Although various soft robotic actuators exist, past research has focused primarily on designing and analyzing single components. Limited effort has been made to combine each component to create an overall capable, integrated soft robot. Ideally, the behavior of such a robot can be accurately modeled, and its motion within an environment uses its proprioception, without requiring external sensors. This work presents a design and modeling process for a Soft continuum Proprioceptive Arm (SoPrA) actuated by pneumatics. The integrated design is suitable for an analytical model due to its internal capacitive flex sensor for proprioceptive measurements and its fiber-reinforced fluidic elastomer actuators. The proposed analytical dynamical model accounts for the inertial effects of the actuator’s mass and the material properties, and predicts in real-time the soft robot’s behavior. Our estimation method integrates the analytical model with proprioceptive sensors to calculate external forces, all without relying on an external motion capture system. SoPrA is validated in a series of experiments demonstrating the model’s and sensor’s accuracy in estimation. SoPrA will enable soft arm manipulation including force sensing while operating in obstructed environments that disallows exteroceptive measurements.

Engineers and scientists often rely on their intuition and experience when designing soft robotic systems. The development of performant controllers and motion plans for these systems commonly requires time-consuming iterations on hardware. We present the SoMo (Soft Motion) toolkit, a software framework that makes it easy to instantiate and control typical continuum manipulators in an accurate physics simulator. SoMo introduces a standardized and human-readable description format for continuum manipulators. It leverages this description format and the Bullet physics engine to enable fast and accurate simulations of soft and soft-rigid hybrid robots in environments with complex contact interactions. This allows users to vary design and control parameters across simulations with minimal effort. We compare the capabilities of SoMo to other physics simulators and highlight the benefits and accuracy of SoMo by demonstrating the agreement between simulation and real-world experiments on several examples; these include an in-hand manipulation task with continuum fingers, an automated exploration of how to design soft fingers for precision grasping, and a brief snake locomotion study. Overall, SoMo provides an accessible way for designers of soft robotic hardware and control systems to gain access to a simulation-accelerated workflow.

Nano quadcopters are ideal for gas source localization (GSL) as they are safe, agile and inexpensive. However, their extremely restricted sensors and computational resources make GSL a daunting challenge. We propose a novel bug algorithm named ‘Sniffy Bug', which allows a fully autonomous swarm of gas-seeking nano quadcopters to localize a gas source in unknown, cluttered, and GPS-denied environments. The computationally efficient, mapless algorithm foresees in the avoidance of obstacles and other swarm members, while pursuing desired waypoints. The waypoints are first set for exploration, and, when a single swarm member has sensed the gas, by a particle swarm optimization-based (PSO) procedure. We evolve all the parameters of the bug (and PSO) algorithm using our novel simulation pipeline, ‘AutoGDM'. It builds on and expands open source tools in order to enable fully automated end-to-end environment generation and gas dispersion modeling, allowing for learning in simulation. Flight tests show that Sniffy Bug with evolved parameters outperforms manually selected parameters in cluttered, real-world environments. Videos: https://bit.ly/37MmtdL

Telerobotic systems combined with miniaturised snake-like or elephant-trunk robotic arms can improve the ergonomics and accessibility in minimally invasive surgical tasks such as knee arthroscopy. Such systems, however, are usually designed in a specific and integral approach, making it expensive to adapt to various procedures or patient anatomies. 3D printed instruments with a detachable design can bring the benefits of patient-specific customisation, affordability, and adaptability to new clinical scenarios. However, the integration of such snake-like instruments to standard telerobotic systems can be challenging in terms of design and control. In this study, a teleoperation system is developed to control and steer the pose of SnakeRaven: a 3D printed, customisable snake-like end-effector attached to the RAVEN II platform for the application of fibre-optic knee arthroscopy. Algorithms for the parametric inverse kinematics and mapping between the RAVEN II joint space to the coupled tendon-driven rolling joints are developed. The controller is tested and validated on the physical prototype interfacing with the RAVEN II platform in a teleoperation experiment. A video demonstrating the main results of this paper can be found via https://youtu.be/ApJjR853kIQ

Meshes are commonly used as 3D maps since they encode the topology of the scene while being lightweight. Unfortunately, 3D meshes are mathematically difficult to handle directly because of their combinatorial and discrete nature. Therefore, most approaches generate 3D meshes of a scene after fusing depth data using volumetric or other representations. Nevertheless, volumetric fusion remains computationally expensive both in terms of speed and memory. In this paper, we leapfrog these intermediate representations and build a 3D mesh directly from a depth map and the sparse landmarks triangulated with visual odometry. To this end, we formulate a non-smooth convex optimization problem that we solve using a primal-dual method. Our approach generates a smooth and accurate 3D mesh that substantially improves the state-of-the-art on direct mesh reconstruction while running in real-time.

Despite the stringent requirements of a real-time system, the reliance of the Robot Operating System (ROS) on the loopback network interface imposes a considerable overhead on the transport of high bandwidth data, while the nodelet package, which is an efficient mechanism for intra-process communication, does not address the problem of efficient local inter-process communication (IPC). To remedy this, we propose a novel integration into ROS of smart pointers and synchronisation primitives stored in shared memory. These obey the same semantics and, more importantly, exhibit the same performance as their C++ standard library counterparts, making them preferable to other local IPC mechanisms. We present a series of benchmarks for our mechanism - which we call LOT (Low Overhead Transport) - and use them to assess its performance on realistic data loads based on Five’s Autonomous Vehicle (AV) system, and extend our analysis to the case where multiple ROS nodes are running in Docker containers. We find that our mechanism performs up to two orders of magnitude better than the standard IPC via local loopback. Finally, we apply industry-standard profiling techniques to explore the hotspots of code running in both user and kernel space, comparing our implementation against alternatives.

In this study, based on an analysis of experimental results, we propose a method for slip modeling of Spiral Zipper. The Spiral Zipper is a prismatic actuator that extracts a length-changeable cylindrical tube from a flexible ABS band. The band tube is driven by a friction wheel; the slip results due to the lack of synchronization between the motion of the band and that of the friction wheel. Through experiments, we verify that the flexibility of the band causes an impact force between the band and the friction wheel. Additionally, we verify that the tube extraction process causes resistance, which contributes to the increase in the relative angular velocity. Considering the factors mentioned above, we develop a slip model by employing different numbers of contact patches that express the flexibility of the band. Subsequently, we validate the model’s performance through simulations. In addition, we propose a coefficients standardization method for obtaining similar results with different number of contact patches We expect that this study will be a basis for design optimization and control system development of the Spiral Zipper.

The integration of multiple cameras and 3D Li-DARs has become basic configuration of augmented reality devices, robotics, and autonomous vehicles. The calibration of multi-modal sensors is crucial for a system to properly function, but it remains tedious and impractical for mass production. Moreover, most devices require re-calibration after usage for certain period of time. In this paper, we propose a single-shot solution for calibrating extrinsic transformations among multiple cameras and 3D LiDARs. We establish a panoramic infrastructure, in which a camera or LiDAR can be robustly localized using data from single frame. Experiments are conducted on three devices with different camera-LiDAR configurations, showing that our approach achieved comparable calibration accuracy with the state-of-the-art approaches but with much greater efficiency.

Grasping in cluttered scenes has always been a great challenge for robots, due to the requirement of the ability to well understand the scene and object information. Previous works usually assume that the geometry information of the objects is available, or utilize a step-wise, multi-stage strategy to predict the feasible 6-DoF grasp poses. In this work, we propose to formalize the 6-DoF grasp pose estimation as a simultaneous multi-task learning problem. In a unified framework, we jointly predict the feasible 6-DoF grasp poses, instance semantic segmentation, and collision information. The whole framework is jointly optimized and end-to-end differentiable. Our model is evaluated on large-scale benchmarks as well as the real robot system. On the public dataset, our method outperforms prior state-of-the-art methods by a large margin (+4.08 AP). We also demonstrate the implementation of our model on a real robotic platform and show that the robot can accurately grasp target objects in cluttered scenarios with a high success rate. Project link: https://openbyterobotics.github.io/sscl.

Recent work has demonstrated real-time mapping and reconstruction from dense perception, while motion planning based on distance fields has been shown to achieve fast, collision-free motion synthesis with good convergence properties. However, demonstration of a fully integrated system that can safely re-plan in unknown environments, in the presence of static and dynamic obstacles, has remained an open challenge. In this work, we first study the impact that signed and unsigned distance fields have on optimisation convergence, and the resultant error cost in trajectory optimisation problems in 2D path planning, arm manipulator motion planning, and whole-body loco-manipulation planning. We further analyse the performance of three state-of-the-art approaches to generating distance fields (Voxblox, Fiesta, and GPU-Voxels) for use in realtime environment reconstruction. Finally, we use our findings to construct a practical hybrid mapping and motion planning system which uses GPU-Voxels and GPMP2 to perform receding- horizon whole-body motion planning that can smoothly avoid moving obstacles in 3D space using live sensor data. Our results are validated in simulation and on a real-world Toyota Human Support Robot (HSR).

Pedestrian trajectories and actions prediction in complex environment is challenging due to the complexity of human behavior and a variety of internal and external stimuli. Much works has gone towards predicting trajectories and actions separately without mining the coupling relationships between them, which is an important information for our humans to reason and predict. Inspired by this, we propose an end-to-end joint context information iterative reasoning network (CIR-Net). Specifically, a novel heterogeneous spatiotemporal graph module (HST-Graph) is proposed to encode and aggregate multiple types of context information of the motion pattern and the scene. And an action-trajectory hybrid guidance module is proposed to enhance the ability of long-time prediction by utilizing the internal coupling between actions and trajectory. Moreover, an iterative reasoning structure is designed to iteratively correcting the trajectory and actions prediction error. Experimental results on the ETH&UCY and VIRAT datasets demonstrate the favorable performance of the framework.

Large workspace localization of magnetic robots is important for medical applications. This paper presents a novel localization strategy to achieve simultaneous localization and actuation of magnetic robots using hall-effect sensors. We integrate 25 sensors into a sensing probe and mount it on to the mobile-coil system, which realizes accurate sensing and actuation of magnetic devices within a cylindrical workspace of ϕ500 mm×150 mm. Simulation results show the average localization error using the proposed method is 1.7 mm. A verification experiment is conducted to prove the design advantages; Another two experiments are conducted to demonstrate the simultaneous actuation and localization of a torque-driven robot and a force-driven floating robot respectively. For the force-driven floating robot, the average variation between the localization results and the desired trajectory is less than 2 mm.

It is well understood that nature has a calming effect on us. But in a physical space remote from nature, might the robotic embodiment of a natural phenomenon have the same effect? To address this question, we have simulated the soothing movement of ocean waves in a soft robotic surface, both as a simulation and in a physical prototype. In this paper, we report on our modeling methods of this natural phenomenon, we present the application of a selected model to both a simulation of the predicted inflation pattern of the soft robot surface and to the physical soft robot surface, and we compare the behavior of the physical robot to our simulation as a means to validate our design. We observed that the prototype’s frequency spectrum successfully shared the general trend and shape as our mathematical model. This research introduces a novel robot for a novel application: a soft robot to promote restoration and help regulate mental wellbeing in tight physical confines.

In this paper, we propose a similarity-aware fusion network (SAFNet) to adaptively fuse 2D images and 3D point clouds for 3D semantic segmentation. Existing fusion-based methods achieve superior performances by integrating information from multiple modalities. However, they heavily rely on the projection-based correspondence between 2D pixels and 3D points and can only perform the information fusion in a fixed manner, so that their performances cannot be easily migrated to a more realistic scenario where the collected data often lack strict pair-wise features for prediction. To address this, we employ a late fusion strategy where we first learn the geometric and contextual similarities between the input and back-projected (from 2D pixels) point clouds and utilize them to guide the fusion of two modalities to further exploit complementary information. Specifically, we employ a geometric similarity module (GSM) to directly compare the spatial coordinate distributions of pair-wise 3D neighborhoods, and a contextual similarity module (CSM) to aggregate and compare spatial contextual information of corresponding central points. The two proposed modules can effectively measure how much image features can help predictions, enabling the network to adaptively adjust the contributions of two modalities to the final prediction of each point. Experimental results on ScanNetV2 [1] benchmark demonstrate that SAFNet outperforms existing state-of-the-art fusion-based approaches across various data integrity.

Learning policies in simulation is promising for reducing human effort when training robot controllers. This is especially true for soft robots that are more adaptive and safe but also more difficult to accurately model and control. The sim2real gap is the main barrier to successfully transfer policies from simulation to a real robot. System identification can be applied to reduce this gap but traditional identification methods require a lot of manual tuning. Data-driven alternatives can tune dynamical models directly from data but are often data hungry, which also incorporates human effort in collecting data. This work proposes a data-driven, end-to-end differentiable simulator focused on the exciting but challenging domain of tensegrity robots. To the best of the authors’ knowledge, this is the first differentiable physics engine for tensegrity robots that supports cable, contact, and actuation modeling. The aim is to develop a reasonably simplified, data-driven simulation, which can learn approximate dynamics with limited ground truth data. The dynamics must be accurate enough to generate policies that can be transferred back to the ground-truth system. As a first step in this direction, the current work demonstrates sim2sim transfer, where the unknown physical model of MuJoCo acts as a ground truth system. Two different tensegrity robots are used for evaluation and learning of locomotion policies, a 6-bar and a 3-bar tensegrity. The results indicate that only 0.25% of ground truth data are needed to train a policy that works on the ground truth system when the differentiable engine is used for training against training the policy directly on the ground truth system.

Reinforcement Learning (RL) methods have been widely applied for robotic manipulations via sim-to-real transfer, typically with proprioceptive and visual information. However, the incorporation of tactile sensing into RL for contact-rich tasks lacks investigation. In this paper, we model a tactile sensor in simulation and study the effects of its feedback in RL-based robotic control via a zero-shot sim-to-real approach with domain randomization. We demonstrate that learning and controlling with feedback from tactile sensor arrays at the gripper, both in simulation and reality, can enhance grasping stability, which leads to a significant improvement in robotic manipulation performance for a door opening task. In real-world experiments, the door open angle was increased by 45% on average for transferred policies with tactile sensing over those without it.

Recently, the Siamese-based method has stood out from multitudinous tracking methods owing to its state-of-the-art (SOTA) performance. Nevertheless, due to various special challenges in UAV tracking, e.g., severe occlusion and fast motion, most existing Siamese-based trackers hardly combine superior performance with high efficiency. To this concern, in this paper, a novel attentional Siamese tracker (SiamAPN++) is proposed for real-time UAV tracking. By virtue of the attention mechanism, we conduct a special attentional aggregation network (AAN) consisting of self-AAN and cross-AAN for raising the representation ability of features eventually. The former AAN aggregates and models the self-semantic interdependencies of the single feature map via spatial and channel dimensions. The latter aims to aggregate the cross-interdependencies of two different semantic features including the location information of anchors. In addition, the anchor proposal network based on dual features is proposed to raise its robustness of tracking objects with various scales. Experiments on two well-known authoritative benchmarks are conducted, where SiamAPN++ outperforms its baseline SiamAPN and other SOTA trackers. Besides, real-world tests onboard a typical embedded platform demonstrate that SiamAPN++ achieves promising tracking results with real-time speed.