ICRA 2022 论文列表

Micro-scale robots introduce great prospective into many different medical applications such as targeted drug delivery, minimally invasive surgery and localized bio-metric diagnostics. This research presents a method for object detection and tracking system of a chain-like magnetic microsphere robots using ultrasound imaging in an in-vitro environment. The method estimates the position of the microrobot in real-time using deep learning techniques. The experiments showed that a spherical microrobot with about 500 m in diameter can be detected and tracked in real-time with a high accuracy in dynamic environments. The results exhibit a high detection and tracking accuracy for one, two and three sphere microrobots with the highest accuracy in detection and tracking around 95 % and 93% respectively.

We present an automated learning framework for a robotic sketching agent that is capable of learning stroke-based rendering and motor control simultaneously. We formulate the robotic sketching problem as a deep decoupled hierarchical reinforcement learning; two policies for stroke-based rendering and motor control are learned independently to achieve sub-tasks for drawing, and form a hierarchy when cooperating for real-world drawing. Without hand-crafted features, drawing sequences or trajectories, and inverse kinematics, the proposed method trains the robotic sketching agent from scratch. We performed experiments with a 6-DoF robot arm with 2F gripper to sketch doodles. Our experimental results show that the two policies successfully learned the sub-tasks and collaborated to sketch the target images. Also, the robustness and flexibility were examined by varying drawing tools and surfaces.

This paper proposes a novel freeform strut-node structured modular self-reconfigurable robot (MSRR) called FreeSN, consisting of strut and node modules. A node module is mainly a low-carbon steel spherical shell. A strut module contains two freeform connectors, which provide strong magnetic connections and flexible spherical motions. The FreeSN system shares the benefits of freeform connection and strut-node structures. The freeform connection brings good adaptability to the environment. The triangle substructures inside the system configuration significantly improve the structural stability. The parallel execution of module motions can superpose the module capabilities and makes the system more scalable. The modules can combine these robot features by selecting the system configuration and better fit different circumstances and tasks. Four demonstrations, including assembly, obstacle crossing, transportation, and object manipulation, are designed to show the capabilities of the FreeSN system in different aspects. The results show the great performance and versatility of this MSRR system.

We apply a novel framework for decomposing and reasoning about free space in an environment to a multi-agent persistent monitoring problem. Our decomposition method represents free space as a collection of ellipsoids associated with a weighted connectivity graph. The same ellipsoids used for reasoning about connectivity and distance during high level planning can be used as state constraints in a Model Predictive Control algorithm to enforce collision-free motion. This structure allows for streamlined implementation in distributed multi-agent tasks in 2D and 3D environments. We illustrate its effectiveness for a team of tracking agents tasked with monitoring a group of target agents. Our algorithm uses the ellipsoid decomposition as a primitive for the coordination, path planning, and control of the tracking agents. Simulations with four tracking agents monitoring fifteen dynamic targets in obstacle-rich environments demonstrate the performance of our algorithm.

The free energy principle from neuroscience provides a brain-inspired perception scheme through a data-driven model learning algorithm called Dynamic Expectation Maximization (DEM). This paper aims at introducing an exper-imental design to provide the first experimental confirmation of the usefulness of DEM as a state and input estimator for real robots. Through a series of quadcopter flight experiments under unmodelled wind dynamics, we prove that DEM can leverage the information from colored noise for accurate state and input estimation through the use of generalized coordinates. We demonstrate the superior performance of DEM for state es-timation under colored noise with respect to other benchmarks like State Augmentation, SMIKF and Kalman Filtering through its minimal estimation error. We demonstrate the similarities in the performance of DEM and Unknown Input Observer (UIO) for input estimation. The paper concludes by showing the influence of prior beliefs in shaping the accuracy-complexity trade-off during DEM's estimation.

We develop a task-independent predictive framework that estimates hip, knee and ankle future behavior from sonomyographic sensing of quadriceps musculature. Two regression models, support vector regression and Gaussian process regression, were trained and tested such that no ambulation mode recognition was required. Sonomyography features of the anterior thigh musculature were extracted during the swing phase of level, incline and stair ambulation tasks as inputs to the two models for continuous prediction of the future stance phase hip, knee and ankle moments. Next, sonomyography features of the anterior thigh musculature were extracted during the stance phase and used to predict the following swing phase hip, knee and ankle angles. Leave-one-stride-out cross-validation is used to evaluate this continuous prediction framework. Additionally, initial, peak and terminal joint moment and angle parameters are extracted from trajectories and evaluated. Both regression models were able to accurately predict continuous future joint moments and angles, as well as initial, peak and terminal value parameters of future joint moments and angles. However, the support vector regression model required relatively lower computational cost. Thus, we recommend the support vector regression model as an optimal model for forward prediction of joint mechanics from sonomyographic sensing during ambulation.

In this work, we consider the problem of controlling the end effector position of a continuum manipulator through local stiffness changes. Continuum manipulators offer the advantage of continuous deformation along their lengths, and recent advances in smart material actuators further enable local compliance changes, which can affect the manipulator's bulk motion. However, leveraging local stiffness change to control motion remains lightly explored. We build a kinematic model of a continuum manipulator as a sequence of segments consisting of symmetrically arranged springs around the perimeter of every segment, and we show that this system has a closed form solution to its forward kinematics. The model includes common constraints such as restriction of torsional or shearing movement. Based on this model, we propose a controller on the spring stiffnesses for a single segment and provide provable guarantees on convergence to a desired goal position. The results are verified in simulation and compared to physical hardware.

This paper investigates the cooperative control problem of choke-point navigation for multiple quadcopters when only their subgroup is equipped with obstacle detecting sensors. We define a quadcopter as a leader if it is equipped with an obstacle detecting sensor; otherwise, it is a follower. In addition, we introduce a virtual leader agent to create the group motion. First, we apply the leader-follower approach and propose a formation-containment tracking controller for multiple quadcopters to track the time-varying velocity of the virtual leader agent. At the same time, the leader quadcopters form the prescribed formation while the follower quadcopters converge inside a safe region, which is the convex hull spanned by those leaders. Then, we introduce a scaling vector into the displacement-based formation constraints. When the leader quadcopters identify the choke-point via their obstacle detecting sensors, they update the scaling variable to adjust the size of the formation (i.e. the safe region) and guide all quadcopters to safely pass through the choke-point. The proposed cooperative controllers are distributed because each quadcopter's control command only relies on the information states from its neighbours. Finally, two autonomous flight experiments, including formation-containment tracking and choke-point navigation, are provided to validate the effectiveness of the proposed cooperative control laws.

In this work, we address the problem of formal safety verification for stochastic cyber-physical systems (CPS) equipped with ReLU neural network (NN) controllers. Our goal is to find the set of initial states from where, with a predetermined confidence, the system will not reach an unsafe configuration within a specified time horizon. Specifically, we consider discrete-time LTI systems with Gaussian noise, which we abstract by a suitable graph. Then, we formulate a Satisfiability Modulo Convex (SMC) problem to estimate upper bounds on the transition probabilities between nodes in the graph. Using this abstraction, we propose a method to compute tight bounds on the safety probabilities of nodes in this graph, despite possible over-approximations of the transition probabilities between these nodes. Additionally, using the proposed SMC formula, we devise a heuristic method to refine the abstraction of the system in order to further improve the estimated safety bounds. Finally, we corroborate the efficacy of the proposed method with simulation results considering a robot navigation example and comparison against a state-of-the-art verification scheme.

To traverse complex scenarios reliably a legged robot needs to move its base aided by the ground reaction forces, which can only be generated by the legs that are momentarily in contact with the ground. A proper selection of footholds is crucial for maintaining balance. In this paper, we propose a foothold evaluation criterion that considers the transition feasibility for both linear and angular dynamics to overcome complex scenarios. We devise convex and nonlinear formulations as a direct extension of [1] in a receding-horizon fashion to grant dynamic feasibility for future behaviours. The criterion is integrated with a Vision-based Foothold Adaptation (VFA) strategy that takes into account the robot kinematics, leg collisions and terrain morphology. We verify the validity of the selected footholds and the generated trajectories in simulation and experiments with the 90kg quadruped robot HyQ.

We present a flow-based control strategy that enables resource-constrained marine robots to patrol gyre-like flow environments on an orbital trajectory with a periodicity in a given range. The controller does not require a detailed model of the flow field and relies only on the robot's location relative to the center of the gyre. Instead of precisely tracking a pre-defined trajectory, the robots are tasked to stay in between two bounding trajectories with known periodicity. Furthermore, the proposed strategy leverages the surrounding flow field to minimize control effort. We prove that the proposed strategy enables robots to cycle in the flow satisfying the desired periodicity requirements. Our method is tested and validated both in simulation and in experiments using a low-cost, underactuated, surface swimming robot, i.e. the Modboat.

We present an approach to synthesize novel views from dynamics scenes captured by multi-view videos of cameras mounted on a driving vehicle. We unify existing methods and propose a new training loss to explicitly disentangle the static background from the dynamic foreground objects using scene flow's magnitude, learnt only from proxy 2D optical flow supervision. We obtain high quality static and dynamic contents separately, which allow us to combine them freely for novel view and time syntheses. We establish a dataset consisting of 5 dynamic scenes with varying difficulties on which we conduct experiments, and show that our method is able to handle challenging scenarios in real-world traffics and create high quality novel view and time syntheses.

Tip force sensing for cable-driven continuum robots are vital to provide the force information for safe and reliable human-robot interaction. However, traditional triaxial force sensors usually have a complicated structure occupying its inner lumen, without enough space for additional instrumental tools. To solve this, this paper proposes a fixed and sliding fiber Bragg grating (FBG) sensors-based triaxial force sensing method for cable-driven continuum robots. The fixed FBG sensors are attached to the circumferential surface of continuum robot at the tip and base, and the sliding optical fibers with FBG sensors are located in the actuation channels as the sensing integrated pulling cables. This configuration guarantees a compact structure and large inner lumen. Two five-degreed-of-freedom (5-DOF) electromagnetic (EM) and a 6-DOF EM sensors are assembled to the tip and the base of the robot respectively, which can obtain the pose of the tip with respect to the base. The tip force in three directions can be decoupled using the information of the Bragg wavelength changes and EM sensors. Results show that the mean errors of force sensing along x-direction, y-direction, and z-direction are 4.1%, 4.7%, and 9.8%, respectively. The proposed sensing method does not rely on the elasticity of continuum robot, enabling its wide applicability for other cable-driven pseudo-continuum robots.

Bionic underwater robots have demonstrated their superiority in many applications. Yet, training their intelligence for a variety of tasks that mimic the behavior of underwater creatures poses a number of challenges in practice, mainly due to lack of a large amount of available training data as well as the high cost in real physical environment. Alternatively, simulation has been considered as a viable and important tool for acquiring datasets in different environments, but it mostly targeted rigid and soft body systems. There is currently dearth of work for more complex fluid systems interacting with immersed solids that can be efficiently and accurately simulated for robot training purposes. In this paper, we propose a new platform called “FishGym”, which can be used to train fish-like underwater robots. The framework consists of a robotic fish modeling module using articulated body with skinning, a GPU-based high-performance localized two-way coupled fluid-structure interaction simulation module that handles both finite and infinitely large domains, as well as a reinforcement learning module. We leveraged existing training methods with adaptations to underwater fish-like robots and obtained learned control policies for multiple benchmark tasks. The training results are demonstrated with reasonable motion trajectories, with comparisons and analyses to empirical models as well as known real fish swimming behaviors to highlight the advantages of the proposed platform.

Dense object tracking, the ability to localize specific object points with pixel-level accuracy, is an important computer vision task with numerous downstream applications in robotics. Existing approaches either compute dense keypoint embeddings in a single forward pass, meaning the model is trained to track everything at once, or allocate their full capacity to a sparse predefined set of points, trading generality for accuracy. In this paper we explore a middle ground based on the observation that the number of relevant points at a given time are typically relatively few, e.g. grasp points on a target object. Our main contribution is a novel architecture, inspired by few-shot task adaptation, which allows a sparse-style network to condition on a keypoint embedding that indicates which point to track. Our central finding is that this approach provides the generality of dense-embedding models, while offering accuracy significantly closer to sparse-keypoint approaches. We present results illustrating this capacity vs. accuracy trade-off, and demonstrate the ability to zero-shot transfer to new object instances (within-class) using a real-robot pick-and-place task.

Masking by Moving (MByM), provides robust and accurate radar odometry measurements through an exhaustive correlative search across discretised pose candidates. However, this dense search creates a significant computational bottleneck which hinders real-time performance when high-end GPUs are not available. Utilising the translational invariance of the Fourier Transform, in our approach, Fast Masking by Moving (f-MByM), we decouple the search for angle and translation. By maintaining end-to-end differentiability a neural network is used to mask scans and trained by supervising pose prediction directly. Training faster and with less memory, utilising a decoupled search allows f-MbyM to achieve significant run-time performance improvements on a CPU (168 %) and to run in real-time on embedded devices, in stark contrast to MbyM. Throughout, our approach remains accurate and competitive with the best radar odometry variants available in the literature – achieving an end-point drift of 2.01 % in translation and 6.3 deg /km on the Oxford Radar RobotCar Dataset.

This paper is focused on the cooperative trajectory planning problem for multiple car-like robots in a cluttered and unstructured environment narrowed by static obstacles. The concerned multi-vehicle trajectory planning (MVTP) problem is challenging because i) the scenario is nonconvex and tiny; ii) the vehicle kinematics is nonconvex; and iii) a feasible homotopy class is unavailable a priori. We propose a two-stage MVTP method: Stage 1 identifies a feasible homotopy class, and Stage 2 quickly finds a local optimum based on the identified homotopy class. Numerical optimal control, adaptive scaling, grouping, and trust region construction strategies are integrated into the proposed planner. Our planner is extensively compared in 100 benchmark cases with the state-of-the-art MVTP methods such as incremental sequential convex programming, numerical optimal control, conflict-based search, priority-based trajectory optimizer, and optimal reciprocal collision avoidance. The simulation results demonstrate our method's superiority in runtime and optimality. Experiments with three car-like robots demonstrate the efficiency of our proposed planner. Source codes are in https://github.com/libai1943/MVTP_benchmark.

The high performance of RGB-D based road segmentation methods contrasts with their rare application in commercial autonomous driving, which is owing to two reasons: 1) the prior methods cannot achieve high inference speed and high accuracy in both ways; 2) the different properties of RGB and depth data are not well-exploited, limiting the reliability of predicted road. In this paper, based on the evidence theory, an uncertainty-aware symmetric network (USNet) is proposed to achieve a trade-off between speed and accuracy by fully fusing RGB and depth data. Firstly, cross-modal feature fusion operations, which are indispensable in the prior RGB-D based methods, are abandoned. We instead separately adopt two light-weight subnetworks to learn road representations from RGB and depth inputs. The light-weight structure guarantees the real-time inference of our method. Moreover, a multi-scale evidence collection (MEC) module is designed to collect evidence in multiple scales for each modality, which provides sufficient evidence for pixel class determination. Finally, in uncertainty-aware fusion (UAF) module, the uncertainty of each modality is perceived to guide the fusion of the two sub-networks. Experimental results demonstrate that our method achieves a state-of-the-art accuracy with real-time inference speed of 43+43+ FPS. The source code is available at https://github.com/morancyc/USNet.

3D LiDAR is widely used in autonomous systems such as self-driving cars and autonomous robots because it provides accurate 3D point clouds of the surrounding environment under harsh conditions. However, a high-resolution LiDAR is expensive and bulky. Although a low-resolution LiDAR is compact and affordable, the obtained point clouds are so sparse that it is difficult to extract features that are meaningful for highlevel tasks. To solve this problem, several upsampling-based approaches have been proposed by estimating high-resolution point clouds from low-resolution point clouds. However, most works have focused on upsampling object-level or synthetic point clouds obtained from CAD models. Additionally, these approaches have a high computational cost, which makes them unusable in real-time applications such as autonomous driving vehicles. In this paper, we propose a real-time upsampling method with LiDAR for outdoor environments. The proposed method builds on conditional neural processes that are capable of uncertainty quantification. With this probabilistic property, we can remove the upsampled points that have high uncertainty, thus achieving high accuracy. Additionally, the proposed method can be trained in a simulated environment, and then directly applied to the real world. The experimental results on a simulated environment and a real-world dataset show that the proposed method is significantly faster than the state-of-the-art methods while achieving comparable performance.