IROS 2022 论文列表

In this work, we consider the Multi-Agent Pickup-and-Delivery (MAPD) problem, where agents constantly engage with new tasks and need to plan collision-free paths to execute them. To execute a task, an agent needs to visit a pair of goal locations, consisting of a pickup location and a delivery location. We propose two variants of an algorithm that assigns a sequence of tasks to each agent using the anytime algorithm Large Neighborhood Search (LNS) and plans paths using the Multi-Agent Path Finding (MAPF) algorithm Priority-Based Search (PBS). LNS-PBS is complete for well-formed MAPD instances, a realistic subclass of MAPD instances, and empirically more effective than the existing complete MAPD algorithm CENTRAL. LNS-wPBS provides no completeness guarantee but is empirically more efficient and stable than LNS-PBS. It scales to thousands of agents and thousands of tasks in a large warehouse and is empirically more effective than the existing scalable MAPD algorithm HBH+MLA*. LNS-PBS and LNS-wPBS also apply to a more general variant of MAPD, namely the Multi-Goal MAPD (MG-MAPD) problem, where tasks can have different numbers of goal locations.

Robots with multi-fingered grippers could perform advanced manipulation tasks for us if we were able to properly specify to them what to do. In this study, we take a step in that direction by making a robot grasp an object like a grasping demonstration performed by a human. We propose a novel optimization-based approach for transferring human grasp demonstrations to any multi-fingered grippers, which produces robotic grasps that mimic the human hand orientation and the contact area with the object, while alleviating interpenetration. Extensive experiments with the Allegro and BarrettHand grippers show that our method leads to grasps more similar to the human demonstration than existing approaches, without requiring any gripper-specific tuning. We confirm these findings through a user study and validate the applicability of our approach on a real robot.

Soft robots have a wide rang of applications due to their compliance, flexibility and low fabrication cost. Compare to rigid robots, soft robots are more safe for human. In this work, we design various multi-DoF actuators with spring reinforce and particle jamming, and two fabrication methods are proposed to make them. Each type of actuator is tested to evaluate the mechanical properties by experiments. Experimental results show that both spring and particle jamming have effect on the stiffness. Besides, Spring reinforced actuator (SRA) increases the maximum allowable inflation pressure, output bending force and gives a good linear relationship of bending. We also integrate spring and particles to fabricate a hybrid actuator whose behaviors are explored by experiments. Finally, we create a snake-like robotic manipulator assembled with two actuators and show its bending motion. Results show that the manipulator is able to achieve several bending shapes steadily and large range of motion.

Multiple sensors, especially cameras and LiDARs, are widely used in autonomous vehicles. In order to fuse data from different sensors accurately, precise calibrations are required, including camera intrinsic parameters, and relative poses between multiple cameras and LiDARs. However, most existing camera-LiDAR calibration methods need to place manually designed calibration objects in multiple locations and multiple times, which are time-consuming and labor-intensive, and are not suitable for frequent use. To address that, in this paper we proposed a novel calibration pipeline that can automatically calibrate multiple cameras and multiple LiDARs in a Structure-from-Motion (SfM) process. In our pipeline, we first perform a global SfM on all images with the help of rough LiDAR data to get the initial poses of all sensors. Then, feature points on lines and planes are extracted from both SfM point cloud and LiDARs. With these features, a global Bundle Adjustment is performed to minimize the point reprojection errors, point-to-line errors, and point-to-plane errors together. During this minimization process, camera intrinsic parameters, camera and LiDAR poses, and SfM point cloud are refined jointly. The proposed method uses the characteristics of natural scenes, does not require manually designed calibration objects, and incorporates all calibration parameters into a unified optimization framework. Experiments on autonomous vehicles with different sensor configurations demonstrate the effectiveness and robustness of the proposed method.

Inter-agent relative localization is critical for any multi-robot system operating in the absence of external positioning infrastructure or prior environmental knowledge. We propose a novel inter-agent relative 2D pose estimation system where each participating agent is equipped with several ultra-wideband (UWB) ranging tags. Prior work typically supplements noisy UWB range measurements with additional continuously transmitted data, such as odometry, making these approaches scale poorly with increased swarm size or decreased communication throughput. This approach addresses these concerns by using only locally collected UWB measurements with no additionally transmitted data. By modeling observed ranging biases and systematic antenna obstructions in our proposed optimization solution, our experimental results demonstrate an improved mean position error (while remaining competitive in other metrics) over a similar state-of-the-art approach that additionally relies on continuously transmitted odometry.

In this study, a topology-guided unified nonlinear model predictive control (NMPC) approach is proposed for autonomous navigation of a class of Hybrid Terrestrial and Aerial Quadrotors (HyTAQs) in unknown environments. The approach can fully exploit the hybrid terrestrial-aerial locomotion of the vehicle and as such ensure a high navigation efficiency. A unified terrestrial-aerial NMPC is first formulated with a type of complementarity constraints involving the hybrid dynamics, together with the collision avoidance constraints for safety. Further, a topological roadmap with both terrestrial and aerial paths is leveraged to guide the kinodynamic path searching and thus the unified NMPC. Then, a complete and distinctive navigation framework is established and validated on our self-developed HyTAQ. Compared with the existing unified terrestrial-aerial planning methods, ours takes the vehicle dynamics into account for the first attempt and achieves a more reasonable decision of modes switching. Experimental results are presented to demonstrate the effectiveness and superiority of the proposed approach.

The complex dynamics of agile robotic legged locomotion requires motion planning to intelligently adjust footstep locations. Often, bipedal footstep and motion planning use mathematically simple models such as the linear inverted pendulum, instead of dynamically-rich models that do not have closed-form solutions. We propose a real-time optimization method to plan for dynamical models that do not have closed form solutions and experience irrecoverable failure. Our method uses a data-driven approximation of the step-to-step dynamics and of a failure margin function. This failure margin function is an oriented distance function in state-action space where it describes the signed distance to success or failure. The motion planning problem is formed as a nonlinear program with constraints that enforce the approximated forward dynamics and the validity of state-action pairs. For illustration, this method is applied to create a planner for an actuated spring-loaded inverted pendulum model. In an ablation study, the failure margin constraints decreased the number of invalid solutions by between 24 and 47 percentage points across different objectives and horizon lengths. While we demonstrate the method on a canonical model of locomotion, we also discuss how this can be applied to data-driven models and full-order robot models.

To efficiently generate safe trajectories for an autonomous vehicle in dynamic environments, a layered motion planning method with decoupled path and speed planning is widely used. This paper studies speed planning, which mainly deals with dynamic obstacle avoidance given a planned path. The main challenges lie in the optimization in a non-convex space and the trade-off between safety, comfort, and efficiency. First, this work proposes to conduct a search in second-order derivative space for generating a comfort-optimal reference trajectory. Second, by combining abstraction and refinement, an algorithm is proposed to construct a convex feasible space for optimization. Finally, a piecewise Bézier polynomial optimization approach with trapezoidal corridors is presented, which theoretically guarantees safety and significantly enlarges the solution space compared with the existing rectangular corridors-based approach. We validate the efficiency and effectiveness of the proposed approach in simulations.

Navigation and obstacle avoidance in aquatic en-vironments for autonomous surface vehicles (ASVs) in high-traffic maritime scenarios is still an open challenge, as the Convention on the International Regulations for Preventing Collisions at Sea (COLREGs) is not defined for multi-encounter situations. Current state-of-the-art methods resolve single-to-single encounters with sequential actions and assume that other obstacles follow COLREGs. Our work proposes a novel real-time non-myopic obstacle avoidance method, allowing an ASV that has only partial knowledge of the surroundings within the sensor radius to navigate in high-traffic maritime scenarios. Specifically, we achieve a holistic view of the feasible ASV action space able to avoid deadlock scenarios, by proposing (1) a clustering method based on motion attributes of other obstacles, (2) a geometric framework for identifying the feasible action space, and (3) a multi-objective optimization to determine the best action. Theoretical analysis and extensive realistic exper-iments in simulation considering real-world traffic scenarios demonstrate that our proposed real-time obstacle avoidance method is able to achieve safer trajectories than other state-of-the-art methods and that is robust to uncertainty present in the current information available to the ASV.

Event cameras are biologically-inspired vision sensors that capture pixel-level illumination changes instead of the intensity image at a fixed frame rate. They offer many advantages over the standard cameras, such as high dynamic range, high temporal resolution (low latency), no motion blur, etc. Therefore, developing state estimation algorithms based on event cameras offers exciting opportunities for autonomous systems and robots. In this paper, we propose monocular visual-inertial odometry for event cameras based on event-corner feature detection and matching with well-designed feature management. More specifically, two different kinds of event representations based on time surface are designed to realize event-corner feature tracking (for front-end incremental estimation) and matching (for loop closure detection). Furthermore, the proposed event representations are used to set mask for detecting the event-corner feature based on the raw event-stream, which ensures the uniformly distributed and spatial consistency characteristic of the event-corner feature. Finally, a tightly coupled, graph-based optimization framework is designed to obtain high-accurate state estimation through fusing pre-integrated IMU measurements and event-corner observations. We validate quantitatively the performance of our system on different resolution event cameras: DAVIS240C (240*180, public dataset, achieve state-of-the-art), DAVIS346 (346*240, real-test), DVXplorer (640*480 real-test). Furthermore, we demonstrate qualitatively the accuracy, robustness, loop closure, and re-localization performance of our framework on different large-scale datasets, and an autonomous quadrotor flight using our Event Visual-inertial Odometry (EVIO) framework. Videos of all the evaluations are presented on the project website.

Depth estimation from panoramic imagery has received minimal attention in contrast to standard perspective imagery, which constitutes the majority of the literature on the key research topic. The vast - and frequently complete - field of view provided by such panoramic photographs makes them appealing for a variety of applications, including robots, autonomous vehicles, and virtual reality. Consumer-level camera systems capable of capturing such images are likewise growing more affordable, and may be desirable complements to autonomous systems' sensor packages. They do, however, introduce significant distortions and violate some assumptions regarding perspective view images. Additionally, many state-of-the-art algorithms are not designed for its projection model, and their depth estimation performance tends to degrade when being applied to panoramic imagery. This paper presents a novel technique for adapting view synthesis-based depth estimation models to omnidirectional vision. Specifically, we: 1) integrate a “virtual” spherical camera model into the training pipeline, facilitating the model training, 2) exploit spherical convolutional layers to perform convolution operations on equirectangular images, handling the severe distortion, and 3) propose an optical flow-based masking scheme to mitigate the effect of unwanted pixels during training. Our qualitative and quantitative results demonstrate that these simple yet efficient designs result in significantly improved depth estimations when compared to previous approaches.

This article presents the novel modular version of the robotic platform Cellulo, a versatile handheld robot initially designed as an educational robot. The use of Cellulo in different contexts and applications over the years has highlighted the need for modularity. Modularity adds versatility by increasing the spectrum of functionalities of the robot, as well as more robustness. Modulo Cellulo consists of three modules: a main module, a battery module, and an interaction module. We describe the new Modulo Cellulo platform, the different modules design, the mechanical and electrical inter-connectivity between them, the new adaptive controller, and the application development framework. As a show case, we present the addition of the reconfigurable robot Mori as a module for Cellulo, in an activity envisioning the collaboration between reconfigurable swarm robots.

Modular robots (MRs) consist of unique robots which interconnect and work as a collective to perform objectives. Coordinating these robots rely on robust communication, as modules moving independently can lead to damaging behaviour. We present a robust structure for modular robot communication, implemented and tested on a new MR. The structure has different communication protocols depending on the importance and bandwidth of the exchanged information, has fast error responses, and considerations which allow for two modules to actuate the same joint. We evaluate the wireless protocols, novel error response, and coordinated actuation empirically, validating the system on a new modular robot, the Mori3. We find two wireless protocols can be used to balance speed and reliability; transmitting errors through both wireless and serial is more consistent and faster; and sharing motor targets, control variables, and measurements allow for motors to operate a shared joint with equal efforts.

Modeling closed loop mechanisms is a necessity for the control and simulation of various systems and poses a great challenge to rigid body dynamics algorithms. Solving the forward and inverse dynamics for such systems require resolution of loop closure constraints which are often solved via numerical procedures. This brings an additional burden to these algorithms as they have to stabilize and control the loop closure errors. In order to avoid this issue, analytical solutions are preferred for commonly studied parallel mechanisms. This paper has two contributions. Firstly, it reports a case study on a modular and hybrid numerical-analytical approach to model and control series-parallel hybrid robots which are subjected to large number of holonomic constraints. The approach exploits the modularity in the robot design to combine analytical loop closure for the known submechanisms and numerical loop closure for submechanisms where analytical solutions are not available. This offers an edge over purely numerical approach in terms of computational efficiency. Secondly, an adaption of the constraint embedding approach in Articulated Body Algorithm (ABA) is presented which yields a recursive algorithm in minimal coordinates for computing the forward dynamics of series-parallel hybrid systems. The proposed modification exploits the Lie group formulations and allows easy implementation of recursive forward dynamics of constrained systems in state of the art multi-body solvers.

This paper proposes novel passive-dynamic walk-ers formed by two cross-shaped frames and eight viscoelastic elements. Since it is a combination of two four-legged rimless wheels via viscoelastic elements, we call it viscoelastically-combined rimless wheel (VCRW). Two types of VCRWs consisting of different cross-shaped frames are introduced; one is formed by combining two Greek-cross-shaped frames (VCRW1), and the other is formed by combining two-link cross-shaped frames that can rotate freely around the central axis (VCRW2). First, we describe the model assumptions and equations of motion and collision. Second, we numerically analyze the basic gait properties of passive dynamic walking. Furthermore, we consider an activation of VCRW2 for gen-erating a stable level gait, and discuss the significance of the study as a novel walking support device.

Sub-structures such as micro-structured magnetic teeth fabricated with an artificial bacteria flagellum (ABF) are designed for achieving more motion modes, higher precision, and better controllability. To achieve these, a more precise model considering the non-circular cross-sectional features is setup without simplifying the structure as a helical filament with a circular cross-section as having been used in previous investigations, making it possible to include the effects of the substructures into the motion equation. Analyses and experiments verified the correctness. Besides of the geometric effects, our experimental observation also shows an anomalous step-out frequency appeared in an ABF. This asynchronous motion is attributed to the lag of magnetization with respect to the external rotating magnetic field due to the geometries and the soft-magnetic materials of the ribbons, which is different from the regular asynchronous motion solely caused by low Reynolds number of fluid to microscopic swimmers. While the lag of magnetization can be further attributed initiatively to the soft magnetic materials adopted, the feasibility to arrange the easy axis will enable many new possibilities, which is of particular interest in generating more modes for swarms such as cascade stepping out of ABFs with the same nominal overall sizes and for more precise positioning using stepping motion.

In human-robot collaboration, robot errors are inevitable—damaging user trust, willingness to work together, and task performance. Prior work has shown that people naturally respond to robot errors socially and that in social interactions it is possible to use human responses to detect errors. However, there is little exploration in the domain of nonsocial, physical human-robot collaboration such as assembly and tool retrieval. In this work, we investigate how people's organic, social responses to robot errors may be used to enable timely automatic detection of errors in physical human-robot interactions. We conducted a data collection study to obtain facial responses to train a real-time detection algorithm and a case study to explore the generalizability of our method with different task settings and errors. Our results show that natural social responses are effective signals for timely detection and localization of robot errors even in nonsocial contexts and that our method is robust across a variety of task contexts, robot errors, and user responses. This work contributes to robust error detection without detailed task specifications.

Magnetically-actuated flexible endoscopes (MAFE) have been well used in minimally-invasive surgery because they can be steered by a magnetic field thus more flexible than traditional endoscopes. Model-free and uncalibrated visual-feedback control makes it possible to manipulate MAFE with a magnetic field without external tracking systems. Because no extra sensor is required to obtain position and posture information, the size of MAFE can be made smaller. However, the traditional control method focuses on 2DoF control, which lacks control over the posture of the end of MAFE. This may result in unnecessary contact between MAFE and tissue and cause injury during the advancement of the endoscope. In this letter, we propose algorithms to enhance the pose control of MAFE to 4DoF and 5DoF based on model-free and uncalibrated visual-feedback control. Experiments in structured environments verify that the control algorithms are able to realize 4DoF manual navigation and 5DoF automatic navigation.

Model-free Deep Reinforcement Learning (DRL) controllers have demonstrated promising results on various challenging non-linear control tasks. While a model-free DRL algorithm can solve unknown dynamics and high-dimensional problems, it lacks safety assurance. Although safety constraints can be encoded as part of a reward function, there still exists a large gap between an RL controller trained with this modified reward and a safe controller. In contrast, instead of implicitly encoding safety constraints with rewards, we explicitly colearn a Twin Neural Lyapunov Function (TNLF) with the control policy in the DRL training loop and use the learned TNLF to build a runtime monitor. Combined with the path generated from a planner, the monitor chooses appropriate waypoints that guide the learned controller to provide collision-free control trajectories. Our approach inherits the scalability advantages from DRL while enhancing safety guarantees. Our experimental evaluation demonstrates the effectiveness of our approach compared to DRL with augmented rewards and constrained DRL methods over a range of high-dimensional safety-sensitive navigation tasks.