IROS 2022 论文列表

Leveraging both the autonomy of robots and the expert knowledge of humans can enable a multi-robot system to complete missions in challenging environments with a high degree of adaptivity and robustness. This paper proposes a multi-modal task-based graphical user interface for controlling a heterogeneous multi-robot team. The core of the interface is an integrated multi-robot task allocation system to allow the user to encode his/her intents to guide the heterogeneous multi-robot team. The design of the interface aims to provide the human operator continuous situational awareness and effective control for rapid decision-making in time-critical missions. Team CSIRO Data61 came in second place utilizing this interface for the DARPA Subterranean (SubT) Challenge. The ideas used for this user interface can apply to other multi-robot applications.

Grasping of objects using multi-fingered robotic hands often fails due to small uncertainties in the hand motion control and the object's pose estimation. To tackle this problem, we propose a grasping adjustment strategy based on tactile seroving. Our technique employs feedback from a sensorized multi-fingered robotic hand to collaboratively servo the fingers and palm to achieve the desired grasp. We demonstrate the performance of our method through simulation and physical experiments by having a robot grasp different objects under conditions of variable uncertainty. The results show that our approach achieved a higher success rate and tolerated greater uncertainty than an open-looped grasp.

This paper presents a novel design of a multi-directional bicycle robot, which is developed for the inspection of steel structures, in particular, steel-reinforced bridges. The locomotion concept is based on arranging two magnetic wheels in a bicycle-like configuration with two independent steering actuators. This configuration allows the robot to possess multi-directional mobility. An additional free joint helps the robot adapt naturally to non-flat and complex steel structures. The robot's design provides the advantage of being mechanically simple and providing high-level mobility across diverse steel structures. In addition, a visual sensor is equipped that allows the data collection for steel defect detection with offline training and validation. The paper also provides a novel pipeline for Steel Defect Detection, which utilizes multiple datasets (one for training and one for validation) from real bridges. The quantitative results have been reported for three Deep Encoder-Decoder Networks (i.e., LinkNet, UNet, DeepLab) with their corresponding Encoder modules (i.e., ResNet-18, ResNet-34, RegNet-X2, EfficientNet-B0, and EfficientNet-B2). Due to space concerns, the qualitative results have been outlined in Appendix, with a link in Fig. 11 caption to access the result provided.

This paper presents reorientation manoeuvres applied to an omnidirectional wheeled robot for impact mitigation during short falls. The proposed robot architecture aims to build upon recent innovations in reorientation robots to attain fast, multi-axis reorientation. Indeed, the use of omnidirectional wheels allows for simplifications to be made with respect to previous mobile robot architectures that make the proposed architecture more efficient for free fall reorientation, while still maintaining free roaming capabilities. To test these improvements, a prototype is built and a free roam and two free fall demonstrations are completed. On the one hand, the free roam demonstration validates that translation along both horizontal axes and rotation about the yaw axis are achieved with the presented prototype. On the other hand, the first free fall demonstration shows that a worst case scenario of a 180-degree reorientation about one axis can be completed in just under 0.45 seconds (one-metre fall) and the second free fall demonstration validates that the prototype is capable of simultaneous reorientation about both the roll and pitch axes. Therefore, the fast, multi-axis reorientation capabilities of the developed prototype are verified.

This paper introduces a novel deep framework for dense 3D reconstruction from multiple image frames, leveraging a sparse set of depth measurements gathered jointly with image acquisition. Given a deep multi-view stereo network, our framework uses sparse depth hints to guide the neural network by modulating the plane-sweep cost volume built during the forward step, enabling us to infer constantly much more accurate depth maps. Moreover, since multiple viewpoints can provide additional depth measurements, we propose a multi-view guidance strategy that increases the density of the sparse points used to guide the network, thus leading to even more accurate results. We evaluate our Multi-View Guided framework within a variety of state-of-the-art deep multi-view stereo networks, demonstrating its effectiveness at improving the results achieved by each of them on BlendedMVG and DTU datasets.

In this paper, we propose a novel distributed method based on attention-based deep reinforcement learning using individual reward shaping, for multiple unmanned aerial vehicles (UAVs) cooperative short-range combat mission. Specifically, a two-level attention distributed policy, composed of observation-level and communication-level attention networks, is designed to enable each UAV to selectively focus on important environmental features and messages, for enhancing the effectiveness of the cooperative policy. Moreover, due to the high complexity and stochasticity of the UAV combat mission, the learning of UAVs is tricky and low efficient. To embed knowledge to accelerate the policy learning, a potential-based individual reward function is constructed by implicitly translating the individual reward into the specific form of dynamic action potentials. In addition, an actor-critic training algorithm based on the centralized training and decentralized execution framework is adopted to train the policy network of UAV maneuver decision. We build a three-dimensional UAV simulation and training platform based on Unity for multi-UAV short-range combat missions. Simulation results demonstrate the effectiveness of the proposed method and the superiority of the attention policy and individual reward shaping.

Semantic segmentation provides scene understanding capability by performing pixel-wise classification of objects within an image. However, the sensitivity of such algorithms towards domain changes requires fine-tuning using an annotated dataset for each novel domain, which is expensive to construct and inefficient. We highlight that irrespective of the training dataset, structural properties of scenes remain the same hence domain sensitivity arises from training methodology. Thus, in this paper, we propose a domain alignment approach wherein multiple synthetic source domains are used to train an underlying segmentation network such that it performs consistently in unknown real target domains. Towards this end, we propose a pixel-wise supervised contrastive learning framework that enforces constraints in latent space resulting in features belonging to the same class being clustered closely and away from different classes. This approach allows for better capturing of global and local semantics while providing domain invariant properties. Our approach can be easily incorporated into prior semantic segmentation approaches without the significant computational overhead. We empirically demonstrate the efficacy of the proposed approach on GTAV → Cityscapes, GTAV+Synthia → Cityscapes, and GTAV+Synthia+Synscapes → Cityscapes scenarios and report state-of-the-art (SoTA) performance without requiring access to images from the target domain.

Neural Networks are the state-of-the-art technology for environmental perception in applications such as autonomous driving. However, they require a large amount of training data in order to perform well, making the selection and annotation of sensor data a time-consuming and expensive task. Active learning is a promising approach to reduce the required amount of training data by selecting samples for annotation that are expected to improve the neural network the most. In this work, we propose a sequence-based active learning approach that selects sequences of consecutive frames instead of individual images. This allows to evaluate tracking algorithms and to reduce the annotation effort by interpolating labels between frames. Our approach is compared to a random sampling strategy as baseline. Over 15 iterations, both approaches select 1000 additional images for training in each iteration. The performance of the neural network trained on the data selected by our sequence-based active learning approach is compared to the performance of the network trained on the data select by the baseline approach. The results show that sequence-based active learning can reduce the required amount of training data by up to 25% while reaching a similar performance. Furthermore, sequence-based active learning can improve the neural network's overall performance by 2 % compared to a random sampling strategy. In this work, the proposed method was evaluated with a new dataset consisting of 15 scenes in railway environments. The dataset has 45,888 frames in total, 14,513 frames contain persons on railway stations or close to tracks.

The depth completion task aims to predict a dense depth map from a sparse LiDAR point cloud and an RGB image. This task is critical because an accurate depth map can be used as prior information to solve many computer vision tasks, such as downstream tasks in autonomous vehicles and robot vision. Previous deep learning methods which focus on the local affinity have achieved impressive results. However, an architecture that is directly designed to extract local affinity has not been proposed yet. In this paper, we propose multi-scaled and densely connected locally convolutional layers to learn the affinity of the neighborhood. We set a different grid factor for each step of this module, and each step consists of several convolutional layers applied only to the local area assigned from the grid factor. In addition, each step is densely connected, sequentially, to take advantage of the multi-scale receptive fields. The proposed module effectively learns the neighbor-hood's affinity in a local area with multiple scales, while keeping the network size small. As a result, our architecture achieves state-of-the-art performance compared to published works on the KITTI depth completion benchmark. On the NYU Depth V2 completion benchmark our method achieves performance comparable to state-of-the-art approaches.

This paper presents a novel approach for multi-robot unknown area exploration. Recently, the frontier tree data structure was used in single robot exploration to memorize frontiers, their positions, exploration state, and the map. This tree could be queried to decide on further exploration steps. In this paper, we take the concept further for multi-robot exploration by proposing a new abstraction called the ‘group,’ meant to share information through a common frontier tree, requisite operations at the group level, and a method to assign goals to multiple robots. A group is a set of robots, the union of whose explored regions forms a contiguous region (a single connected region in a topological sense). As a group has precisely one tree, the robots share a common state of the exploration task. We propose techniques to merge groups and their frontier trees once their maps overlap. Finally, we suggest a method to designate and assign exploration goals to the individual robots by choosing nodes from the frontier tree. The proposed approach outperforms seven state-of-the-art research works in simulation.

We present a centralized algorithm for labeled, disk-shaped Multi-Robot Path Planning (MPP) in a continuous planar workspace with polygonal boundaries. Our method automatically transform the continuous problem into a discrete, graph-based variant termed the pebble motion problem, which can be solved efficiently. To construct the underlying pebble graph, we identify inscribed circles in the workspace via a medial axis transform and organize robots into layers within each inscribed circle. We show that our layered pebble-graph enables collision-free motions, allowing all graph-restricted MPP instances to be feasible. MPP instances with continuous start and goal positions can then be solved via local navigations that route robots from and to graph vertices. We tested our method on several environments with high robot-packing densities (up to 61.6% of the workspace). For environments with narrow passages, such density violates the well-separated assumptions made by state-of-the-art MPP planners, while our method achieves an average success rate of 83%.

Motivated by the use of robots for pest control in agriculture, this work introduces the Multi-Robot Dynamic Swarm Disablement problem, in which a team of robots is required to disable a swarm of agents (for example, locust agents) passing through an area while minimizing the cumulative time of the swarm members (equivalent to the cumulative damage they cause) in the area. Showing that the problem is hard even in naive settings, we turn to examine algorithms seeking to optimize the robots' performance against the swarm by exploiting the known movement pattern of the swarm agents. Motivated by the poor performance when a weak group of robots attempts to catch a large swarm of agents, whether it is a significant numerical minority or poor speed gaps, we suggest the use of blocking lines: the robots form lines that block the agents along their movement in the environment. We show by both theoretical analysis and rigorous empirical evaluation in different settings that these algorithms outperform common task-assignment-based algorithms, especially for limited robots versus a large swarm.

Virtual Fixtures facilitate teleoperation, for in-stance by guiding the human operator. Developing these Virtual Fixtures in tasks with tight tolerances remains challenging. Fixtures with a high stiffness allow for more precise guidance, whereas a lower stiffness is required to allow for corrections. We observed that many assembly operations can be split into different phases - approaching, positioning, in-contact manipulation - each with different accuracy requirements. Therefore, we propose to use multi-modal fixtures, satisfying the different requirements of these phases: i.e. a position-based Trajectory Fixture for approaching and a more accurate Visual Servoing Fixture for the positioning phase. A state estimation and arbitration component ensures smooth transitions between the fixtures to provide optimal support for the operator and to achieve global availability paired with local precision at the same time. It also allows a high stiffness to be used throughout, thus achieving good guidance for all phases. The approach is validated in an application from a space scenario, consisting of the assembly of a CubeSat subsystem. The empirical results from a pilot study on this task show that our approach is faster and requires less interaction force from the operator than the baseline method.

Multi-agent systems are deployed to accomplish tasks that take a long time with a single agent. The task allocation problem becomes particularly difficult when the objectives are conflicting with one another (e.g. minimizing the mission time while respecting the task priorities, while simultaneously maximizing agent's fitness for the task). This paper presents an algorithm to create task assignments for a group of autonomous agents with competing objectives. We consider a variety of constraints including agent capabilities to perform tasks, priorities set by a human supervisor, as well as temporal constraints such as arrival time or coalition formation. We propose a multi-objective Particle Swarm Optimization (PSO) that uses a hierarchical cost function by leveraging the paradigm of lexicographic optimization. The particles are driven by higher ranked objectives with lower ranked objectives used to break ties. We demonstrate the effectiveness of this algorithm in a battlefield scenario where sub-teams of aerial vehicles are assigned to perform area reconnaissance, target strikes, and intelligence gathering.

Multi-objective optimization models that encode ordered sequential constraints provide a solution to model various challenging problems including encoding preferences, modeling a curriculum, and enforcing measures of safety. A recently developed theory of topological Markov decision processes (TMDPs) captures this range of problems for the case of discrete states and actions. In this work, we extend TMDPs towards continuous spaces and unknown transition dynamics by formulating, proving, and implementing the policy gradient theorem for TMDPs. This theoretical result enables the creation of TMDP learning algorithms that use function approximators, and can generalize existing deep reinforcement learning (DRL) approaches. Specifically, we present a new algorithm for a policy gradient in TMDPs by a simple extension of the proximal policy optimization (PPO) algorithm. We demonstrate this on a real-world multiple-objective navigation problem with an arbitrary ordering of objectives both in simulation and on a real robot.

The performance of a robot is closely related to its structure. From the initial design of link lengths to structural optimization, it is still the research hotspot in recent years. To make the manipulator lightweight and ensure its working range and flexibility, researchers have proposed many optimization methods, most of which are for specific working scenarios, requirements, and robot structures, therefore their generality is limited. The optimization of the manipulator should be a comprehensive method. That is, we should pay attention to the joint configuration and each link length at the beginning of the design. Particularly, the geometric parameters of each link, which not only affect the range of the workspace but also have a direct impact on the working space, working efficiency, and flexibility of the manipulator. In this paper, a generalized optimization framework is proposed for multi-link manipulators. Starting from the optimization of manipulator link lengths, firstly, the geometry of the manipulator and workspace is parameterized; then the performance indicators are established; lastly, the geometric size of the manipulator is optimized according to the workspace limits and task requirements. Besides, we verified its feasibility and generality by applying this method to different TBM scenarios.

In a typical fulfillment center, the order fulfilling process is managed by a warehouse management system (WMS). For efficiency, WMS usually applies batch picking, also called multi-order picking, to collect the same items for multiple orders. Suppose an item appears in multiple orders, instead of repeatedly revisiting the exact picking location multiple times, a picker will be instructed to pick up multiple same items at once and bring them to a sorting station, also called a re-bin station. It is at the re-bin station, where the workers sort the picked items into separate orders. We have seen many robotic technologies being developed for sorting. However, we have not seen any feasible robotic technology for batch picking. Transferring multiple objects between bins is a common task. In robotics, a standard approach is to transfer a single object at a time. However, grasping multiple objects and transferring them at once is more efficient. This paper presents a set of novel strategies for efficiently grasping and transferring multiple objects. The grasping strategies enable a robotic hand to grasp multiple objects by identifying an optimal ready hand configuration (pre-grasp), calculating a flexion synergy based on the desired quantity of objects to be grasped, and utilizing a deep learning model to signal the completion of a grasp. The transferring strategies demonstrate an approach that models the problem as a Markov decision process (MDP) and defines specific grasping actions to efficiently transfer objects when the required quantity is larger than the capability of a single grasp. Using the MDP model, the approach can generate an optimal pick-transfer policy that minimizes the number of transfers. The complete proposed approach has been evaluated in both a simulation environment and on a real robotic system. The proposed approach reduces the number of transfers by 59% and the number of lifts by 58% compared to an optimal single object pick-transfer solution. Show More

In this paper we present a motion planner for LIMMS, a modular multi-agent, multi-modal package delivery platform. A single LIMMS unit is a robot that can operate as an arm or leg depending on how and what it is attached to, e.g., a manipulator when it is anchored to walls within a delivery vehicle or a quadruped robot when 4 are attached to a box. Coordinating amongst multiple LIMMS, when each one can take on vastly different roles, can quickly become complex. For such a planning problem we first compose the necessary logic and constraints. The formulation is then solved for skill exploration and can be implemented on hardware after refinement. To solve this optimization problem we use alternating direction method of multipliers (ADMM). The proposed planner is experimented under various scenarios which shows the capability of LIMMS to enter into different modes or combinations of them to achieve their goal of moving shipping boxes.

Lidar technology has evolved significantly over the last decade, with higher resolution, better accuracy, and lower cost devices available today. In addition, new scanning modalities and novel sensor technologies have emerged in recent years. Public datasets have enabled benchmarking of algorithms and have set standards for the cutting edge technology. However, existing datasets are not representative of the technological landscape, with only a reduced number of lidars available. This inherently limits the development and comparison of general-purpose algorithms in the evolving landscape. This paper presents a novel multi-modal lidar dataset with sensors showcasing different scanning modalities (spinning and solid-state), sensing technologies, and lidar cameras. The focus of the dataset is on low-drift odometry, with ground truth data available in both indoors and outdoors environment with sub-millimeter accuracy from a motion capture (MOCAP) system. For comparison in longer distances, we also include data recorded in larger spaces indoors and outdoors. The dataset contains point cloud data from spinning lidars and solid-state lidars. Also, it provides range images from high resolution spinning lidars, RGB and depth images from a lidar camera, and inertial data from built-in IMUs. This is, to the best of our knowledge, the lidar dataset with the most variety of sensors and environments where ground truth data is available. This dataset can be widely used in multiple research areas, such as 3D LiDAR simultaneous localization and mapping (SLAM), performance comparison between multi-modal lidars, appearance recognition and loop closure detection. The datasets are available at: https://github.com/TIERS/tiers-lidars-dataset.