IROS 2023 论文列表

This paper presents the development of a simulation for dynamic plant models, generated from L-system functional models. A key application of these dynamic plant models is to aid in developing new methods for robotic manipulation of plants that minimize damage due to physical interaction. We present a use case of the dynamic plant model by evaluating its performance against standard RRT and novel keyhole robot arm pruning algorithms in comparison with a physical plant. Through this paper, we show that the simulated plant model was able to predict the failure modes for each pruning algorithm. The dynamic plant model was also able to predict the performance difference between algorithms, simulated experiments predicting an increase in target point capture success rate from 57% RRT to 90% keyhole compared with 65% RRT to 86% keyhole when applied to a physical sample; thus validating it as a useful simulation tool for developing and testing novel robotic methods for plant handling within cluttered and unstructured environments.

In this article, we present a mechanism and related path planning algorithm to construct light-duty barriers out of extruded, inflated tubes weaved around existing environmental features. Our extruded tubes are based on everted vine-robots and in this context, we present a new method to steer their growth. We characterize the mechanism in terms of accuracy resilience, and, towards their use as barriers, the ability of the tubes to withstand distributed loads. We further explore an algorithm which, given a feature map and the size and direction of the external load, can determine where and how to extrude the barrier. Finally, we showcase the potential of this method in an autonomously extruded two-layer wall weaved around three pipes. While preliminary, our work indicates that this method has potential for barrier construction in cluttered environments, e.g. shelters against wind or snow. Future work may show how to achieve tighter weaves, how to leverage weave friction for improved strength, how to assess barrier performance for feedback control, and how to operate the extrusion mechanism off of a mobile robot.

The purpose of this research is to study the impact of robot participation in group conversations and assess the effectiveness of different addressing policies. The study involved a total of 300 participants, who were divided into groups of four and engaged in a dialogue with a humanoid robot. The robot acted as a moderator, using information obtained during the conversation to determine which speaker to address. The study found that the policy used by the robot significantly impacted the conversation dynamics. Specifically, the robot provided more balanced attention to each participant and reduced the number of subgroups.

Using robots to collect data is an effective way to obtain information from the environment and communicate it to a static base station. Furthermore, robots have the capability to communicate with one another, potentially decreasing the time for data to reach the base station. We present a Mixed Integer Linear Program that reasons about discrete routing choices, continuous robot paths, and their effect on the latency of the data collection task. We analyze our formulation, discuss optimization challenges inherent to the data collection problem, and propose a factored formulation that finds optimal answers more efficiently. Our work is able to find paths that reduce latency by up to 101% compared to treating all robots independently in our tested scenarios.

Though mopping the floor is a mundane and tedious daily task, enabling robots to perform it comparably to humans remains a challenge. Hand-coding desired mopping behaviors for variable surfaces and situations is particularly difficult. In this paper, we develop a robotic system for mopping the floor by mimicking the human behavior demonstrated in videos. Our baseline robotic system uses traditional computer vision techniques for tracking and inverse kinematics. Our proposed robot mop learning system comprises advanced computer vision techniques, Time Contrastive Network (TCN), and reinforcement learning. Using these, we devise a reward function for the mopping task. We use a Universal 10e robotic arm attached to a mop to perform the mopping task and a first-person camera attached on top of the robotic arm to provide feedback for robotic learning. We evaluate our proposed robot mop learning system's imitative similarity using optical flow, distance in mop location, and force applied to the floor, as well as cleaning efficiency using a white glove method.

For letting mobile robots travel flexibly through complicated environments, increasing attention has been paid to the whole-body collision evaluation. Most existing works either opt for the conservative corridor-based methods that impose strict requirements on the corridor generation, or ESDF-based methods that suffer from high computational overhead. It is still a great challenge to achieve fast and accurate whole-body collision evaluation. In this paper, we propose a Robo-centric ESDF (RC-ESDF) that is pre-built in the robot body frame and is capable of seamlessly applied to any-shape mobile robots, even for those with non-convex shapes. RC-ESDF enjoys lazy collision evaluation, which retains only the minimum information sufficient for whole-body safety constraint and significantly speeds up trajectory optimization. Based on the analytical gradients provided by RC-ESDF, we optimize the position and rotation of robot jointly, with whole-body safety, smoothness, and dynamical feasibility taken into account. Extensive simulation and real-world experiments verified the reliability and generalizability of our method.

Inspired by human proteins that are synthesized from only 20 types of amino acids, the development of self-assembly methods that allow robots to be built simply by randomly stirring the parts has been explored for many years. The key challenges include how to synthesize parts in pieces into a three-dimensional functional structure in a practical time, and subsequently, achieve a controlled robotic motion, all with minimal human intervention. This study proposes a method of self-assembling a 3D robot by first self-assembling random parts into a 2D structure and then self-folding it into a 3D shape. Once self-folded, the robot, whose compositional parts contain magnets, becomes capable of performing basic tasks such as block-pushing upon an application of an external magnetic field. Self-assembly from parts into a two-dimensional structure was performed by repeatedly colliding the parts with each other, and combining them with complementary-shaped parts, like matching jigsaw puzzle pieces. Self-folding was performed by shrinking a heat-responsive film attached across the hinge of each assembly part in hot water, causing the entire 2D structure to self-fold. The experiment demonstrated a series of 13 parts self-assembling into the shape of a 3D beetle, then walking and pushing an object in 13 minutes. The self-assembly process is programmed (mechanically) to generate the same geometry even if the number of parts is greater than the necessary number for the structure, thus is capable of generating multiple structures simultaneously.

Effective coordination of heterogeneous multi-robot teams requires optimizing allocations, schedules, and motion plans in order to satisfy complex multi-dimensional task requirements. This challenge is exacerbated by the fact that real-world applications inevitably introduce uncertainties into robot capabilities and task requirements. In this paper, we extend our previous work on trait-based time-extended task allocation to account for such uncertainties. Specifically, we leverage the Sequential Probability Ratio Test to develop an algorithm that can guarantee that the probability of failing to satisfy task requirements is below a user-specified threshold. We also improve upon our prior approach by accounting for temporal deadlines in addition to synchronization and precedence constraints in a Mixed-Integer Linear Programming model. We evaluate our approach by benchmarking it against three baselines in a simulated battle domain in a city environment and compare its performance against a state-of-the-art framework in a pandemic-inspired multi-robot service coordination problem. Results demonstrate the effectiveness and advantages of our approach, which leverages redundancies to manage risk while simultaneously minimizing makespan.

Mobile robot navigation in a human-populated environment has been of great interest to the research community in recent years, referred to as crowd navigation. Currently, offline reinforcement learning (RL)-based method has been introduced to this domain, for its ability to alleviate the sim2real gap brought by online RL which relies on simulators to execute training, and its scalability to use the same dataset to train for differently customized rewards. However, the performance of the navigation policy suffered from the distributional shift between the training data and the input during deployment, since when it gets an input out of the training data distribution, the learned policy has the risk of choosing an erroneous action that leads to catastrophic failure such as colliding with a human. To realize risk sensitivity and improve the safety of the offline RL agent during deployment, this work proposes a multipolicy control framework that combines offline RL navigation policy with a risk detector and a force-based risk-avoiding policy. In particular, a Lyapunov density model is learned using the latent feature of the offline RL policy and works as a risk detector to switch the control to the risk-avoiding policy when the robot has a tendency to go out of the area supported by the training data. Experimental results showed that the proposed method was able to learn navigation in a crowded scene from the offline trajectory dataset and the risk detector substantially reduces the collision rate of the vanilla offline RL agent while maintaining the navigation efficiency outperforming the state-of-the-art methods.

Improving the safety and efficiency of maritime shipping has the potential to reduce carbon emissions and improve profitability. Stochastic ship routing, the problem of finding a safe, efficient, and timely route for a ship is difficult in large part because of uncertainty in weather forecasts, which come as an ensemble, a collection of many possible future weather conditions. Previous safety-aware ship routing methods have used either conservative interpretations of the ensemble by assuming the worst-case weather conditions, leading to excessive fuel consumption, or by using the average weather conditions, leading to potentially unsafe routes. In this paper, we investigate the use of the well-known Conditional Value-at-risk (CVaR) in the objective and constraint functions for ship routing problems, which allows a range of risk tolerances between the average and the worst case. We illustrate the advantages of using CVaR for the problem of ship routing in several simulation examples using real weather forecasts.

Decentralized control schemes are increasingly favored in various domains that involve multi-agent systems due to the need for computational efficiency as well as general applicability to large-scale systems. However, in the absence of an explicit global coordinator, it is hard for distributed agents to determine how to efficiently interact with others. In this paper, we present a risk-aware decentralized control framework that provides guidance on how much relative responsibility share (a percentage) an individual agent should take to avoid collisions with others while moving efficiently without direct communications. We propose a novel Control Barrier Function (CBF)-inspired risk measurement to characterize the aggregate risk agents face from potential collisions under motion uncertainty. We use this measurement to allocate responsibility shares among agents dynamically and develop risk-aware decentralized safe controllers. In this way, we are able to leverage the flexibility of robots with lower risk to improve the motion flexibility for those with higher risk, thus achieving improved collective safety. We demonstrate the validity and efficiency of our proposed approach through two examples: ramp merging in autonomous driving and a multi-agent position-swapping game.

An in-flight loss of thrust poses a risk to the aircraft, its passengers, and people on the ground. When a loss of thrust happens, the (auto)pilot is forced to perform an emergency landing, possibly toward one of the reachable airports. If none of the airports is reachable, the aircraft is forced to land at another location, which can be risky in urban environments. In this work, we present a generalization of the previous work on planning safe emergency landing in the case of in-flight loss of thrust such that the risk induced by the loss of thrust can be assessed if none of the airports are reachable. The proposed method relies on planning space discretization and efficient risk propagation through the risk map. The approach can find the least risky landing site and corresponding forced landing trajectory for any configuration in the planning space. The method has been empirically evaluated in a realistic urban scenario. The results support its suitability for risk-aware planning of an emergency landing in the case of in-flight loss of thrust.

Trust-aware human-robot interaction (HRI) has received increasing research attention, as trust has been shown to be a crucial factor for effective HRI. Research in trust-aware HRI discovered a dilemma - maximizing task rewards often leads to decreased human trust, while maximizing human trust would compromise task performance. In this work, we address this dilemma by formulating the HRI process as a two-player Markov game and utilizing the reward-shaping technique to improve human trust while limiting performance loss. Specifically, we show that when the shaping reward is potential-based, the performance loss can be bounded by the potential functions evaluated at the final states of the Markov game. We apply the proposed framework to the experience-based trust model, resulting in a linear program that can be efficiently solved and deployed in real-world applications. We evaluate the proposed framework in a simulation scenario where a human-robot team performs a search-and-rescue mission. The results demonstrate that the proposed framework successfully modifies the robot's optimal policy, enabling it to increase human trust at a minimal task performance cost.

Dynamic vision sensors or event cameras provide rich complementary information for video frame interpolation. Existing state-of-the-art methods follow the paradigm of combining both synthesis-based and warping networks. However, few of those methods fully respect the intrinsic characteristics of events streams. Given that event cameras only encode intensity changes and polarity rather than color intensities, estimating optical flow from events is arguably more difficult than from RGB information. We therefore propose to incorporate RGB information in an event-guided optical flow refinement strategy. Moreover, in light of the quasi-continuous nature of the time signals provided by event cameras, we propose a divide-and-conquer strategy in which event-based intermediate frame synthesis happens incrementally in multiple simplified stages rather than in a single, long stage. Extensive experiments on both synthetic and real-world datasets show that these modifications lead to more reliable and realistic intermediate frame results than previous video frame interpolation methods. Our findings underline that a careful consideration of event characteristics such as high temporal density and elevated noise benefits interpolation accuracy.

Depth completion, which aims to generate high-quality dense depth maps from sparse depth maps, has attracted increasing attention in recent years. Previous work usually employs RGB images as guidance, and introduces iterative spatial propagation to refine estimated coarse depth maps. However, most of the propagation refinement methods require several iterations and suffer from a fixed receptive field, which may contain irrelevant and useless information with very sparse input. In this paper, we address these two challenges simultaneously by revisiting the idea of deformable convolution. We propose an effective architecture that leverages deformable kernel convolution as a single-pass refinement module, and empirically demonstrate its superiority. To better understand the function of deformable convolution and exploit it for depth completion, we further systematically investigate a variety of representative strategies. Our study reveals that, different from prior work, deformable convolution needs to be applied on an estimated depth map with a relatively high density for better performance. We evaluate our model on the large-scale KITTI dataset and achieve state-of-the-art level performance in both accuracy and inference speed. Our code is available at https://github.com/AlexSunNiklReDC.

High altitude balloons have proved useful for ecological aerial surveys, atmospheric monitoring, and communication relays. However, due to weight and power constraints, there is a need to investigate alternate modes of propulsion to navigate in the stratosphere. Very recently, reinforcement learning has been proposed as a control scheme to maintain balloons in the region of a fixed location, facilitated through diverse opposing wind-fields at different altitudes. Although air-pump based station keeping has been explored, there is no research on the control problem for venting and ballasting actuated balloons, which is commonly used as a low-cost alternative. We show how reinforcement learning can be used for this type of balloon. Specifically, we use the soft actor-critic algorithm, which on average is able to station-keep within 50 km for on average 25% of the flight, consistent with state-of-the-art. Furthermore, we show that the proposed controller effectively minimises the consumption of resources, thereby supporting long duration flights. We frame the controller as a continuous control reinforcement learning problem, which allows for a more diverse range of trajectories, as opposed to current state-of-the-art work, which uses discrete action spaces. Furthermore, through continuous control, we can make use of larger ascent rates which are not possible using air-pumps. The desired ascent-rate is decoupled into desired altitude and time-factor to provide a more transparent policy, compared to low-level control commands used in previous works. Finally, by applying the equations of motion, we establish appropriate thresholds for venting and ballasting to prevent the agent from exploiting the environment. More specifically, we ensure actions are physically feasible by enforcing constraints on venting and ballasting.

This paper revisits Kimera-Multi, a distributed multi-robot Simultaneous Localization and Mapping (SLAM) system, towards the goal of deployment in the real world. In particular, this paper has three main contributions. First, we describe improvements to Kimera-Multi to make it resilient to large-scale real-world deployments, with particular emphasis on handling intermittent and unreliable communication. Second, we collect and release challenging multi-robot benchmarking datasets obtained during live experiments conducted on the MIT campus, with accurate reference trajectories and maps for evaluation. The datasets include up to 8 robots traversing long distances (up to 8 km) and feature many challenging elements such as severe visual ambiguities (e.g., in underground tunnels and hallways), mixed indoor and outdoor trajectories with different lighting conditions, and dynamic entities (e.g., pedestrians and cars). Lastly, we evaluate the resilience of Kimera-Multi under different communication scenarios, and provide a quantitative comparison with a centralized baseline system. Based on the results from both live experiments and subsequent analysis, we discuss the strengths and weaknesses of Kimera-Multi, and suggest future directions for both algorithm and system design. We release the source code of Kimera-Multi and all datasets to facilitate further research towards the reliable real-world deployment of multi-robot SLAM systems.

The light and soft characteristics of Buoyancy Assisted Lightweight Legged Unit (BALLU) robots have a great potential to provide intrinsically safe interactions in environments involving humans, unlike many heavy and rigid robots. However, their unique and sensitive dynamics impose challenges to obtaining robust control policies in the real world. In this work, we demonstrate robust sim-to-real transfer of control policies on the BALLU robots via system identification and our novel residual physics learning method, Environment Mimic (EnvMimic). First, we model the nonlinear dynamics of the actuators by collecting hardware data and optimizing the simulation parameters. Rather than relying on standard supervised learning formulations, we utilize deep reinforcement learning to train an external force policy to match real-world trajectories, which enables us to model residual physics with greater fidelity. We analyze the improved simulation fidelity by comparing the simulation trajectories against the real-world ones. We finally demonstrate that the improved simulator allows us to learn better walking and turning policies that can be successfully deployed on the hardware of BALLU.

In order to guarantee autonomous vehicles' autonomy, mobility, and ride quality in rugged environments, a real-time speed planning method based on the time-frequency transformation of terrain characteristics is designed to achieve adaptive speed planning of autonomous vehicles in rough ground. On the one hand, the vertical profile of the lidar's point cloud data is converted from the time domain to the frequency domain in real time, and the integrated area of the sub-frequency range in the frequency domain is chosen as the relative roughness quantification value to realize the roughness quantification under various terrains. On the other hand, to model the relationship between vehicle speed and relative roughness, iterative search is utilized to create a speed and roughness model, and sliding windows are employed to update the roughness to achieve continuous mapping between speed and roughness. Ultimately, a number of tests were conducted on various rough roads using the oil exploration vehicle EV-56 as the study object. The experimental results show that the proposed method can identify the terrain roughness changes under complex terrain and change their speed within 0.2 m accuracy.