IROS 2023 论文列表

Deep reinforcement learning (DRL) algorithms have proven effective in robot navigation, especially in unknown environments, by directly mapping perception inputs into robot control commands. However, most existing methods ignore the local minimum problem in navigation and thereby cannot handle complex unknown environments. In this paper, we propose the first DRL-based navigation method modeled by a semi-Markov decision process (SMDP) with continuous action space, named Adaptive Forward Simulation Time (AFST), to overcome this problem. Specifically, we reduce the dimensions of the action space and improve the distributed proximal policy optimization (DPPO) algorithm for the specified SMDP problem by modifying its GAE to better estimate the policy gradient in SMDPs. Experiments in various unknown environments demonstrate the effectiveness of AFST.

We propose an automata-theoretic approach for reinforcement learning (RL) under complex spatio-temporal constraints with time windows. The problem is formulated using a Markov decision process under a bounded temporal logic constraint. Different from existing RL methods that can eventually learn optimal policies satisfying such constraints, our proposed approach enforces a desired probability of constraint satisfaction throughout learning. This is achieved by translating the bounded temporal logic constraint into a total automaton and avoiding “unsafe” actions based on the available prior information regarding the transition probabilities, i.e., a pair of upper and lower bounds for each transition probability. We provide theoretical guarantees on the resulting probability of constraint satisfaction. We also provide numerical results in a scenario where a robot explores the environment to discover high-reward regions while fulfilling some periodic pick-up and delivery tasks that are encoded as temporal logic constraints.

The majority of current research on reinforcement learning (RL) for snake robot control do not sufficiently account for the spatial and temporal dependencies within the robot or its interaction with its environment during movement. To address this issue, we propose an RL based multi-layer Bayesian method for autonomous snake robot control, which handles challenging scenarios and improves navigation efficiency. There are three major contributions: 1) An innovative hierarchical Bayesian framework unifies gait control, locomotion control, and stimulus reaction; 2) The dynamics of environment is modeled by density propagation and exploited by an LSTM-based agent to improve the learning process; 3) A stimulus reaction model is derived by combining spatial correlation among robot modules and temporal dependency along time sequence. Comparison experiments with a simulated snake robot show that performance of the proposed approach with challenging obstacles is superior to state-of-the-art baseline.

Motion planning is challenging for multiple robots in cluttered environments without communication, especially in view of real-time efficiency, motion safety, distributed computation, and trajectory optimality, etc. In this paper, a reinforced potential field method is developed for distributed multi-robot motion planning, which is a synthesized design of reinforcement learning and artificial potential fields. An observation embedding with a self-attention mechanism is presented to model the robot-robot and robot-environment interactions. A soft wall-following rule is developed to improve the trajectory smoothness. Our method belongs to reactive planning, but environment properties are implicitly encoded. The total amount of robots in our method can be scaled up to any number. The performance improvement over a vanilla APF and RL method has been demonstrated via numerical simulations. Experiments are also performed using quadrotors to further illustrate the competence of our method.

In this work, we present GraspFlow, a refinement approach for generating context-specific grasps. We formulate the problem of grasp synthesis as a sampling problem: we seek to sample from a context-conditioned probability distribution of successful grasps. However, this target distribution is unknown. As a solution, we devise a discriminator gradient-flow method to evolve grasps obtained from a simpler distribution in a manner that mimics sampling from the desired target distribution. Unlike existing approaches, GraspFlow is modular, allowing grasps that satisfy multiple criteria to be obtained simply by incorporating the relevant discriminators. It is also simple to implement, requiring minimal code given existing auto-differentiation libraries and suitable discriminators. Experiments show that GraspFlow generates stable and executable grasps on a real-world Panda robot for a diverse range of objects. In particular, in 60 trials on 20 different household objects, the first attempted grasp was successful 94% of the time, and 100% grasp success was achieved by the second grasp. Moreover, incorporating a functional discriminator for robot-human handover improved the functional aspect of the grasp by up to 33%.

In this paper, a position-based visual-servoing control approach is introduced for a robotic camera holder to improve ergonomics and reduce mental stress during brain surgery. The visual tracking system controls and moves the robotic camera holder by following a selected surgical instrument without the need for artificial markers. The system was validated using a 7 Degree-of-Freedoms (DoFs) redundant robotic manipulator with an eye-in-hand stereo camera configuration and compared with conventional control methods using NASA TLX survey and four objective metrics, including execution time, time out of field of view (FoV), target score, and path length. Experimental results demonstrate that the proposed system can reduce the surgeon's workload during brain surgery-related task execution, improve ergonomics and achieve higher performance than traditional control methods.

Deep Reinforcement Learning (RL) has shown promise in addressing complex robotic challenges. In real-world applications, RL is often accompanied by failsafe controllers as a last resort to avoid catastrophic events. While necessary for safety, these interventions can result in undesirable behaviors, such as abrupt braking or aggressive steering. This paper proposes two safety intervention reduction methods: proactive replacement and proactive projection, which change the action of the agent if it leads to a potential failsafe intervention. These approaches are compared to state-of-the-art constrained RL on the OpenAI safety gym benchmark and a human-robot collab-oration task. Our study demonstrates that the combination of our method with provably safe RL leads to high-performing policies with zero safety violations and a low number of failsafe interventions. Our versatile method can be applied to a wide range of real-world robotic tasks, while effectively improving safety without sacrificing task performance.

Many planning problems in robotics require long planning horizon and uncertain in nature. The Par-tially Observable Markov Descision Process (POMDP) is a mathematically principled framework for planning under uncertainty. To alleviate the difficulties of computing good approximate POMDP solutions for long horizon problems, one often plans using macro actions, where each macro action is a chain of primitive actions. Such a strategy reduces the effective planning horizon of the problem, and hence reduces the computational complexity for solving. The difficulty is in generating a set of suitable macro actions. In this paper, we present a simple recurrent neural network that learns to generate suitable sets of candidate macro actions that exploits environment information. Key to this learning method is to represent the raw partial information from the environment as a latent problem instance, and sequentially generate macro actions conditioned on the past information. We compare our proposed method with state-of-the-art [1] on four dif-ferent long horizon planning tasks with various difficulties. The results indicate the quality of the policies computed using macro actions generated by our proposed method consistently exceeds benchmarks. Our implementation can be accessed at https://github.com/YC-Liang/Recurrent-Macro-Action-Generator.

Identifying intentions is a critical task for comprehending the actions of others, anticipating their future behavior, and making informed decisions. However, it is challenging to recognize intentions due to the uncertainty of future human activities and the complex influence factors. In this work, we explore the method of recognizing intentions alluded under human behaviors in the real world, aiming to boost intelligent systems' ability to recognize potential intentions and understand human behaviors. We collect data containing real-world human behaviors before using a hand dispenser and a temperature scanner at the building entrance. These data are processed and labeled into intention categories. A questionnaire is conducted to survey the human ability in inferring the intentions of others. Skeleton data and image features are extracted inspired by the answer to the questionnaire. For skeleton-based intention recognition, we propose a spatial-temporal graph convolutional network that performs graph convolutions on both part-based graphs and adaptive graphs, which achieves the best performance compared with baseline models in the same task. A deep-learning-based method using multimodal features is proposed to automatically infer intentions, which is demonstrated to accurately predict intentions based on past behaviors in the experiment, significantly outperforming humans.

Tensegrity robots, composed of rigid rods and flexible cables, exhibit high strength-to-weight ratios and significant deformations, which enable them to navigate unstructured terrains and survive harsh impacts. They are hard to control, however, due to high dimensionality, complex dynamics, and a coupled architecture. Physics-based simulation is a promising avenue for developing locomotion policies that can be transferred to real robots. Nevertheless, modeling tensegrity robots is a complex task due to a substantial sim2real gap. To address this issue, this paper describes a Real2Sim2Real (R2S2R) strategy for tensegrity robots. This strategy is based on a differentiable physics engine that can be trained given limited data from a real robot. These data include offline measurements of physical properties, such as mass and geometry for various robot components, and the observation of a trajectory using a random control policy. With the data from the real robot, the engine can be iteratively refined and used to discover locomotion policies that are directly transferable to the real robot. Beyond the R2S2R pipeline, key contributions of this work include computing non-zero gradients at contact points, a loss function for matching tensegrity locomotion gaits, and a trajectory segmentation technique that avoids conflicts in gradient evaluation during training. Multiple iterations of the R2S2R process are demonstrated and evaluated on a real 3-bar tensegrity robot.

In real-world settings, bipedal robots must avoid collisions with people and their environment. Further, a biped can choose between modes of avoidance: (1) adjust its pose while standing or (2) step to gain maneuverability. We present a real-time motion planner and multibody control framework for dynamic bipedal robots that avoids multiple moving obstacles and automatically switches between standing and stepping modes as necessary. By leveraging a reduced-order model (i.e. Linear Inverted Pendulum Model) and a half-space relaxation of the safe region, the planner is formulated as a convex optimization problem (i.e. Quadratic Programming) that can be used for real-time application with Model-Predictive-Control (MPC). To facilitate mode switching, we introduce center-of-pressure related slack-variables to the convex planning optimization that both shapes the planning cost function and provides a mode switching criterion for dynamic locomotion. Finally, we implement the proposed algorithm on a 3D Cassie bipedal robot and present hardware experiments showing real-time bipedal standing avoidance, stepping avoidance, and automatic switching of avoidance modes.

The work in this paper delves into the challenge of whole elongated body's obstacle avoidance during path following for a class of bionic snake robots. Currently, most studies focus solely on preventing the robot's head from colliding with obstacles through designed controllers. However, due to the unique elongated structure and biomimetic locomotion modes of snake robots, it is unavoidable that the rest of the robot's body could still collide with obstacles. To resolve this problem, we propose a novel real-time optimization obstacle avoidance strategy for a class of terrestrial snake robots with multi-link elongated body using model predictive control (MPC). Moreover, by leveraging the elongated body characteristics of the robot, an improved path guidance strategy is also developed. The effectiveness of the proposed strategies is verified and validated through extensive simulations and experiments on a custom-built nine-link elongated snake robot. The results demonstrate that all links of the robot can well avoid obstacles while continuing to track the given path.

This paper presents a Video Inpainting algorithm that enables monocular-camera-laser-based pipeline inspection robots to capture both color and 3D information using only one video stream. Conventional monocular-camera-laser inspection methods are limited to capture either 2D color images or 3D point clouds since the laser tends to overexpose the actual color of the scanning area. We propose a real-time Video Inpainting method to solve this problem with minimal hardware needs that can be easily integrated with conventional pipeline profiling robots. The algorithm is accelerated by two components: a lightweight network that directly predicts the complete optical flow and simplifies the algorithm pipeline, and the Polar coordinate transformation, which significantly reduces the image processing compexity. Real-world experiments demonstrate that our online algorithm has comparable or better color estimation accuracy against state-of-the-art offline algorithms, while is capable of running at 23 frames per second (FPS) on a laptop computer with a resolution of 1024 × 1024 pixels. In addition, we verify that this method can be used for video pre-processing for downstream tasks that require high-quality visual inputs, such as Simultaneously Localization and Mapping (SLAM). To the best of our knowledge, this is the first real-time Video Inpainting algorithm that can be used for in-pipe environments, serving as an important building block for highly compact RGB-D inspection sensors and robots for the pipeline industry.

We consider the motion planning problem for stochastic nonlinear systems in uncertain environments. More precisely, in this problem the robot has stochastic nonlinear dynamics and uncertain initial locations, and the environment contains multiple dynamic uncertain obstacles. Obstacles can be of arbitrary shape, can deform, and can move. All uncertainties do not necessarily have Gaussian distribution. This general setting has been considered and solved in [1]. In addition to the assumptions above, in this paper, we consider long-term tasks, where the planning method in [1] would fail, as the uncertainty of the system states grows too large over a long time horizon. Unlike [1], we present a real-time online motion planning algorithm. We build discrete-time motion primitives and their corresponding continuous-time tubes offline, so that almost all system states of each motion primitive are guaranteed to stay inside the corresponding tube. We convert probabilistic safety constraints into a set of deterministic constraints called risk contours. During online execution, we verify the safety of the tubes against deterministic risk contours using sum-of-squares (SOS) programming. The provided SOS-based method verifies the safety of the tube in the presence of uncertain obstacles without the need for uncertainty samples and time discretization in real-time. By bounding the probability the system states staying inside the tube and bounding the probability of the tube colliding with obstacles, our approach guarantees bounded probability of system states colliding with obstacles. We demonstrate our approach on several long-term robotics tasks.

Social group detection is a crucial aspect of various robotic applications, including robot navigation and human-robot interactions. To date, a range of model-based techniques have been employed to address this challenge, such as the F-formation and trajectory similarity frameworks. However, these approaches often fail to provide reliable results in crowded and dynamic scenarios. Recent advancements in this area have mainly focused on learning-based methods, such as deep neural networks that use visual content or human pose. Although visual content based methods have demonstrated promising performance on large-scale datasets, their computational complexity poses a significant barrier to their practical use in real-time applications. To address these issues, we propose a simple and efficient framework for social group detection. Our approach explores the impact of motion trajectory on social grouping and utilizes a novel, reliable, and fast data-driven method. We formulate the individuals in a scene as a graph, where the nodes are represented by LSTM-encoded trajectories and the edges are defined by the distances between each pair of tracks. Our framework employs a modified graph transformer module and graph clustering losses to detect social groups. Our experiments on the popular JRDB-Act dataset reveal noticeable improvements in performance, with relative improvements ranging from 2% to 11%. Furthermore, our framework is significantly faster, with up to 12x faster inference times compared to state-of-the-art methods under the same computation resources. These results demonstrate that our proposed method is suitable for real-time robotic applications. .

In this paper, we present a realtime method for simultaneous object-level scene understanding and grasp prediction. Specifically, given a single RGBD image of a scene, our method localizes all the objects in the scene and for each object, it generates the following: full 3D shape, scale, pose with respect to the camera frame, and a dense set of feasible grasps. The main advantage of our method is its computation speed as it avoids sequential perception and grasp planning. With detailed quantitative analysis of reconstruction quality and grasp accuracy, we show that our method delivers competitive performance compared to the state-of-the-art methods, while providing fast inference at 30 frames per second speed.

Signal Temporal Logic (STL) is a rigorous specification language that allows one to express various spatio-temporal requirements and preferences. Its semantics (called robustness) allows quantifying to what extent are the STL specifications met. In this work, we focus on enabling STL constraints and preferences in the Real-Time Rapidly Exploring Random Tree (RT-RRT*) motion planning algorithm in an environment with dynamic obstacles. We propose a cost function that guides the algorithm towards the asymptotically most robust solution, i.e. a plan that maximally adheres to the STL specification. In experiments, we applied our method to a social navigation case, where the STL specification captures spatio-temporal preferences on how a mobile robot should avoid an incoming human in a shared space. Our results show that our approach leads to plans adhering to the STL specification, while ensuring efficient cost computation.

In this paper, we propose a system for real-time rat pose estimation based on stereo vision. The system is dedicated to robot-rat interaction research. First, we design a lightweight, high-resolution network (RRKDNet) for keypoint detection of the rat. The network is trained on a dataset of rat images, which are captured by the robotic rat in first-person view. Second, based on the keypoint detection results, the pose of the rat is obtained by stereo vision model calculation and robot coordinate transformation. At last, we complete a real-time simulation experiment to reproduce the pose of the rat and the robotic rat. The system has been subjected to a series of experiments and the results demonstrate that our network performs better in speed and performance than similar networks. Compared to similar networks, our network has about one-third the number of parameters, while the detection rate increases by 45.25% (the detection rate is 71.57%). The inference speed (34.42 FPS with dual model simultaneous inference) is also faster. The validation error is only 13.85 pixels on the homemade dataset, which is lower than all backbones in Deeplabcut (a toolbox more frequently used for rat keypoint detection). Thus, this work is a significant step in the autonomous intelligent interaction between robots and rats.

As global living standards continue to rise and urbanization accelerates, cities worldwide face high demands for parking. To tackle this issue and foster the efficient use of limited parking capacity, automated valet parking (AVP) solutions have emerged as a resource optimization and hassle-free alternative. This paper addresses the problem of reducing retrieval time in AVP systems while ensuring the integrity of both the robot and the car through critical adherence conditions. The key novelty of our approach is that we dovetail these conditions as acceleration constraints into a multi-stage motion generation framework formulated as a real-time nonlinear model predictive control (NMPC) scheme that approximates time-optimal behavior by maximizing progress on a path. The NMPC generates trajectories based on the motion direction, allowing for convenient parameterization of controller instances and unlimited parking maneuvers. Thanks to high-performance software implementations, the resulting quadratic subprograms can be solved in the order of milliseconds. Comparative analysis against the currently implemented pure pursuit algorithm and an experimental validation on Stanley Robotics' robot have shown a 31% time improvement following a path and constraint satisfaction in the loaded scenario for the proposed controller.