ICRA 2022 论文列表

A particular challenge for both autonomous and human driving is dealing with risk associated with dynamic occlusion, i.e., occlusion caused by other vehicles in traffic. Based on the theory of hypergames, we develop a novel multi-agent dynamic occlusion risk (DOR) measure for assessing situational risk in dynamic occlusion scenarios. Furthermore, we present a white-box, scenario-based, accelerated safety validation framework for assessing safety of strategic planners in AV. Based on evaluation over a large naturalistic database, our proposed validation method achieves a 4000% speedup compared to direct validation on naturalistic data, a more diverse coverage, and ability to generalize beyond the dataset and generate commonly observed dynamic occlusion crashes in traffic in an automated manner.

In this paper, we are interested in the problem of generating target grasps by understanding freehand sketches. The sketch is useful for the persons who cannot formulate language and the cases where a textual description is not available on the fly. However, very few works are aware of the usability of this novel interactive way between humans and robots. To this end, we propose a method to generate a potential grasp configuration relevant to the sketch -depicted objects. Due to the inherent ambiguity of sketches with abstract details, we take the advantage of the graph by incorporating the structure of the sketch to enhance the representation ability. This graph-represented sketch is further validated to improve the generalization of the network, capable of learning the sketch-queried grasp detection by using a small collection (around 100 samples) of hand-drawn sketches. Additionally, our model is trained and tested in an end-to-end manner which is easy to be implemented in real-world applications. Experiments on the multi-object VMRD and GraspNet-1Billion datasets demonstrate the good generalization of the proposed method. The physical robot experiments confirm the utility of our method in object-cluttered scenes.

There is an increasing interest in soft robotic grippers as they exhibit an ability to grip objects of differing shapes, sizes, textures, and even deformable materials, all of which present a difficult challenge to traditional rigid grippers. An ideal soft gripper would exhibit universal gripping with high gripping force and consists of low-cost materials with simple fabrication processes. This paper investigates the development of a strong and scalable hydraulic soft tubular gripper (HSTG) using facile fabrication method and low-cost materials. The HSTG which consists of a single long hydraulically actuated artificial muscle, soft 3D printed element, and commercial weaving yarn can expand and contract its orifice to grasp objects using a miniature hydraulic syringe. Grasping experiments show that the new HSTG can successfully grasp convex, nonconvex, and flat objects as well as the ones with cavity. The soft gripper uniquely exhibits high normal contact force at minimal pressure and energy use due to the nature of its working principle. A 26 g HSTG can produce at least 40 N of gripping force, can hold at least 88 N in external gripping mode (~346 times of its weight), 0.34 N in internal mode, and 1.74 N in suction gripping mode. The design and mechanical properties of its components can be fine-tuned to produce tailored performance for different grasping tasks.

This paper presents a proposal of new hydraulic circuit, designated as a modular hydraulic servo booster (MHSB), aimed at the realization of modular hydraulic robots. The modular robots, however, have important shortcomings compared to non-modular robots, such as separate power sources and power imbalance between axes when applied to serially configured robots. To mitigate those difficulties, we take advantage of pressure boost and flow summing by multiple servo pumps and switching valves, which are connected through the hydraulic rails shared between the circuits. Coordinated control of the pump and valve greatly improves energy efficiency over conventional servo valve systems. After presenting realization of the circuit and possible operating modes, the control method for each mode is explained. The experimentally obtained results for position control of a two-link serial manipulator validate the proposed method, especially by the shared boost mode, where small pumps work together to have high torque and speed of the proximal joint.

6-DoF grasp pose detection of multi-grasp and multi-object is a challenge task in the field of intelligent robot. To imitate human reasoning ability for grasping objects, data driven methods are widely studied. With the introduction of large-scale datasets, we discover that a single physical metric usually generates several discrete levels of grasp confidence scores, which cannot finely distinguish millions of grasp poses and leads to inaccurate prediction results. In this paper, we propose a hybrid physical metric to solve this evaluation insufficiency. First, we define a novel metric is based on the force-closure metric, supplemented by the measurement of the object flatness, gravity and collision. Second, we leverage this hybrid physical metric to generate elaborate confidence scores. Third, to learn the new confidence scores effectively, we design a multi-resolution network called Flatness Gravity Collision GraspNet (FGC-GraspNet). FGC-GraspNet proposes a multi-resolution features learning architecture for multiple tasks and introduces a new joint loss function that enhances the average precision of the grasp detection. The network evaluation and adequate real robot experiments demonstrate the effectiveness of our hybrid physical metric and FGC-GraspNet. Our method achieves 90.5% success rate in real-world cluttered scenes. Our code is available at https://github.com/luyh20IFGC-GraspNet.

Geometric methods for solving open-world off-road navigation tasks, by learning occupancy and metric maps, provide good generalization but can be brittle in outdoor environments that violate their assumptions (e.g., tall grass). Learning-based methods can directly learn collision-free behavior from raw observations, but are difficult to integrate with standard geometry-based pipelines. This creates an unfortunate conflict – either use learning and lose out on well-understood geometric navigational components, or do not use it, in favor of extensively hand-tuned geometry-based cost maps. In this work, we reject this dichotomy by designing the learning and non-learning-based components in a way such that they can be effectively combined in a self-supervised manner. Both components contribute to a planning criterion: the learned component contributes predicted traversability as rewards, while the geometric component contributes obstacle cost information. We instantiate and comparatively evaluate our system in both in-distribution and out-of-distribution environments, showing that this approach inherits complementary gains from the learned and geometric components and significantly outperforms either of them.

Many controllers for legged robotic systems leverage open- or closed-loop control at discrete hybrid events to enhance stability. These controllers appear in several well studied phenomena such as the Raibert stepping controller, paddle juggling, and swing leg retraction. This work introduces hybrid event shaping (HES): a generalized method for analyzing and designing stable hybrid event controllers. HES utilizes the saltation matrix, which gives a closed-form equation for the effect that hybrid events have on stability. We also introduce shape parameters, which are higher order terms that can be tuned completely independently from the system dynamics to promote stability. Optimization methods are used to produce values of these parameters that optimize a stability measure. Hybrid event shaping captures previously developed control methods while also producing new optimally stable trajectories without the need for continuous-domain feedback.

Humans noticeably swing their arms for balancing and locomotion. Although the underlying biomechanical mechanisms have been studied, it is unclear how robots can fully take advantage of these appendages. Most controllers that exploit arms for balance and locomotion rely on feedback and cannot anticipate incoming disturbances and future states. Model predictive controllers readily address these drawbacks but are computationally expensive. Here, we leverage recent work on model hierarchy predictive control (MHPC). We develop an MHPC formulation that plans arm motions in reaction to expected or unexpected disturbances. We tested multiple model compositions using simulated balance experiments with the MIT Humanoid undergoing various disturbances. We found that an MHPC formulation that plans over a full-body kino-dynamic model for a 0.3 s horizon followed by a single rigid body model for 0.5 s horizon runs at 40 Hz and increases the set of disturbances that the robot can withstand. Arms allow the robot to dissipate momentum quickly and move the center of mass independently from the lower body. This kinematic advantage helps generate ground wrenches while avoiding kinematic singularities and keeping the center of mass and center pressure within the support polygon. We note similar advantages when allowing the MHPC to anticipate incoming disturbances.

Human-robot shared control, which integrates the advantages of both humans and robots, is an effective approach to facilitate efficient surgical operation. Learning from demonstration (LfD) techniques can be used to automate some of the surgical sub tasks for the construction of the shared control mechanism. However, a sufficient amount of data is required for the robot to learn the manoeuvres. Using a surgical simulator to collect data is a less resource-demanding approach. With sim-to-real adaptation, the manoeuvres learned from a simulator can be transferred to a physical robot. To this end, we propose a sim-to-real adaptation method to construct a human-robot shared control framework for robotic surgery. In this paper, a desired trajectory is generated from a simulator using LfD method, while dynamic motion primitives (DMP) is used to transfer the desired trajectory from the simulator to the physical robotic platform. Moreover, a role adaptation mechanism is developed such that the robot can adjust its role according to the surgical operation contexts predicted by a neural network model. The effectiveness of the proposed framework is validated on the da Vinci Research Kit (dVRK). Results of the user studies indicated that with the adaptive human-robot shared control framework, the path length of the remote controller, the total clutching number and the task completion time can be reduced significantly. The proposed method outperformed the traditional manual control via teleoperation.

Motion planning is a core problem in robotics, with a range of existing methods aimed to address its diverse set of challenges. However, most existing methods rely on complete knowledge of the robot environment; an assumption that seldom holds true due to inherent limitations of robot perception. To enable tractable motion planning for high-DOF robots under partial observability, we introduce BLIND, an algorithm that leverages human guidance. BLIND utilizes inverse reinforcement learning to derive motion-level guidance from human critiques. The algorithm overcomes the computational challenge of reward learning for high-DOF robots by projecting the robot's continuous configuration space to a motion-planner-guided discrete task model. The learned reward is in turn used as guidance to generate robot motion using a novel motion planner. We demonstrate BLIND using the Fetch robot and perform two simulation experiments with partial observability. Our experiments demonstrate that, despite the challenge of partial observability and high dimensionality, BLIND is capable of generating safe robot motion and outperforms baselines on metrics of teaching efficiency, success rate, and path quality.

Assistance robots have gained widespread attention in various industries such as logistics and human assistance. The tasks of guiding or following a human in a crowded environment such as airports or train stations to carry weight or goods is still an open problem. In these use cases, the robot is not only required to intelligently interact with humans, but also to navigate safely among crowds. Thus, especially highly dynamic environments pose a grand challenge due to the volatile behavior patterns and unpredictable movements of humans. In this paper, we propose a Deep-Reinforcement-Learning-based agent for human-guiding and -following tasks in crowded environments. Therefore, we incorporate semantic information to provide the agent with high-level information like the social states of humans, safety models, and class types. We evaluate our proposed approach against a benchmark approach without semantic information and demonstrated enhanced navigational safety and robustness. Moreover, we demonstrate that the agent could learn to adapt its behavior to humans, which improves the human-robot interaction significantly.

Although human navigational intent inference has been studied in the literature, none have adequately considered both the dynamics that describe human motion and internal human parameters that may affect human navigational behaviour. In this paper, we propose a general probabilistic framework to infer the probability distribution over future navigational states of a human. Our framework incorporates an extended Dubins car dynamics to model human movement, which captures differences in human navigational behaviour depending on their position, heading, and movement speed. We assume a noisily rational model of human behaviour that incorporates a) human navigational intent that may change over time, b) how optimal a person's actions are given the navigational intent, and c) how far ahead in time a person considers when choosing navigational actions. These parameters are recursively and continuously updated in a Bayesian fashion. To make the Bayesian update and inference tractable, we exploit properties of the time-to-reach value function from optimal control and the extended Dubins car dynamics to construct a utility function on which the human policy is based, and employ particle representations of probability distributions where necessary. We demonstrate the effectiveness of our method by comparing our results with a recent approach using synthetic data and validate it on real world data.

Curbs are one of the essential elements of urban and highway traffic environments. Robust curb detection provides road structure information for motion planning in an autonomous driving system. Commonly, video cameras and 3D LiDARs are mounted on autonomous vehicles for curb detection. However, camera-based methods suffer from challenging illumination conditions. During the long period of time before wide application of Deep Neural Network (DNN) with point clouds, LiDAR-based curb detection methods are based on hand-crafted features, which suffer from poor detection in some complex scenes. Recently, DNN-based dynamic object detection using LiDAR data has become prevalent, while few works pay attention to curb detection with a DNN approach due to lack of labeled data. A dataset with curb annotations or an efficient curb labeling approach, hence, is of high demand. In this paper, we present how to build a curb dataset with LiDAR data for autonomous driving highly automatically. Firstly, a Semantic High Definition map (SHD map) in a global coordinate frame is generated by applying both SLAM and semantic segmentation on consecutive LiDAR frames. Next, a Road HD map (RHD map) is generated from the SHD map by removing its dynamic noise caused by road users e.g. cars. After that, a Curb Instance map (CI map) can be obtained from the filtered RHD map by a series of curb point extraction and growing. Finally, the CI map can be projected back to single frames for direct, highly automatic curb labeling. In order to validate our proposed labeling method, on top of an open public LiDAR semantic dataset SemanticKITTI [1], an additional curb dataset is built. We run both semantic segmentation and instance segmentation methods on this built dataset. Experimental results show that the curb annotations have good consistency and accuracy. We released this dataset and it is publicly available at https://download.mindspore.cn.

Real-time semantic segmentation is a crucial but challenging dense prediction task for scene parsing. However, the existing CNN-based methods commonly bias the model in favor of speed-boosting compromising spatial resolution due to business requirements and hardware constrains, which impedes the high-accuracy segmentation result. To address the dilemma, we provide a novel Holographic Segmentation Network (HoloSeg), which presents a strong ability of comprehensive information preservation and extraction, and achieves a better trade-off between speed and accuracy. We first design a Lossless Sample Pair (LSP) without any stride for early spatial preservation and later resolution recovery while modeling long-range context dependence. Then, we propose Distributed Pyramid Learning (DPL) to efficiently extract multiscale features and saves a lot of computation. Finally, we propose Resolution Fusion and Restoration (RFR) to fuse multi-level semantic representations across stages and generate output without decoder. Without bells and whistles, HoloSeg achieves state-of-the-art performance on the Cityscapes benchmark which reports 76.24% mIoU at 231 FPS. Code is available online: https://github.com/LiShuTJ/HoloSeg.

Due to the difficulty and expense of underwater field trials, a high fidelity underwater simulator is a necessity for testing and developing algorithms. To fill this need, we present HoloOcean, an open source underwater simulator, built upon Unreal Engine 4 (UE4). HoloOcean comes equipped with multi-agent support, various sensor implementations of common underwater sensors, and simulated communications support. We also implement a novel sonar sensor model that leverages an octree representation of the environment for efficient and realistic sonar imagery generation. Due to being built upon UE4, new environments are straightforward to add, enabling easy extensions to be built. Finally, HoloOcean is controlled via a simple python interface, allowing simple installation via pip, and requiring few lines of code to execute simulations.

In this paper we present a complete framework for Underwater SLAM utilizing a single inexpensive sensor. Over the recent years, imaging technology of action cameras is producing stunning results even under the challenging conditions of the underwater domain. The GoPro 9 camera provides high definition video in synchronization with an Inertial Measurement Unit (IMU) data stream encoded in a single mp4 file. The visual inertial SLAM framework is augmented to adjust the map after each loop closure. Data collected at an artificial wreck of the coast of South Carolina and in caverns and caves in Florida demonstrate the robustness of the proposed approach in a variety of conditions.

Representations are crucial for a robot to learn effective navigation policies. Recent work has shown that mid-level perceptual abstractions, such as depth estimates or 2D semantic segmentation, lead to more effective policies when provided as observations in place of raw sensor data (e.g., RGB images). However, such policies must still learn latent three-dimensional scene properties from mid-level abstractions. In contrast, high-level, hierarchical representations such as 3D scene graphs explicitly provide a scene's geometry, topology, and semantics, making them compelling representations for navigation. In this work, we present a reinforcement learning framework that leverages high-level hierarchical representations to learn navigation policies. Towards this goal, we propose a graph neural network architecture and show how to embed a 3D scene graph into an agent-centric feature space, which enables the robot to learn policies that map 3D scene graphs to a platform-agnostic control space (e.g., go straight, turn left). For each node in the scene graph, our method uses features that capture occupancy and semantic content, while explicitly retaining memory of the robot trajectory. We demonstrate the effectiveness of our method against commonly used visuomotor policies in a challenging multi-object search task. These experiments and supporting ablation studies show that our method leads to more effective object search behaviors, exhibits improved long-term memory, and successfully leverages hierarchical information to guide its navigation objectives.

Retrieving objects from clutters is a complex task, which requires multiple interactions with the environment until the target object can be extracted. These interactions involve executing action primitives like grasping or pushing as well as setting priorities for the objects to manipulate and the actions to execute. Mechanical Search (MS) [1] is a framework for object retrieval, which uses a heuristic algorithm for pushing and rule-based algorithms for high-level planning. While rule-based policies profit from human intuition in how they work, they usually perform sub-optimally in many cases. Deep reinforcement learning (RL) has shown great performance in complex tasks such as taking decisions through evaluating pixels, which makes it suitable for training policies in the context of object-retrieval. In this work, we first formulate the MS problem in a principled formulation as a hierarchical POMDP. Based on this formulation, we propose a hierarchical policy learning approach for the MS problem. For demonstration, we present two main parameterized sub-policies: a push policy and an action selection policy. When integrated into the hierarchical POMDP's policy, our proposed sub-policies increase the success rate of retrieving the target object from less than 32% to nearly 80%, while reducing the computation time for push actions from multiple seconds to less than 10 milliseconds.

Place recognition or loop closure detection is one of the core components in a full SLAM system. In this paper, aiming at strengthening the relevancy of local neighboring points and the contextual dependency among global points simultaneously, we investigate the exploitation of transformer-based network for feature extraction, and propose a Hierarchical Transformer for Place Recognition (HiTPR). The HiTPR consists of four major parts: point cell generation, short-range transformer (SRT), long-range transformer (LRT) and global descriptor aggregation. Specifically, the point cloud is initially divided into a sequence of small cells by down-sampling and nearest neighbors searching. In the SRT, we extract the local feature for each point cell. While in the LRT, we build the global dependency among all of the point cells in the whole point cloud. Experiments on several standard benchmarks demonstrate the superiority of the HiTPR in terms of average recall rate, achieving 93.71 % at top 1 % and 86.63 % at top 1 on the Oxford RobotCar dataset for example.