ICRA 2021 论文列表

In this paper we propose a deep architecture and the associated learning strategy for end-to-end direct visual servoing. The considered approach allows to sequentially predict, in se(3), the velocity of a camera mounted on the robot’s end-effector for positioning tasks. Positioning is achieved with high precision despite large initial errors in both cartesian and image spaces. Training is fully done in simulation, alleviating the burden of data collection. We demonstrate the efficiency of our method in experiments in both simulated and real-world environments. We also show that the proposed approach is able to handle multiple scenes.

This paper deals with the problem of needle steering in deformable tissues subject to physiological motions. A novel shared control method is proposed, which combines an automatic needle steering algorithm with the motions applied by the radiologist, in order to place the needle tip at the desired location. The core motivation is to leave potentially dangerous decisions and actions to the practitioner, whereas complex non-intuitive manipulations of the needle are performed automatically, in particular to compensate for breathing motions. The most original part of the present work lies in the method used to combine user inputs with a closed-loop automatic needle steering control method based on inverse Finite Element simulations. The method is evaluated with a realistic virtual environment using 2D X-ray projection images. The results are compared with those obtained with a fully teleoperated system, on the one hand, and with a fully automatic solution, on the other hand. These experiments show that the shared control solution allows for a better needle tip placement when only projection imaging is available.

Humans show specialized strategies for efficient collaboration. Transferring similar strategies to humanoid robots can improve their capability to interact with other agents, leading the way to complex collaborative scenarios with multiple agents acting on a shared environment. In this paper we present a control framework for robot-robot collaborative lifting. The proposed shared controller takes into account the joint action of both the robots thanks to a centralized controller that communicates with them, and solves the whole-system optimization. Efficient collaboration is ensured by taking into account the ergonomic requirements of the robots through the optimization of posture and contact forces. The framework is validated in an experimental scenario with two iCub humanoid robots performing different payload lifting sequences.

Teleoperation deals with extraordinary situations where an external operator takes over the control of an autonomous vehicle. Especially in complex urban scenarios, this may cause a too high workload for the human operator, resulting in suboptimal solutions. This contribution presents a teleoperation paradigm to raise the autonomy level of teleoperated driving, while the operator still remains the main decision-maker in all driving tasks. The introduced approach generates collision-free paths using LiDAR sensor information and suggests them to the operator. Therefore, a new hybrid path planning method has been developed, which searches and clusters in the first phase all feasible paths in the environment using a modified Rapidly-Exploring-Random Tree (RRT). In the second phase, the path selected by the operator is optimized online by a modified CHOMP algorithm. Real driving experiments confirm the effectiveness of the approach and highlight both the achieved driving safety and real time capability.

The potential benefits of model-free reinforcement learning to real robotics systems are limited by its uninformed exploration that leads to slow convergence, lack of data-efficiency, and unnecessary interactions with the environment. To address these drawbacks we propose a method that combines reinforcement and imitation learning by shaping the reward function with a state-and-action-dependent potential that is trained from demonstration data, using a generative model. We show that this accelerates policy learning by specifying high-value areas of the state and action space that are worth exploring first. Unlike the majority of existing methods that assume optimal demonstrations and incorporate the demonstration data as hard constraints on policy optimization, we instead incorporate demonstration data as advice in the form of a reward shaping potential trained as a generative model of states and actions. In particular, we examine both normalizing flows and Generative Adversarial Networks to represent these potentials. We show that, unlike many existing approaches that incorporate demonstrations as hard constraints, our approach is unbiased even in the case of suboptimal and noisy demonstrations. We present an extensive range of simulations, as well as experiments on the Franka Emika 7DOF arm, to demonstrate the practicality of our method.

In this paper, we try to improve exploration in Blackbox methods, particularly Evolution strategies (ES), when applied to Reinforcement Learning (RL) problems where intermediate waypoints/subgoals are available. Since Evolutionary strategies are highly parallelizable, instead of extracting just a scalar cumulative reward, we use the state-action pairs from the trajectories obtained during rollouts/evaluations, to learn the dynamics of the agent. The learnt dynamics are then used in the optimization procedure to speed-up training. Lastly, we show how our proposed approach is universally applicable by presenting results from experiments conducted on Carla driving and UR5 robotic arm simulators.

We propose a new, data-efficient approach for skill transfer to novel objects, accounting for known categorical shape variation. A low-dimensional shape representation embedding is learned from a set of deformations, sampled between known objects within a category. This latent representation is mapped to a set of control parameters that result in successful execution of a category-level skill on that object. This method generalizes a learned manipulation policy to unseen objects with few training examples. We demonstrate this approach on pouring from cups and scooping with spatulas, where there is complex, nonlinear variation of successful control parameters across objects.

Shape sensors are important for safer and more dexterous manipulation of the medical catheters. Among the electromagnetic based shape sensors, a voice coil shape sensor measures the variation of a mutual inductance between the coils placed along the tube due to the bending of the tube. Owing to the design flexibility of a voice coil, it offers the small size without the external magnetic field generator outside patient body. This paper presents an improved voice coil shape sensor in terms of modeling and measurement. Analytic model that incorporates the bending of an exciter coils is used to improve the accuracy of the sensor and band-pass filter is applied to simplify the measurements. Bending angle from sensors placed in multiple locations can be measured through a single channel from frequency sweep input. The simulation and experimental results verify the improvement and demonstrate that the sensor system can reconstruct a shape of a catheter placed through larynx.

Robots in semi-structured environments such as homes and warehouses sporadically require computation of high-dimensional motion plans. Cloud and fog-based parallelization of motion planning can speed up planning. This can be further made efficient by the use of "serverless" on-demand computing as opposed to always-on high end computers. This paper explores parallelizing the computation of a sampling-based multi-query motion planner based on asymptotically-optimal Probabilistic Road Maps (PRM*) using the simultaneous execution of 100s of cloud-based serverless functions. We propose an algorithm to overcome the communication and bandwidth limitations of serverless computing and use different work-sharing techniques to further optimize the cost and run time. Additionally, we provide proofs of probabilistic completeness and asymptotic optimality. In experiments on synthetic benchmarks and on a physical Fetch robot performing a sequence of decluttering motions, we observe up to a 50x speedup relative to a 4 core edge computer with only a marginally higher cost.

We have developed service robot management system to facilitate effective collaboration between multiple units and types of robots in operation. This system is implemented by serverless architecture on cloud and using cellular based IoT communication. So it has not only usual cloud system advantage that it is not necessary to prepare dedicated server and network equipment, but it reduces management efforts of servers. We have tested the system with robots in a public facility, and successfully confirmed its performance and functionality.

We carry out a structural and algorithmic study of a mobile sensor coverage optimization problem targeting 2D surfaces embedded in a 3D workspace. The investigated settings model multiple important applications including camera net- work deployment for surveillance, geological monitoring/survey of 3D terrains, and UVC-based surface disinfection for the prevention of the spread of disease agents (e.g., SARS-CoV-2). Under a unified general "sensor coverage" problem, three concrete formulations are examined, focusing on optimizing visibility, single-best coverage quality, and cumulative quality, respectively. After demonstrating the computational intractability of all these formulations, we describe approximation schemes and mathematical programming models for near-optimally solving them. The effectiveness of our methods is thoroughly evaluated under realistic and practical scenarios.

Applications of micro unmanned aerial vehicles (UAVs) are gradually expanding into complex urban and natural environments. Despite noticeable progress, flying robots in obstacle-rich environments is still challenging. On-board processing for detecting and avoiding obstacles is possible, but at a significant computational expense, and with significant limitations (e.g., for obstacles with small cross sections, such as wires). A low-cost alternative is to mitigate physical contacts through a cage or other similar protective devices. In this paper, we propose to transform these passive protective devices into functional sensors: we introduce a suspended rim combined with a central base measuring the relative displacement of the rim; we provide a full mechanical design, and derive solutions to the inverse kinematics for recovering the collision direction in real time. As a proof of concept, we show the benefits of this novel form of sensing by embedding it in a traditional particle filter for self-localization in a known environment; our experiments show that localization is possible with a minimal sacrifice in payload capacity.

This paper presents a novel computational model to address the problem that contact is an infinite phenomena involving continuous regions of interaction. The problem is cast as a semi-infinite program with complementarity constraints (SIPCC). Rather than pre-discretize contacting surfaces into a finite number of contact points, we use semi-infinite programming (SIP) techniques that operate on the underlying continuous geometry, but dynamically determine a finite number of constraints that are most relevant to solving the problem. Then we solve the series of problems whose solutions converge toward one that contains a true optimum of the original SIPCC. We apply the model to a grasping pose optimization problem for a gripper and a humanoid robot, and our model enables the robots to find a feasible pose to hold (non-)convex objects while ensuring force and torque balance.

Mobile robots in unstructured, mapless environments must rely on an obstacle avoidance module to navigate safely. The standard avoidance techniques estimate the locations of obstacles with respect to the robot but are unaware of the obstacles’ identities. Consequently, the robot cannot take advantage of semantic information about obstacles when making decisions about how to navigate. We propose an obstacle avoidance module that combines visual instance segmentation with a depth map to classify and localize objects in the scene. The system avoids obstacles differentially, based on the identity of the objects: for example, the system is more cautious in response to unpredictable objects such as humans. The system can also navigate closer to harmless obstacles and ignore obstacles that pose no collision danger, enabling it to navigate more efficiently. We validate our approach in two simulated environments: one terrestrial and one underwater. Results indicate that our approach is feasible and can enable more efficient navigation strategies.

Compared to widely used LiDAR-based mapping in autonomous driving field, image-based mapping method has the advantages of low cost, high resolution, and no need for complex calibration. However, the image-based 3D mapping depends heavily on the texture richness and always leaves holes and outliers in low-textured areas, such as the road surface. To this end, this paper proposed a novel semantically guided Multi-View Stereo method for dense 3D road mapping, which integrates semantic information into PatchMatch-based MVS pipeline and uses image semantic segmentation as soft constraints in neighbor views selection, depth-map initialization, depth propagation, and depth-map completion. Experimental results on public and our own datasets show that, with the help of semantics, the proposed method achieves superior completeness with comparable accuracy for 3D road mapping compared to state-of-the-art MVS methods.

Searching for objects in indoor organized environments such as homes or offices is part of our everyday activities. When looking for a desired object, we reason about the rooms and containers the object is likely to be in; the same type of container will have a different probability of containing the target depending on which room it is in. We also combine geometric and semantic information to infer what container is best to search, or what other objects are best to move, if the target object is hidden from view. We use a 3D scene graph representation to capture the hierarchical, semantic, and geometric aspects of this problem. To exploit this representation in a search process, we introduce Hierarchical Mechanical Search (HMS), a method that guides an agent’s actions towards finding a target object specified with a natural language description. HMS is based on a novel neural network architecture that uses neural message passing of vectors with visual, geometric, and linguistic information to allow HMS to process data across layers of the graph while combining semantic and geometric cues. HMS is trained on 1000 3D scene graphs and evaluated on a novel dataset of 500 3D scene graphs with dense placements of semantically related objects in storage locations, and is shown to be significantly better than several baselines at finding objects. It is also close to the oracle policy in terms of the median number of actions required. Additional qualitative results can be found at https://ai.stanford.edu/mech-search/hms

It is often desirable to capture and map semantic information of an environment during simultaneous localization and mapping (SLAM). Such semantic information can enable a robot to better distinguish places with similar low-level geometric and visual features and perform high-level tasks that use semantic information about objects to be manipulated and environments to be navigated. While semantic SLAM has gained increasing attention, there is little research on semantic-level data association based on semantic objects, i.e., object-level data association. In this paper, we propose a novel object-level data association algorithm based on bag of words algorithm [1], formulated as a maximum weighted bipartite matching problem. With object-level data association solved, we develop a quadratic-programming-based semantic object initialization scheme using dual quadric and introduce additional constraints to improve the success rate of object initialization. The integrated semantic-level SLAM system can achieve high-accuracy object-level data association and real-time semantic mapping as demonstrated in the experiments. The online semantic map building and semantic-level localization capabilities facilitate semantic-level mapping and task planning in a priori unknown environment.

Large-scale visual place recognition (VPR) is inherently challenging because not all visual cues in the image are beneficial to the task. In order to highlight the task-relevant visual cues in the feature embedding, the existing attention mechanisms are either based on artificial rules or trained in a thorough data-driven manner. To fill the gap between the two types, we propose a novel Semantic Reinforced Attention Learning Network (SRALNet), in which the inferred attention can benefit from both semantic priors and data-driven fine-tuning. The contribution lies in two-folds. (1) To suppress misleading local features, an interpretable local weighting scheme is proposed based on hierarchical feature distribution. (2) By exploiting the interpretability of the local weighting scheme, a semantic constrained initialization is proposed so that the local attention can be reinforced by semantic priors. Experiments demonstrate that our method outperforms state-of-the-art techniques on city-scale VPR benchmark datasets.

Deep learning on 3D point clouds has drawn much attention, due to its large variety of applications in intelligent perception for automated and robotic systems. Unlike structured 2D images, it is challenging to extract features and implement convolutional networks over these unordered points. Although a number of previous works achieved high accuracies for point cloud recognition, they tend to process local point information in such a way that semantic information is not fully encoded. In this paper, we propose a deep neural network for 3D point cloud processing that utilizes effective feature aggregation methods emphasizing both generalizability and relevance. In particular, our method uses fixed-radius grouping for pooling layers and spherical kernel convolution for semantics mining. To address the issue of gradient degradation and memory consumption of a deep network, a parallel feature feed-forward mechanism and bottleneck layers are implemented to reduce the number of parameters. Experiments show that our algorithm achieves state-of-the-art results and competitive accuracy in both classification and part segmentation while maintaining an efficient architecture.