ICRA 2022 论文列表

Robust localization in dense urban scenarios using a low-cost sensor setup and sparse HD maps is highly relevant for the current advances in autonomous driving, but remains a challenging topic in research. We present a novel monocular localization approach based on a sliding-window pose graph that leverages predicted uncertainties for increased precision and robustness against challenging scenarios and per-frame failures. To this end, we propose an efficient multi-task uncertainty-aware perception module, which covers semantic segmentation, as well as bounding box detection, to enable the localization of vehicles in sparse maps, containing only lane borders and traffic lights. Further, we design differentiable cost maps that are directly generated from the estimated uncertainties. This opens up the possibility to minimize the reprojection loss of amorphous map elements in an association-free and uncertainty-aware manner. Extensive evaluation on the Lyft 5 dataset shows that, despite the sparsity of the map, our approach enables robust and accurate 6D localization in challenging urban scenarios using only monocular camera images and vehicle odometry.

Robust dynamic interactions are required to move robots in daily environments alongside humans. Optimisation and learning methods have been used to mimic and reproduce human movements. However, they are often not robust and their generalisation is limited. This work proposed a hierarchical control architecture for robot manipulators and provided capabilities of reproducing human-like motions during unknown interaction dynamics. Our results show that the reproduced end-effector trajectories can preserve the main characteristics of the initial human motion recorded via a motion capture system, and are robust against external perturbations. The data indicate that some detailed movements are hard to reproduce due to the physical limits of the hardware that cannot reach the same velocity recorded in human movements. Nevertheless, these technical problems can be addressed by using better hardware and our proposed algorithms can still be applied to produce imitated motions.

The rapid development of affordable and compact high-fidelity sensors (e.g., cameras and LIDAR) allows robots to construct detailed estimates of their states and environments. However, the availability of such rich sensor information introduces two challenges: (i) the lack of analytic sensing models, which makes it difficult to design controllers that are robust to sensor failures, and (ii) the computational expense of processing the high-dimensional sensor information in real time. This paper addresses these challenges using the theory of differential privacy, which allows us to (i) design controllers with bounded sensitivity to errors in state estimates, and (ii) bound the amount of state information used for control (i.e., to impose decision-making under bounded rationality). The resulting framework approximates the separation principle and allows us to derive an upper-bound on the cost incurred with a faulty state estimator in terms of three quantities: the cost incurred using a perfect state estimator, the magnitude of state estimation errors, and the level of differential privacy. We demonstrate the efficacy of our framework numerically on different robotics problems, including nonlinear system stabilization and motion planning.

Optical coherence tomography (OCT) has revolutionized diagnostics in ophthalmology. However, it requires pa-tient cooperation to fixate on multiple targets and stabilize their head utilizing both chin and forehead rests. Patient cooperation is particularly important for image montaging, where patients are asked to fixate on multiple targets to sequentially image different regions of interest on the retina. These individual volumes are then combined into a single large field of view volume. To automate the OCT image acquisition process, we previously developed a robot-mounted OCT scanner that auto-aligned with the retinal region of interest while compensating for subject motion. We utilized this system to self-align at multiple regions-of-interest and acquire stabilized volumes. We then montaged volume projections into a larger field of view image. The system tracked the 3D location of the subject's eye as well as their gaze orientation using a combination of face and pupil tracking cameras. We demonstrated automated OCT acquisition for live image montaging on free-standing subjects and evaluated the consistency of our live volumetric mapping on human subject data. Our results demonstrate that the system not only stabilized images, but also provided sufficient control of region-of-interest specific alignment to automatically acquire and montage OCT images, synthetically increasing the system's field of view by 20^{\circ}20^{\circ}.

This paper presents the design of a novel expert system for robotic manipulators performing sanding tasks on work surfaces. The expert system adjusts the velocity of the robotic manipulator based on the observed surface quality. These observation are obtained by an analysis of the raw force data provided by a force-torque sensor at the end-effector level. The expert system consists of two governing control laws that act in parallel, a variable velocity generation law and a pose regulation-based law. The variable velocity law regulates the velocity of the manipulator along a set path, in the tangent direction, based on an analysis of the frequency and amplitude of the force signal generated during the sanding process. The pose regulation-based law drives the manipulator in the bi-normal and rotational direction, ensuring the manipulators remain on the sanding path with the desired orientation. The proposed strategy is experimentally evaluated using the UR5e collaborative robotic manipulator sanding wood and metal panels. The obtained results show that such an approach is beneficial to ensure accurate contact between the sanding tool and the working environment, robust path tracking, and smart sanding.

Soft tissue cutting is used for incision, separation and removal of tissues or cells. Due to high deformation of soft tissues resulting from their viscosity and elasticity, it is challenging to accurately cut the tissue along a desired path and control the force applied to the tissue for reducing invasiveness, especially at the microscale. This paper presents a robotic biopsy system for cutting and collecting trophectoderm cells from a highly deformable blastocyst. The system, for the first time, enables TE cell junction detection for laser ablation throughout the blastocyst biopsy process by using a convolutional neural network. The overall detection error was 2.13% in every 1,000 cell junctions with position RMSE of 1.63\ \mu \mathrm{m}\pm 0.29\ \mu \mathrm{m}1.63\ \mu \mathrm{m}\pm 0.29\ \mu \mathrm{m}. A dynamics model was developed to describe the motion of the trophectoderm cells inside a biopsy micropipette. Based on this model, an adaptive control method was developed for trophectoderm cell aspiration and positioning inside the biopsy micropipette. Experimental results revealed that the controller was capable of effectively compensating for the cell positioning error by updating the varying system parameters according to the adaptation law. The success rate was 100%, the cell aggregate positioning accuracy was \pm 1\ \mu \mathrm{m}\pm 1\ \mu \mathrm{m}, the average settling time was 2 s, and the largest overshoot was 4.3\ \mu \mathrm{m}4.3\ \mu \mathrm{m}. Compared to manual blastocyst biopsy, the robotic biopsy system shortened the blastocyst's recovery time (35 min vs. 50 min) which indicates lower invasiveness.

Autonomous mobile manipulation robots that can collect trolleys are widely used to liberate human resources and fight epidemics. Most prior robotic trolley collection solutions only detect trolleys with 2D poses or are merely based on spe-cific marks and lack the formal design of planning algorithms. In this paper, we present a novel mobile manipulation system with applications in luggage trolley collection. The proposed system integrates a compact hardware design and a progressive perception and planning framework, enabling the system to efficiently and robustly collect trolleys in dynamic and complex environments. For perception, we first develop a 3D trolley detection method that combines object detection and keypoint estimation. Then, a docking process in a short distance is achieved with an accurate point cloud plane detection method and a novel manipulator design. On the planning side, we formulate the robot's motion planning under a nonlinear model predictive control framework with control barrier functions to improve obstacle avoidance capabilities while maintaining the target in the sensors' field of view at close distances. We demonstrate our design and framework by deploying the system on actual trolley collection tasks, and their effectiveness and robustness are experimentally validated. (Video 11Video demonstration: https://youtu.be/6SwjgGvRtno.)

AA core challenge for an autonomous agent acting in the real world is to adapt its repertoire of skills to cope with its noisy perception and dynamics. To scale learning of skills to long-horizon tasks, robots should be able to learn and later refine their skills in a structured manner through trajectories rather than making instantaneous decisions individually at each time step. To this end, we propose the Soft Actor- Critic Gaussian Mixture Model (SAC-GMM), a novel hybrid approach that learns robot skills through a dynamical system and adapts the learned skills in their own trajectory distribution space through interactions with the environment. Our approach combines classical robotics techniques of learning from demonstration with the deep reinforcement learning framework and exploits their complementary nature. We show that our method utilizes sensors solely available during the execution of preliminarily learned skills to extract relevant features that lead to faster skill refinement. Extensive evaluations in both simulation and real-world environments demonstrate the effectiveness of our method in refining robot skills by leveraging physical interactions, high-dimensional sensory data, and sparse task completion rewards. Videos, code, and pre-trained models are available at http://sac-gmm.cs.uni-freiburg.de.

For collaborative tasks, involving handovers, humans are able to exploit visual, non-verbal cues, to infer physical object properties, like mass, to modulate their actions. In this paper, we investigate how the different levels of liquid inside a cup can be inferred from the observation of the movement of the person handling the cup. We model this mechanism from human experiments and incorporate it in an online human-to-robot handover. Finally, we provide a new dataset with human eye+head+hand motion data for human-to-human handovers and human pick-and-place of a cup with three levels of liquid: empty, half-full, and full of water. Our results show that it is possible to model (non-verbal) signals exchanged by humans during interaction and classify the level of water inside the cup being handed over.

Robot grasping applications are faced with challenges and limitations leading to errors that affect their performance and accuracy. Although these errors are reduced in expensive industrial systems, low-cost robots are still prone to inaccurate perception and execution due to their limited hardware and software capabilities. To mitigate these challenges and limitations, this work develops a Joint-Initiative Supervised Autonomy (JISA) framework for robot grasping. In the proposed system, a human supervisor provides assistance to the robot's perception and planning modules, where the assistance is triggered by requests made by the robot based on its self confidence (SC) metric. Moreover, the human supervisor can also assist the robot in eliminating execution errors based on his/her situation awareness (SA). Through experimental validation, we show that including a human supervisor in the loop for grasping tasks in low-cost robots outperforms full autonomy. In fact, our proposed system showed a marked performance improvement by increasing the end-to-end success rate of the baseline approach, which we implemented JISA on, from 35.0% to 87.6%.

How do we make concrete progress towards de-signing robots that can navigate ethically sensitive contexts? Almost two decades after the word ‘roboethics’ was coined, translating interdisciplinary roboethics discussions into techni-cal design still remains a daunting task. This paper describes our first attempt at addressing these challenges through a roboethics-themed design competition. The design competition setting allowed us to (a) formulate ethical considerations as an engineering design task that anyone with basic programming skills can tackle; and (b) develop a prototype evaluation scheme that incorporates diverse normative perspectives of multiple stakeholders. The initial implementation of the competition was held online at the RO-MAN 2021 conference. The competition task involved programming a simulated mobile robot (TIAGo) that delivers items for individuals in the home environment, where many of these tasks involve ethically sensitive con-texts (e.g., an underage family member asks for an alcoholic drink). This paper outlines our experiences implementing the competition and the lessons we learned. We highlight design competitions as a promising mechanism to enable a new wave of roboethics research equipped with technical design solutions.

Most autonomous driving and robotic applications require retrieving map data around the vehicle's current location. Those maps can cover large areas and are often stored in a compressed form to save memory and allow for efficient transmission. In this paper, we address the problem of place recognition in a compressed point cloud map. To this end, we propose a novel deep neural network architecture that directly operates on a compressed feature representation produced by a compression encoder. This enables us to bypass compute-heavy decompression of the map and exploits the compact as well as descriptive nature of the compressed features. Additionally, we propose an alternative to the commonly used NetVLAD layer to aggregate local descriptors. Here, we utilize an attention mechanism between local features and a latent code. Our experiments suggest that this produces a more descriptive feature representation of the point clouds for place recognition. We experimentally validate all architectural choices we made by our ablation studies and compare our performance to other state-of-the-art baselines on two commonly used datasets.

LiDAR sensors are becoming crucial for achieving higher levels of autonomy. With the current sensor technology, LiDAR sensors are still susceptible to erroneous measurements in adverse weather conditions due to weather artifacts observed in the point cloud data. In this work, we analyze the performance of deep learning LiDAR object detectors in adverse weather conditions. We study the under-researched albeit crucial aspect of deep learning - the size and the content of the training data, as we believe that efforts in data curation bring more benefit than often preferred complex algorithmic enhancements. We argue that with sufficient data, learning-based object detectors are inherently capable of distinguishing between points from a true object and weather-induced artifacts in an end-to-end manner, thus, making explicit, handcrafted and computationally expensive point cloud prefiltering steps obsolete. We corroborate our hypothesis by conducting experiments on a variety of LiDAR object detection architectures over two subsets of training data - one comprising of all weather conditions and the other one comprising of only clear conditions. Contrary to popular belief that object detectors need to be trained on data from adverse weather conditions to be performant in adverse weather, we show that training on datasets with a higher number of annotated objects, predominantly acquired in clear conditions, is sufficient to achieve almost similar or sometimes better KPIs across all weather conditions. This makes the data collection more accessible and convenient compared to adverse weather conditions that are often hardly accessible (e.g., heavy snow and fog). Finally, our proposed methodology streamlines the LiDAR perception pipeline in the direction of keeping well-known end-to-end trainable architectures, removing additional pre-processing blocks that are often handcrafted and bring an additional computational cost.

Medical steerable needles can follow 3D curvilinear trajectories inside body tissue, enabling them to move around critical anatomical structures and precisely reach clinically significant targets in a minimally invasive way. Automating needle steering, with motion planning as a key component, has the potential to maximize the accuracy, precision, speed, and safety of steerable needle procedures. In this paper, we introduce the first resolution-optimal motion planner for steerable needles that offers excellent practical performance in terms of runtime while simultaneously providing strong theoretical guarantees on completeness and the global optimality of the motion plan in finite time. Compared to state-of-the-art steerable needle motion planners, simulation experiments on realistic scenarios of lung biopsy demonstrate that our proposed planner is faster in generating higher-quality plans while incorporating clinically relevant cost functions. This indicates that the theoretical guarantees of the proposed planner have a practical impact on the motion plan quality, which is valuable for computing motion plans that minimize patient trauma.

In this paper we introduce a novel technique that aims to control a two-module bio-inspired soft-robotic arm in order to qualitatively reproduce human demonstrations. The main idea behind the proposed methodology is based on the assumption that a complex trajectory can be derived from the composition and asynchronous activation of learned parameterizable simple movements constituting a knowledge base. The present work capitalises on recent research progress in Movement Primitive (MP) theory in order to initially build a library of Probabilistic MPs (ProMPs), and subsequently to compute on the fly their proper combination in the task space resulting in the requested trajectory. At the same time, a model learning method is assigned with the task to approximate the inverse kinematics, while a replanning procedure handles the sequential and/or parallel ProMPs' asynchronous activation. Taking advantage of the mapping at the primitive-level that the ProMP framework provides, the composition is transferred into the actuation space for execution. The proposed control architecture is experimentally evaluated on a real soft-robotic arm, where its capability to simplify the trajectory control task for robots of complex unmodeled dynamics is exhibited.

Due to hemiparesis, stroke survivors frequently develop a dysfunctional gait that is often characterized by an overall decrease in walking speed and a unilateral decrease in step length. With millions currently affected by this dys-functional gait, robust and effective rehabilitation protocols are needed. Although robotic devices have been used in numerous rehabilitation protocols for gait, the lack of significant afteref-fects that translate to effective therapy makes their application still questionable. This paper proposes a novel type of robot-assisted intervention that results in significant aftereffects that last much longer than any other previous study. With the utilization of a novel robotic device, the Variable Stiffness Treadmill (VST), the stiffness of the walking surface underneath one leg is decreased for a number of steps. This unilateral stiffness perturbation results in a significant aftereffect that is both useful for stroke rehabilitation and often lasts for over 200 gait cycles after the intervention has concluded. More specifically, the aftereffect created is an increase in both left and right step lengths, with the unperturbed step length increasing significantly more than the perturbed. These effects may be helpful in correcting two of the most common issues in post-stroke gait: overall decrease in walking speed and a unilateral shortened step length. The results of this work show that a robot-assisted therapy protocol involving repeated unilateral stiffness perturbations can lead to a more permanent and effective solution to post-stroke gait.

Repeated jumping is crucial to the mobility of jumping robots. In this paper, we extend upon the REBOund jumping robot design, an origami-inspired jumping robot that uses the Reconfigurable Expanding Bistable Origami (REBO) pattern as its body. The robot design takes advantage of the pattern's bistability to jump with controllable timing. For jump repeatedly, we also add self-righting legs that utilize a single motor actuation mechanism. We describe a dynamic model that captures the compression of the REBO pattern and the REBOund self-righting process and compared it to the physical robot. Our experiments show that the REBOund is able to successfully self-right and jump repeatedly over tens of jumps.

Real-time video denoising finds applications in several fields like mobile robotics, satellite television, and surveillance systems. Traditional denoising approaches are more common in such systems than their deep learning-based counterparts despite their inferior performance. The large size and heavy computational requirements of neural network-based denoising models pose a serious impediment to their deployment in real-time applications. In this paper, we propose RepAr-Net, a simple yet efficient architecture for fast video de noising. We propose to use temporally separable encoders to generate feature maps called arsenals that can be cached for reuse. We also incorporate re-parameterizable blocks that improve the representative power of the network without affecting the run-time. We benchmark our model on the Set-8 and 2017 DAVIS-Test datasets. Our model achieves state-of-the-art results with up to 29.62% improvement in PSNR and a 50% decrease in run times over existing methods. Our codes are open-sourced at: github.com/spider-tronix/RepAr-Net.

Robots need the capability of placing objects in arbitrary, specific poses to rearrange the world and achieve various valuable tasks. Object reorientation plays a crucial role in this as objects may not initially be oriented such that the robot can grasp and then immediately place them in a specific goal pose. In this work, we present a vision-based manipulation system, ReorientBot, which consists of 1) visual scene understanding with pose estimation and volumetric reconstruction using an onboard RGB-D camera; 2) learned waypoint selection for successful and efficient motion generation for reorientation; 3) traditional motion planning to generate a collision-free trajectory from the selected waypoints. We evaluate our method using the YCB objects in both simulation and the real world, achieving 93% overall success, 81% improvement in success rate, and 22% improvement in execution time compared to a heuristic approach. We demonstrate extended multi-object rearrangement showing the general capability of the system.