IROS 2022 Papers

This paper presents a novel algorithm for monitoring marine environments utilizing a resource-constrained robot. Collecting water quality data from large bodies of water is paramount for monitoring the ecosystem's health, particularly for predicting harmful cyanobacteria blooms. The large spatial dimensions of such bodies of water and the slow varying of water quality parameters make exhaustive, complete coverage impractical and unnecessary. This work explores a new strategy for efficiently measuring water quality quantities with an autonomous surface vehicle (ASV). The method utilizes the medial axis of the water body producing a guideline for the ASV trajectory that visits representative areas of the environment. The proposed method ensures data collection in the narrower parts of the lake, where researchers have historically observed harmful blooms while also visiting open water areas. It also presents an analysis of the Spatio-temporal sensitivity of the target sensor. A comparison with the traditional lawnmower algorithm demonstrates that the conventional BCD-based complete coverage method cannot sample the small coves of a lake. As such, we show that the proposed method captures more diverse regions of the area with a partial coverage technique. Offline analysis of several lakes and reservoirs and results from field deployments at Lake Murray, SC, USA, demonstrate the proposed method's effectiveness.

This paper mainly studies the localization and mapping of range sensing robots in the confidence-rich map (CRM) and then extends it to provide a full state estimate for information-theoretic exploration. Most previous works about active simultaneous localization and mapping and exploration always assumed the known robot poses or utilized inaccurate information metrics to approximate pose uncertainty, resulting in imbalanced exploration performance and efficiency in the unknown environment. This inspires us to extend the confidence-rich mutual information (CRMI) with measurable pose uncertainty. Specifically, we propose a Rao- Blackwellized particle filter-based localization and mapping scheme (RBPF -CLAM) for CRM, then we develop a new closed-form weighting method to improve the localization accuracy without scan matching. We further derive the uncertain CRMI (UCRMI) with the weighted particles by a more accurate approximation. Simulations and experimental evaluations show the localization accuracy and exploration performance of the proposed methods.

Visual Servoing has been effectively used to move a robot into specific target locations or to track a recorded demonstration. It does not require manual programming, but it is typically limited to settings where one demonstration maps to one environment state. We propose a modular approach to extend visual servoing to scenarios with multiple demonstration sequences. We call this conditional servoing, as we choose the next demonstration conditioned on the observation of the robot. This method presents an appealing strategy to tackle multi-step problems, as individual demonstrations can be combined flexibly into a control policy. We propose different selection functions and compare them on a shape-sorting task in simulation. With the reprojection error yielding the best overall results, we implement this selection function on a real robot and show the efficacy of the proposed conditional servoing. For videos of our experiments, please check out our project page: https://lmb.informatik.uni-freiburg.de/projects/conditional_servoing/

Using Domain Randomized synthetic data for training deep learning systems is a promising approach for addressing the data and the labeling requirements for supervised techniques to bridge the gap between simulation and the real world. We propose a novel approach for generating and applying class-specific Domain Randomization textures by using randomly cropped image patches from real-world data. In evaluation against the current Domain Randomization texture application techniques, our approach outperforms the highest performing technique by 4.94 AP and 6.71 AP when solving object detection and semantic segmentation tasks on the YCB-M [1] real-world robotics dataset. Our approach is a fast and inexpensive way of generating Domain Randomized textures while avoiding the need to handcraft texture distributions currently being used.

Gravity based caging grasps are robotic grasps where the robot hand passively supports an object against gravity. When a robot hand supports an object at a local minimum of the object gravitational energy, the robot hand forms a basket like grasp of the object. Any object movement in a basket grasp requires an increase of the object gravitational energy, thus allowing secure object pickup and transport with robot hands that use a small number fingers. The basket grasp depth measures the minimal additional energy the object must acquire to escape the basket grasp. This paper describes a computation scheme that determines the depth of entire sets of candidate basket grasps associated with alternative finger placements on the object boundary before pickup. The computation relies on categorization of escape stances that mark the basket grasp depth: double-support escapes are first analyzed and computed, then single-support escapes are analyzed and computed. The minimum energy combination of both types of escape stances defines the depth of entire sets of candidate basket grasps, which is then used to identify the deepest and hence most secure basket grasp. The computation scheme is fully implemented and demonstrated on several examples with reported run-times.

Safety guarantees in motion planning for autonomous driving typically involve certifying the trajectory to be collision-free under any motion of the uncontrollable participants in the environment, such as the human-driven vehicles on the road. As a result they usually employ a conservative bound on the behavior of such participants, such as reachability analysis. We point out that planning trajectories to rigorously avoid the entirety of the reachable regions is unnecessary and too restrictive, because observing the environment in the future will allow us to prune away most of them; disregarding this ability to react to future updates could prohibit solutions to scenarios that are easily navigated by human drivers. We propose to account for the autonomous vehicle's reactions to future environment changes by a novel safety framework, Comprehensive Reactive Safety. Validated in simulations in several urban driving scenarios such as unprotected left turns and lane merging, the resulting planning algorithm called Reactive ILQR demonstrates strong negotiation capabilities and better safety at the same time.

For autonomous agents to act as trustworthy partners to human users, they must be able to reliably communicate their competency for the tasks they are asked to perform. Towards this objective, we develop probabilistic world models based on deep generative modelling that allow for the simulation of agent trajectories and accurate calculation of tasking outcome probabilities. By combining the strengths of conditional variational autoencoders with recurrent neural networks, the deep generative world model can probabilistically forecast trajectories over long horizons to task completion. We show how these forecasted trajectories can be used to calculate outcome probability distributions, which enable the precise assessment of agent competency for specific tasks and initial settings.

Extended Kalman filtering is a common approach to achieve floating base estimation of a humanoid robot. These filters rely on measurements from an Inertial Measurement Unit (IMU) and relative forward kinematics for estimating the base position-and-orientation and its linear velocity along with the augmented states of feet position-and-orientation. We refer to such filters as flat-foot filters. However, the availability of only partial measurements often poses the question of consistency in the filter design. In this paper, we perform an experimental comparison of state-of-the-art flat-foot filters based on the representation choice of state, observation, matrix Lie group error and system dynamics evaluated for filter consistency and trajectory errors. The comparison is performed over simulated and real-world experiments conducted on the iCub humanoid platform. It is observed that filters on Lie groups that exploit properties of invariant filtering tend to perform better as consistent estimators while discrete-time filters in general provide higher accuracy along observable directions.

Learning state representations enables robotic planning directly from raw observations such as images. Several methods learn state representations by utilizing losses based on the reconstruction of the raw observations from a lower-dimensional latent space. The similarity between observations in the space of images is often assumed and used as a proxy for estimating similarity between the underlying states of the system. However, observations commonly contain task-irrelevant factors of variation which are nonetheless important for reconstruction, such as varying lighting and different camera viewpoints. In this work, we define relevant evaluation metrics and perform a thorough study of different loss functions for state representation learning. We show that models exploiting task priors, such as Siamese networks with a simple contrastive loss, outperform reconstruction-based representations in visual task planning in case of task-irrelevant factors of variations.

In this paper we present swimming and modeling for Trident, a three-link lamprey inspired robot that is able to climb on flat smooth walls. We explore two gaits proposed to work for linear swimming, and three gaits for turning maneuvers. We compare the experimental results obtained from these swimming experiments with two different reduced order fluid interaction models, one a previously published potential flow model, and the other a slender cylinder model we developed. We find that depending on the the parameters of swimming chosen, we are able to move forward, backward and sideways with a peak speed of 2.5 cm/s. We identify the conditions when these models apply and aspects that will require additional complexity.

In this paper, a novel prototype for hanging produce harvesting is presented, that is productive, versatile, and robust. In our methodology, the robot-mounted tube approaches, and eventually surrounds the produce of interest at the entry side, that can be as small as the produce diameter, plus a small margin. The stem is then cut by a laser beam, with the optics set up for a distant focal point. Such arrangement allows for minimal hardware at the produce-entry side and in turn, the interaction, and possible disturbance to the local environment. This is essential for fruit reachability and avoiding it dislocation. Experiments has been conducted to drive the laser power to time of cut relation, as well as successful demonstration to sample harvest strawberry.

In this paper, we study the effects of impaired communications on the performances of auction-based task allocation in a dynamic surveillance scenario. We propose a novel connectivity term to include in the bid valuation formula, that aims at improving communications in the multi-robot team. We evaluate our method as well as another state-of-the-art method using robot inter-distance to maintain communication, on randomly generated scenarios and on a real-world scenario. We demonstrate that including our connectivity term in the bid valuation formula improves the performances of the auction scheme.

Flexible Endoscopes (FEs) for colonoscopy present several limitations due to their inherent complexity, resulting in patient discomfort and lack of intuitiveness for clinicians. Robotic FEs with autonomous control represent a viable solution to reduce the workload of endoscopists and the training time while improving the procedure outcome. Prior works on autonomous endoscope FE control use heuristic policies that limit their generalisation to the unstructured and highly deformable colon environment and require frequent human intervention. This work proposes an image-based FE control using Deep Reinforcement Learning, called Deep Visuomotor Control (DVC), to exhibit adaptive behaviour in convoluted sections of the colon. DVC learns a mapping between the images and the FE control signal. A first user study of 20 expert gastrointestinal endoscopists was carried out to compare their navigation performance with DVC using a realistic virtual simulator. The results indicate that DVC shows equivalent performance on several assessment parameters, being more safer. Moreover, a second user study with 20 novice users was performed to demonstrate easier human supervision compared to a state-of-the-art heuristic control policy. Seamless supervision of colonoscopy procedures would enable endoscopists to focus on the medical decision rather than on the control of FE.

The paper presents a practical approach for planning trajectories for multiple vehicles where both collision avoidance and minimum travelling time are simultaneously considered. It is first proposed to exploit the mixed-integer programming (MIP) approach to formulate the collision avoidance paradigm, where the linear dynamic models are utilized to derive the linear constraints. Moreover, travelling time of each vehicle is compromised among them and set to be minimized so that all the vehicles can practically reach the expected destinations at the shortest time. Unfortunately, the formulated optimization problem is NP-hard. In order to effectively address it, we propose to employ the alternating direction method of multipliers (ADMM), which can share the computational burdens to distributive optimization solvers. Thus, the proposed method can enable each vehicle to obtain an expected trajectory in a practical time. Convergence of the proposed algorithm is also discussed. To verify effectiveness of our approach, we implemented it in a numerical example, where the obtained results are highly promising.

Soft robots can safely interact with environments because of their mechanical compliance. Self-collision is also employed in the modern design of soft robots to enhance their performance during different tasks. However, developing an efficient and reliable simulator that can handle the collision response well, is still a challenging task in the research of soft robotics. This paper presents a collision-aware simulator based on geometric optimization, in which we develop a highly efficient and realistic collision checking / response model incorporating a hyperelastic material property. Both actuated deformation and collision response for soft robots are formulated as geometry-based objectives. The collision-free body of a soft robot can be obtained by minimizing the geometry-based objective function. Unlike the FEA-based physical simulation, the proposed pipeline performs a much lower computational cost. Moreover, adaptive remeshing is applied to achieve the improvement of the convergence when dealing with soft robots that have large volume variations. Experimental tests are conducted on different soft robots to verify the performance of our approach.

To safely deploy legged robots in the real world it is necessary to provide them with the ability to reliably detect unexpected contacts and accurately estimate the corresponding contact force. In this paper, we propose a collision detection and identification pipeline for a quadrupedal manipulator. We first introduce an approach to estimate the collision time span based on band-pass filtering and show that this information is key for obtaining accurate collision force estimates. We then improve the accuracy of the identified force magnitude by compensating for model inaccuracies, unmodeled loads, and any other potential source of quasi-static disturbances acting on the robot. We validate our framework with extensive hardware experiments in various scenarios, including trotting and additional unmodeled load on the robot.

This paper presents a path planning refinement technique that allows the efficient collision and rollover-free motion planning for mobile manipulator robots working on rough terrain. First, the necessary theoretical background on a mobile manipulator's kinematics and dynamic stability measure is introduced. Then, after the brief introduction of the sampling-based path planning problem, the additional refinement stage and its problem formulation will be introduced. Within the refinement stage, the novel Bézier control point addition method is introduced to allow for fast, collision and rollover-free path smoothing using curvature-continuous parametrized curves. Analytical proofs and simulated comparisons are provided in the paper to show effectiveness. The beneficial effect of the refined path on trajectory planning will also be demonstrated through simulation.

Collective decision-making enables self-organizing robot swarms to act autonomously on a swarm level and is essential to coordinate their actions as a whole. When robots only share and communicate information locally a distributed and decentralized approach is required. In a previous paper [4], an efficient method based on a distributed Bayesian algorithm was created to distinguish a binary environment. We extended it to have the capability of dealing with dynamic environments. Therefore, it must avoid global lock-in states. In many realistic applications the robot swarm needs to adapt to (collectively) measurable changes at runtime by revising previous collective decisions. The trade-off between decision-making speed and readiness to revise previous decisions is a seemingly unavoidable challenge. We present our extension of the former approach and study how this trade-off can efficiently be balanced.