ICRA 2023 论文列表

To operate safely and efficiently alongside human workers, collaborative robots (cobots) require the ability to quickly understand the dynamics of manipulated objects. However, traditional methods for estimating the full set of inertial parameters rely on motions that are necessarily fast and unsafe (to achieve a sufficient signal-to-noise ratio). In this work, we take an alternative approach: by combining visual and force-torque measurements, we develop an inertial parameter identification algorithm that requires slow or “stop-and-go” motions only, and hence is ideally tailored for use around humans. Our technique, called Homogeneous Part Segmentation (HPS), leverages the observation that man-made objects are often composed of distinct, homogeneous parts. We combine a surface-based point clustering method with a volumetric shape segmentation algorithm to quickly produce a part-level segmentation of a manipulated object; the segmented representation is then used by HPS to accurately estimate the object's inertial parameters. To benchmark our algorithm, we create and utilize a novel dataset consisting of realistic meshes, segmented point clouds, and inertial parameters for 20 common workshop tools. Finally, we demonstrate the real-world performance and accuracy of HPS by performing an intricate ‘hammer balancing act’ autonomously and online with a low-cost collaborative robotic arm. Our code and dataset are open source and freely available.

Benchmarking Simultaneous Localization and Mapping (SLAM) algorithms is important to scientists and users of robotic systems alike. But through their many configuration options in hardware and software, SLAM systems feature a vast parameter space that scientists up to now were not able to explore. The proposed SLAM Hive Benchmarking Suite is able to analyze SLAM algorithms in 1000's of mapping runs, through its utilization of container technology and deployment in a cluster. This paper presents the architecture and open source implementation of SLAM Hive and compares it to existing efforts on SLAM evaluation. Furthermore, we highlight the function of SLAM Hive by exploring some open source algorithms on public datasets in terms of accuracy. We compare the algorithms against each other and evaluate how parameters effect not only accuracy but also CPU and memory usage. Through this we show that SLAM Hive can become an essential tool for proper comparisons and evaluations of SLAM algorithms and thus drive the scientific development in the research on SLAM.

Mechanical systems naturally evolve on principal bundles describing their inherent symmetries. The ensuing factorization of the configuration manifold into a symmetry group and an internal shape space has provided deep insights into the locomotion of many robotic and biological systems. On the other hand, the property of differential flatness has enabled efficient, effective planning and control algorithms for various robotic systems. Yet, a practical means of finding a flat output for an arbitrary robotic system remains an open question. In this work, we demonstrate surprising new connections between these two domains, for the first time employing symmetry directly to construct a flat output. We provide sufficient conditions for the existence of a trivialization of the bundle in which the group variables themselves are a flat output. We call this a geometric flat output, since it is equivariant (i.e. it preserves the symmetry) and often global or almost global, properties not typically enjoyed by other flat outputs. In such a trivialization, the motion planning problem is easily solved, since a given trajectory for the group variables will fully determine the trajectory for the shape variables that exactly achieves this motion. We provide a partial catalog of robotic systems with geometric flat outputs and worked examples for the planar rocket, planar aerial manipulator, and quadrotor.

We present the Reflectance Field Map, a reliable real-time method for detecting shiny surfaces, like glass, metal, and mirrors, with lidar. The Reflectance Field Map combines the theory developed for Light Field Mapping, common in computer graphics, with occupancy grid mapping. Like early methods for sonar-based robot mapping, we show how the addition of angular viewpoint information to a standard 2D grid map enables robust mapping in the presence of specular reflections. However unlike previous approaches, our method works in dynamic environments. Additionally, unlike recent approaches for lidar-based mapping of specular surfaces, our approach is sensor-agnostic and has no reliance on either intensity or multi-return measurements. We demonstrate the ability of the Reflectance Field Map to accurately map a campus environment containing numerous pedestrians and significant plate glass, both straight and curved. The algorithm runs in real-time (75+Hz) on a single core of a standard desktop processor. An open source implementation of the algorithm is available at https://github.com/collinej/reflectance_field_map.

Robots have predominantly been used in automating tasks in structured industrial environments, however, with the advances in technology they are starting to take part in roles in dynamic everyday life scenarios. As a result, the tasks executed by robotic systems will also grow in sophistication. Grasping and dexterous manipulation are critical aspects that allow humans to execute these sophisticated tasks, enabling them to interact with their environment. As such, emulating the human hand can be advantageous for interacting with a world designed for humans. However, directly replicating the anatomical structure of the hand produces designs that are fully actuated, expensive, and which require sophisticated controls and sensing to operate efficiently. In this paper, we present two different versions of the New Dexterity adaptive, humanlike robot hand that is capable of executing robust caging grasps under a wide range of environmental uncertainties (e.g., object pose uncertainties). One of the versions has a classic, fixed thumb base while the second one incorporates an additional degree of freedom at the thumb base, which enables a translational motion for repositioning the thumb and adjusting the aperture. This design choice enhances the inhand manipulation capabilities of the robot hand, improving also the power grasping capabilities for larger objects. The performances of the proposed robot hand designs are experimentally validated and compared through three different tests: i) grasping experiments involving everyday-life objects, ii) force experiments that evaluate their force exertion capabilities, and iii) in-hand manipulation experiments that demonstrate and compare their dexterity.

In human-aware navigation, the robot tacitly games with humans, balancing safety and efficiency according to human intentions. Poor balance or bad intent recognition causes the robot to stop conservatively or advance rashly, resulting in a deadlock or even a collision respectively. To address the issue, this paper proposes an improved limit cycle for collaboratively parameterizing human intentions and planning robot motions. The human-robot interaction is modeled as a dynamic chicken game with incomplete information, where the human gaze is introduced to depict the unique characteristics of each person, allowing the robot to approach with different safety margins. Our method is tested in challenging indoor scenarios and outperforms traditional methods in both safety and efficiency. We enable robots to utilize human wisdom to solve problems that cannot be solved on their own. The robot bravely goes through oncoming crowds by getting closer to people with higher attention on it and has the foresight to stably cross in front or behind people.

Testing black-box perceptual-control systems in simulation faces two difficulties. Firstly, perceptual inputs in simulation lack the fidelity of real-world sensor inputs. Secondly, for a reasonably accurate perception system, encountering a rare failure trajectory may require running infeasibly many simulations. This paper combines perception error models-surrogates for a sensor-based detection system-with state-dependent adaptive importance sampling. This allows us to efficiently assess the rare failure probabilities for real-world perceptual control systems within simulation. Our experiments with an autonomous braking system equipped with an RGB obstacle-detector show that our method can calculate accurate failure probabilities with an inexpensive number of simulations. Further, we show how choice of safety metric can influence the process of learning proposal distributions capable of reliably sampling high-probability failures.

Test-time domain adaptation, i.e. adapting source-pretrained models to the test data on-the-fly in a source-free, unsupervised manner, is a highly practical yet very challenging task. Due to the domain gap between source and target data, inference quality on the target domain can drop drastically especially in terms of absolute scale of depth. In addition, unsupervised adaptation can degrade the model performance due to inaccurate pseudo labels. Furthermore, the model can suffer from catastrophic forgetting when errors are accumulated over time. We propose a test-time domain adaptation framework for monocular depth estimation which achieves both stability and adaptation performance by benefiting from both self-training of the supervised branch and pseudo labels from self-supervised branch, and is able to tackle the above problems: our scale alignment scheme aligns the input features between source and target data, correcting the absolute scale inference on the target domain; with pseudo label consistency check, we select confident pixels thus improve pseudo label quality; regularisation and self-training schemes are applied to help avoid catastrophic forgetting. Without requirement of further supervisions on the target domain, our method adapts the source-trained models to the test data with significant improvements over the direct inference results, providing scale-aware depth map outputs that outperform the state-of-the-arts. Code is available at https://github.com/Malefikus/ada-depth.

Is it possible for a synthetic to realistic domain adapted neural network in single image depth estimation to truly generalize on real world data? The resultant, adapted model will only generalize on the realistic domain dataset, which only reflects a small portion of the true, real world. As a result, the network still has to cope with the potential danger of domain shift between the realistic domain dataset and the real world data. Instead, a viable solution is to design the model to be capable of continuously adapting to the distribution of data it receives at test-time. In this paper, we propose a depth estimation method that is capable of adapting to the domain shift at test-time. Our method adapts to the unseen test-time domain, by updating the network using our proposed objective functions. Following former work, we reduce the entropy of the current prediction for refinement and adaptation. We propose a Logit Order Enforcement loss that can prevent the network from deviating into wrong solutions, which can result from the mere reduction of the aforementioned entropy. Qualitative and quantitative results show the effectiveness of our method. Our method reduces the dependency on training data by 5.8× on average, while achieving comparable performance to state-of-the-art unsupervised domain adaptation (UDA) and domain generalization methods (DG) on the KITTI dataset.

We present a new approach to tackle the problem of metamorphic robots' reconfiguration. Given the chain-type metamorphic robot's initial and target configuration, we compute a reconfiguration plan that is provably physically collision-free. Our solution employs a specific heuristic. The robot initially reconfigures to a shape that resembles an octopus with many tentacles. After that, the tentacles gradually reconnect to each other using inverse kinematics, separating one tentacle from the body and keeping the other one connected. This strategy eventually leads to a snake-like structure of the robot. For the target configuration, we compute the reconfiguration plan with the same procedure, however, we reverse the plan to reconfigure the robot from the snake-like structure to the target shape. According to our experimental evaluation, our newly introduced strategy for finding reconfiguration plans is successful. It efficiently finds collision-free plans even for robots consisting of hundreds of modules.

Growing global demand for food, coupled with continuing labor shortages, motivate the need for automated agricultural harvesting. While some specialty crops (e.g., apples, peaches, blueberries) can be harvested via existing harvesting modalities, fruits such as blackberries and raspberries require delicate handling to mitigate fruit damage that could significantly impact marketability. This motivates the development of soft robotic solutions that enable efficient, delicate harvesting. This paper presents the design, fabrication, and feasibility testing of a tendon-driven soft gripping system focused on blackberries, which are a fragile fruit susceptible to post-harvest damage. The gripper is low-cost and small form factor, allowing for the integration of a micro-servo for tendon retraction, a near-infrared (NIR) based blackberry ripeness sensor utilizing the reflectance modality for identifying fully ripe blackberries, and an endoscopic camera for visual servoing. The gripper was used to harvest 139 berries with manual positioning in two separate field tests. Field testing found an average retention force of 2.06 N and 6.08 N for ripe and unripe blackberries, respectively. Sensor tests identified an average reflectance of 16.78 and 21.70 for ripe and unripe blackberries, respectively, indicating a clear distinction between the two ripeness levels. Finally, the soft robotic gripper was integrated onto a UR5 robot arm and successfully harvested fifteen artificial blackberries in a lab setting using visual servoing.

This paper presents an agent-agnostic framework to control swarms of robots tasked with temporal and logical missions expressed as Metric Temporal Logic (MTL) formulas. We consider agents that can receive global commands from a high-level planner, but no inter-agent communication. Moreover, agents are grouped into sub-swarms whose number can vary over the mission time horizon due to splitting and merging. However, a strict upper bound on the maximum number of sub-swarms is imposed to ensure their safe operation in the environment. We propose a two-phase approach. In the first phase, we compute the trajectories of the sub-swarms, splitting, and merging actions using a Mixed Integer Linear Programming approach that ensures the satisfaction of the MTL specification with minimal swarm division over the mission time horizon. Moreover, it enforces the upper bound on the number of sub-swarms. In the second phase, splitting fractions for sub-swarms resulting from splitting actions are computed. A distributed randomized protocol with no interagent communication ensures agent assignments matching the splitting fractions. Finally, we show the operation and performance of the approach in simulations with multiple tasks that require swarm splitting or merging.

Humans possess an innate ability to incorporate tools into our body schema to perform a myriad of tasks not possible with our natural limbs. Human-in-the-loop telerobotic systems (HiLTS) are tools that extend human manipulation capabilities to remote and virtual environments. Unlike most hand-held tools, however, HiLTS often possess complex electromechanical architectures that introduce non-trivial transmission dynamics between the robot's leader and follower, which alter or obfuscate the environment's dynamics. While considerable research has focused on negating or circumventing these dynamics, it is not well understood how capable human operators are at incorporating these transmission dynamics into their sensorimotor control scheme. To begin answering this question, we recruited \mathrm{N}=12\mathrm{N}=12 participants to use a novel reconfigurable teleoperator with varying transmission dynamics to perform a visuo-haptic tracking task. Contrary to our original hypothesis, our findings demonstrate that humans can account for substantial differences in teleoperator transmission dynamics and produce the compensatory strategies necessary to adequately control the teleoperator. These findings suggest that advances in transparency algorithms and haptic feedback approaches must be coupled with control designs that leverage the unique capabilities of the human operator in the loop.

Mobile manipulators have gained attention for the potential in performing large-scale tasks which are beyond the reach of fixed-base manipulators. The Robotic Task Sequencing Problem for mobile manipulators often requires optimizing the motion sequence of the robot to visit multiple targets while reducing the number of base placements. A two-step approach to this problem is clustering the task-space into clusters of targets before sequencing the robot motion. In this paper, we propose a task-space clustering method which formulates the clustering step as a Set Cover Problem using bipartite graph and reachability analysis, then solves it to obtain the minimum number of target clusters with corresponding base placements. We demonstrated the practical usage of our method in a mobile drilling experiment containing hundreds of targets. Multiple simulations were conducted to benchmark the algorithm and also showed that our proposed method found, in practical time, better solutions than the existing state-of-the-art methods.

The integration of variable stiffness actuators (VSA) in robotic systems endows them with intrinsic flexibility and therefore robustness to unknown disturbances. However, this characteristic presents a challenge: choosing the best intrinsic stiffness setting guaranteeing the required force ap-plication capability while keeping the system as adaptable to uncertainties as possible. This paper proposes a method to set the optimal stiffness for a multi-finger VSA hand to perform a desired manipulation task. The task is generically represented as a force (with unknown magnitude) applied along a reference direction. According to the force application's direction and the hand's kinematic state, the fingers assume a certain role to split the collective force application. We employ the endpoint stiffness ellipsoid to analyze the required finger stiffness to fulfill the task. We evaluate the optimized stiffness settings in a door opening application with an iterative adaption of the stiffness behavior to handle the unknown force requirement. The results show a successful collective behavior of the fingers, where the stiffness setting considers a task-oriented force-adaptability trade-off and effective use of independent VSA fingers.

Embodied AI agents in large scenes often need to navigate to find objects. In this work, we study a naturally emerging variant of the object navigation task, hierarchical relational object navigation (HRON), where the goal is to find objects specified by logical predicates organized in a hierarchical structure-objects related to furniture and then to rooms-such as finding an apple on top of a table in the kitchen. Solving such a task requires an efficient representation to reason about object relations and correlate the relations in the environment and in the task goal. HRON in large scenes (e.g. homes) is particularly challenging due to its partial observability and long horizon, which invites solutions that can compactly store the past information while effectively exploring the scene. We demonstrate experimentally that scene graphs are the best-suited representation compared to conventional representations such as images or 2D maps. We propose a solution that uses scene graphs as part of its input and integrates graph neural networks as its backbone, with an integrated task-driven attention mechanism, and demonstrate its better scalability and learning efficiency than state-of-the-art baselines.

Representing and reasoning about uncertainty is crucial for autonomous agents acting in partially observable environments with noisy sensors. Partially observable Markov decision processes (POMDPs) serve as a general framework for representing problems in which uncertainty is an important factor. Online sample-based POMDP methods have emerged as efficient approaches to solving large POMDPs and have been shown to extend to continuous domains. However, these solutions struggle to find long-horizon plans in problems with significant uncertainty. Exploration heuristics can help guide planning, but many real-world settings contain significant task-irrelevant uncertainty that might distract from the task objective. In this paper, we propose STRUG, an online POMDP solver capable of handling domains that require long-horizon planning with significant task-relevant and task-irrelevant uncertainty. We demonstrate our solution on several temporally extended versions of toy POMDP problems as well as robotic manipulation of articulated objects using a neural perception frontend to construct a distribution of possible models. Our results show that STRUG outperforms the current sample-based online POMDP solvers on several tasks.

Crop inspection is a critical part of modern agricultural practices that helps farmers assess the current status of a field and then make crop management decisions. Current crop inspection methods are labour-intensive tasks, which makes them rather slow and expensive to apply. In this paper, we exploit recent advancements in implicit mapping to tackle the challenging context of agricultural environments to create dense maps of crop rows with high enough fidelity to be useful for automated crop inspection. Specifically, we map strawberry and sweet pepper crop rows using RGB images captured by a wheeled mobile field robot inside a greenhouse and then use this data to build 3D maps to document the development of plants and fruits. Our Target-Aware Implicit Mapping system (TAIM) uses a SLAM-based pose initialization strategy for robust pose convergence, an efficient information-guided training sample selection framework for faster loss reduction, and focuses on exploiting training samples for fruit regions of the scene, which are critical for crop inspection tasks, to create more accurate maps in less time.

We study the problem of object retrieval in scenarios where visual sensing is absent, object shapes are unknown beforehand and objects can move freely, like grabbing objects out of a drawer. Successful solutions require localizing free objects, identifying specific object instances, and then grasping the identified objects, only using touch feedback. Unlike vision, where cameras can observe the entire scene, touch sensors are local and only observe parts of the scene that are in contact with the manipulator. Moreover, information gathering via touch sensors necessitates applying forces on the touched surface which may disturb the scene itself. Reasoning with touch, therefore, requires careful exploration and integration of information over time - a challenge we tackle. We present a system capable of using sparse tactile feedback from fingertip touch sensors on a dexterous hand to localize, identify and grasp novel objects without any visual feedback. Videos are available at https://sites.google.com/view/tactofind.