ICRA 2023 论文列表

Ocean observation is an emerging field, and sailing robots have several promising features (e.g., long-range sailing, environmental friendliness, energy-saving and low-noise) to perform tasks. In this paper, we define an ocean observation mission in a restricted target area as a station keeping problem. Inspired by an orientation-restricted Dubins path method, the robot keeps sailing and collecting data in a smooth reciprocation, where the trajectories consist of sailing against wind segments and turning downwind parts divided by a goal area and an acceptable area. The upwind sailing segments are of interest for data acquisition. However, the system stability can not be guaranteed during the whole reciprocation especially for sailing outside the goal area. Hereby, we refer to a switched systems approach and propose a desired heading generation scheme to realize safe and stable control in both areas. The stability for subsystems is proved with Lyapunov-like functions. The stable station keeping scheme is verified in both simulation and real experiments. Finally, we completed continuous and effective observation within 50 minutes in the goal area with a radius of 50 meters by a catamaran robot named OceanVoy460.

This study aims to design a motion/force controller for an aerial manipulator which guarantees the tracking of time-varying motion/force trajectories as well as the stability during the transition between free and contact motions. To this end, we model the force exerted on the end-effector as the Kelvin-Voigt linear model and estimate its parameters by recursive least-squares estimator. Then, the gains of the disturbance-observer (DOB)-based motion/force controller are calculated based on the stability conditions considering both the model uncertainties in the dynamic equation and switching between the free and contact motions. To validate the proposed controller, we conducted the time-varying motion/force tracking experiments with different approach speeds and orientations of the surface. The results show that our controller enables the aerial manipulator to track the time-varying motion/force trajectories.

Modern collaborative robotic applications require robot motions that are predictable for human coworkers. Therefore, trajectories often need to be planned in task space rather than configuration space (\mathcal{C}\mathcal{C}-space). While the interpolation of translations in Euclidean space is straightforward, the interpolation of rotations in SO(3)SO(3) is more complex. Most approaches originating from computer graphics do not exhibit the often desired \mathcal{C}^{2}\mathcal{C}^{2}-continuity in robotics. Our main contribution is a \mathcal{C}^{2}\mathcal{C}^{2}-continuous, zero-clamped interpolation scheme for quaternions that computes a fast synchronized motion given a set of waypoints. As a second contribution, we present modifications to two state-of-the-art quaternion interpolation schemes, Spherical Quadrangle Interpolation (SQUAD) and Spherical Parabolic Blends (SPB), to enable them to compute \mathcal{C}^{2}\mathcal{C}^{2}-continuous, zero-clamped trajectories. In experiments, we demonstrate that for the time optimization of trajectories, our approach is computationally efficient and at the same time computes smooth trajectory profiles.

Semantic segmentation for robotic systems can enable a wide range of applications, from self-driving cars and augmented reality systems to domestic robots. We argue that a spherical representation is a natural one for egocentric pointclouds. Thus, in this work, we present a novel framework exploiting such a representation of LiDAR pointclouds for the task of semantic segmentation. Our approach is based on a spherical convolutional neural network that can seamlessly handle observations from various sensor systems (e.g., different LiDAR systems) and provides an accurate segmentation of the environment. We operate in two distinct stages: First, we encode the projected input pointclouds to spherical features. Second, we decode and back-project the spherical features to achieve an accurate semantic segmentation of the pointcloud. We evaluate our method with respect to state-of-the-art projection-based semantic segmentation approaches using well-known public datasets. We demonstrate that the spherical representation enables us to provide more accurate segmentation and to have a better generalization to sensors with different field-of-view and number of beams than what was seen during training.

Industry 4.0 facilitates a high number of product variants, posing significant challenges for modern manufacturing. One of them is the automatic creation of assembly sequences. This can be achieved with the assembly-by-disassembly (AbD) approach, which is currently highly inefficient. We aim at speeding up AbD by leveraging deep learning. AbD relies on iteratively testing parts for removal, which makes the order in which parts are tested highly relevant for its run-time. We optimize this order by training a graph neural network (GNN) based on the shape of parts and the shape of local part connections. For each part, it predicts a removability probability. We use these probabilities to optimize the order in which parts are tested for removal. This reduces the number of parts tested by approximately 64%-90%, depending on the tested product. Further improvements are achieved by combining our approach with bookkeeping, another approach for speeding up AbD. Finally, we separately analyze the impact of the parts and their connections on the removability probabilities predicted by the GNN. We found that most of the important information regarding a part's removability can be derived from its connections alone.

Communication technologies enable coordination among connected and autonomous vehicles (CAVs). However, it remains unclear how to utilize shared information to improve the safety and efficiency of the CAV system in dynamic and complicated driving scenarios. In this work, we propose a framework of constrained multi-agent reinforcement learning (MARL) with a parallel Safety Shield for CAVs in challenging driving scenarios that includes unconnected hazard vehicles. The coordination mechanisms of the proposed MARL include information sharing and cooperative policy learning, with Graph Convolutional Network (GCN)-Transformer as a spatial-temporal encoder that enhances the agent's environment awareness. The Safety Shield module with Control Barrier Functions (CBF)-based safety checking protects the agents from taking unsafe actions. We design a constrained multi-agent advantage actor-critic (CMAA2C) algorithm to train safe and cooperative policies for CAVs. With the experiment deployed in the CARLA simulator, we verify the performance of the safety checking, spatial-temporal encoder, and coordination mechanisms designed in our method by comparative experiments in several challenging scenarios with unconnected hazard vehicles. Results show that our proposed methodology significantly increases system safety and efficiency in challenging scenarios.

A domain shift exists between the distributions of large scale, outdoor lidar datasets due to being captured using different types of lidar sensors, in different locations, and under varying weather conditions. Inclement weather in particular affects the quality of lidar data, adding artifacts such as scattered and missed points, leading to a drop in performance of 3D object detection networks trained on standard lidar datasets. Domain adaptation methods seek to adapt source-trained neural networks to a target domain. Pseudo-label based self training approaches are popular methods for source-free unsupervised domain adaptation. However, their efficacy depends on the quality of the labels generated by the source trained model. These labels may be incorrect with high confidence, rendering thresholding methods ineffective. In order to avoid reinforcing errors caused by label noise, we propose an uncertainty-aware mean teacher framework which implicitly filters incorrect pseudo-labels during training. Leveraging model uncertainty allows the mean teacher network to perform implicit filtering by down-weighing losses corresponding to uncertain pseudo-labels. Effectively, we perform automatic soft-sampling of pseudo-labeled data while aligning predictions from the student and teacher networks. We demonstrate our domain adaptation method on an adverse weather dataset created by augmenting lidar scenes from KITTI with rain, snow, and fog and show that it out-performs current domain adaptation frameworks. We make our code publicly available 11https://github.com/deeptibhegde/UncertaintyAwareMeanTeacher.

Developing embodied agents in simulation has been a key research topic in recent years. Exciting new tasks, algorithms, and benchmarks have been developed in various simulators. However, most of them assume deaf agents in silent environments, while we humans perceive the world with multiple senses. We introduce Sonicverse, a multisensory simulation platform with integrated audio-visual simulation for training household agents that can both see and hear. Sonicverse models realistic continuous audio rendering in 3D environments in real-time. Together with a new audio-visual VR interface that allows humans to interact with agents with audio, Sonicverse enables a series of embodied AI tasks that need audio-visual perception. For semantic audio-visual navigation in particular, we also propose a new multi-task learning model that achieves state-of-the-art performance. In addition, we demonstrate Sonicverse's realism via sim-to-real transfer, which has not been achieved by other simulators: an agent trained in Sonicverse can successfully perform audio-visual navigation in real-world environments. Sonicverse is available at: https://github.com/StanfordVL/Sonicverse.

Robot end effectors with proximity detection and contact sensing capabilities can reactively position the gripper to align objects and ensure successful grasps. In this paper, we introduce SonicFinger, an acoustic aura based sensing system capable of full-surface pre-touch and contact sensing. A single piezoelectric transducer embedded within a novel 3D printed finger is excited using a monotone to create an acoustic aura encompassing the finger; this enables pre-touch sensing and gripper alignment, while changes in finger-transducer acoustic coupling indicate contact. SonicFinger is low-cost, compact, and easy to manufacture and assemble. Sensing capabilities are evaluated using a set of objects with various physical properties such as optical reflectivity, dielectric constants, mechanical properties, and acoustic absorption. A dataset with over 8,000 proximity and contact events is collected. Our system shows a pre-touch detection true positive rate (TPR) of 92.4% and a true negative rate (TNR) of 95.3%. Contact detection experiments show a TPR of 93.7% and a TNR of 98.7%. Furthermore, pretouch detection information from Sonic Finger is used to adjust the robot grippers pose to align a target object at the center of both fingers.

Tactile sensors are becoming more prevalent in numerous research domains, including robotics, human-robot interaction, and grasping. As the development of customized soft tactile skin for various applications continues to gain momentum, there is an increasing demand for the automation of design and manufacturing processes based on user specifications. Our work presents a partially automated framework for designing and customizing silicone-based skin-like sensors for objects of arbitrary shapes. We assess the performance of stretch and contact sensors featuring custom patterns on complex surfaces, subjecting them to position control, grasping, and manipulation scenarios. Our study's findings demonstrate the feasibility of fabricating skin-like sensors effectively within a semi-automated framework, with potential applications in the aforementioned research domains.

We investigate and develop algorithms for social fairness in coverage control problems. Existing coverage control methods are efficient, optimizing the average expected distance from any event to the nearest robot. However, in societal applications like disaster response or transportation, these conventional objectives lead to disparate coverage costs with respect to different groups within a population. We formulate social fairness for coverage control as the minimization of the maximum coverage cost among a set of groups within a population. Our approach uses Voronoi iteration to solve this novel problem by approximating the non-differentiable objective with the log-sum-exp and defining a gradient based controller that prioritizes fairness while also optimizing average performance when disparities between groups are low. We show convergence properties of this proposed control law and demonstrate the approach in simulations of randomly generated population densities as well as environments generated from U.S. census data on population rates and demographics. Our approach provides greater fairness than existing methods while maintaining similar computational time and convergence properties.

This work proposes a novel framework for socially-aware robot navigation in dynamic, crowded environments using a Deep Inverse Reinforcement Learning. To address the social navigation problem, our multi-modal learning based planner explicitly considers social interaction factors, as well as social-awareness factors, into the DIRL pipeline to learn a reward function from human demonstrations. Moreover, we propose a novel trajectory ranking score using the sudden velocity change of pedestrians around the robot to address the sub-optimality in human demonstrations. Our evaluation shows that this method can successfully make a robot navigate in a crowded social environment and outperforms the state-of-art social navigation methods in terms of the success rate, navigation time, and invasion rate.

Adverse weather has raised a big challenge for autonomous vehicles. Unreliable measurements due to sensor degradation could seriously affect the performance of autonomous driving tasks, such as perception and localization. In this work, we study sensor degradation in rainy weather and present a novel method that evaluates the uncertainty for each laser measurement from a 3D LiDAR. With uncertainty estimation, downstream tasks that rely on LiDAR input (e.g., perception or localization) can increase their reliability by adjusting their reliance on laser measurements with varying fidelity. Alternatively, uncertainty estimation can be used for sensor performance evaluation. Our proposed method, SmartRainNet, uses an attention-based Mixture Density Network to model the dependence between neighboring laser measurements and then calculate the probability density for each laser measurement as an uncertainty score. We evaluate SmartRainNet on synthetic and naturalistic sensor degradation datasets and provide qualitative and quantitative results to demonstrate the effectiveness of our method in evaluating uncertainty. Finally, we demonstrate three practical applications of uncertainty estimation to address autonomous driving challenges in rainy weather.

In this paper, we propose a novel learning framework for autonomous systems that uses a small amount of “auxiliary information” that complements the learning of the main modality, called “small-shot auxiliary modality distillation network (AMD-S-Net)”. The AMD-S-Net contains a two-stream framework design that can fully extract information from different types of data (i.e., paired/unpaired multi-modality data) to distill knowledge more effectively. We also propose a novel training paradigm based on the “reset operation” that enables the teacher to explore the local loss landscape near the student domain iteratively, providing local landscape information and potential directions to discover better solutions by the student, thus achieving higher learning performance. Our experiments show that AMD-S-Net and our training paradigm outperform other SOTA methods by up to 12.7% and 18.1% improvement in autonomous steering, respectively.

Precise localization is critical for autonomous vehicles. We present a self-supervised learning method that employs transformers for the first time for the task of outdoor localization using LiDAR data. We propose a pre-text task that reorganizes the slices of a 360° LiDAR scan to leverage its axial properties. Our model, called Slice Transformer, employs multi-head attention while systematically processing the slices. To the best of our knowledge, this is the first instance of leveraging multi-head attention for outdoor point clouds. We additionally introduce the Perth-Wadataset, which provides a large-scale LiDAR map of Perth city in Western Australia, covering ~4km2area. Localization annotations are provided for Perth - Wa.The proposed localization method is thoroughly evaluated on Perth-WA and Appollo-SouthBay datasets. We also establish the efficacy of our self-supervised learning approach for the common downstream task of object classification using ModelNet40 and ScanNN datasets. The code and Perth-WA data will be publicly released.

Skill-based programming has proven to be advantageous for assembly tasks, but still requires expert knowledge, especially for force-controlled applications. However, it is error-prone due to the multitude of parameters, e.g. different coordinate frames and either position-, velocity- or force-controlled motions on the axes of a frame. We propose a mixed reality based solution, which systematically visualizes the geometric constraints of advanced high-level skills directly in the real-world robotic environment and provides a user interface to create applications efficiently and safely in mixed reality. Therefore, state-machine information is also visualized, and a holographic digital twin allows the user to ad-hoc validate the program via force-enabled simulation. The approach is evaluated on a top hat rail mounting task, proving the capability of the system to handle advanced assembly programming tasks efficiently and tangibly.

We propose a method that simultaneously estimates and controls extrinsic contact with tactile feedback. The method enables challenging manipulation tasks that require controlling light forces and accurate motions in contact, such as balancing an unknown object on a thin rod standing upright. A factor graph-based framework fuses a sequence of tactile and kinematic measurements to estimate and control the interaction between gripper-object-environment, including the location and wrench at the extrinsic contact between the grasped object and the environment and the grasp wrench transferred from the gripper to the object. The same framework simultaneously plans the gripper motions that make it possible to estimate the state while satisfying regularizing control objectives to prevent slip, such as minimizing the grasp wrench and minimizing frictional force at the extrinsic contact. We show results with sub-millimeter contact localization error and good slip prevention even on slippery environments, for multiple contact formations (point, line, patch contact) and transitions between them. See supplementary video and results at https://sites.google.com/view/sim-tact.

Aerial manipulation describes a process that includes physical interaction between an unmanned aircraft system (UAS) and its environment. We aim to apply aerial manipulation to sample leaves and small branches from rain forest trees. Current approaches to aerial manipulation involve extended periods of UAS-environment interaction, during which forces and moments can lead to a loss in attitude or position control in underactuated multicopters. By adapting intelligent foot placement strategies found in dynamically stable hopping robots, this work proposes a strategy involving carefully managed intentional collisions between the UAS and its environment. We designed an attitude controller denoted a Velocity Matching controller that aligns a UAS-mounted pogo-stick foot with the center of mass velocity vector during collision approach to maximize UAS ability to recover a hover state after collision. We propose the use of a flight envelope involving altitude and horizontal speed states to assess recoverability prior to initiating each approach to collision. We identify this flight envelope from a simulation study built on a model of flight in Conventional Waypoint Following and Velocity Matching control modes as well as a model of collision response. Experimental flight testing evaluates the simulation-based envelope resulting in an actual envelope that is somewhat smaller but similarly shaped to the envelope identified in simulation.

Lower-limb exoskeletons are wearable devices whose main purposes are human rehabilitation and bilateral locomotion assistance. In particular, there is a growing interest for their use to symmetrize the gait of hemiparetic patients. This often consists in using the kinematics of the less affected side as a reference for the most affected one. In this work, we followed this approach to design a symmetrization algorithm using the formalism of motor primitives, i.e. a low-dimensional set of signals that provide the desired assistance through their combination. The amount of variables to be stored in memory is thus intrinsically limited, and this framework is particularly adapted to include other modes of assistance and/or transitions between locomotion tasks. In this paper, we report the preliminary validation of this newly developed algorithm with a hip exoskeleton and a single participant replicating hemiparetic walking. Results show that the algorithm effectively managed to reduce both temporal and spatial gait asymmetry.