ICRA 2023 论文列表

The application of mechanical and other physical properties to the development of robotic systems that can easily adapt to changing external situations is known as mechanical intelligence. Following this concept, many robot hand designs can produce self-adaptive and versatile grasps with simple underactuated fingers and open-loop control, while mechanical- intelligent strategies for dexterous manipulation are still limited. This paper proposes a mechanical-intelligent technique to facilitate dexterous manipulation, in particular prehensile inhand manipulation. The proposed strategy is based on the generation of complex spatial trajectories of the hand-object system, controlled in open loop with the minimum number of actuators and using simple low-level non-position modes. This approach is exemplified by the rigorous analysis and testing of a three-fingered two-actuator underactuated robot hand, called the helical hand, which is capable of generating helical prehensile in-hand manipulation of diversiform objects under error tolerance controlled by constant speed algorithm.

Modern 3D semantic instance segmentation approaches predominantly rely on specialized voting mechanisms followed by carefully designed geometric clustering techniques. Building on the successes of recent Transformer-based methods for object detection and image segmentation, we propose the first Transformer-based approach for 3D semantic instance segmentation. We show that we can leverage generic Transformer building blocks to directly predict instance masks from 3D point clouds. In our model - called Mask3D - each object instance is represented as an instance query. Using Transformer decoders, the instance queries are learned by iteratively attending to point cloud features at multiple scales. Combined with point features, the instance queries directly yield all instance masks in parallel. Mask3D has several advantages over current state-of-the-art approaches, since it neither relies on (1) voting schemes which require hand-selected geometric properties (such as centers) nor (2) geometric grouping mechanisms requiring manually-tuned hyper-parameters (e.g. radii) and (3) enables a loss that directly optimizes instance masks. Mask3D sets a new state-of-the-art on ScanNet test (+6.2mAP), S3DIS 6-fold (+10.1 mAP), STPLS3D (+11.2 mAP) and ScanNet200 test (+12.4 mAP).

Mobile robots are well suited for environmental surveys because they can travel to any area of interest and react to observations without the need for pre-existing infrastructure or significant setup time. However, vehicle motion constraints limit where and when measurements occur. This is challenging for a single vehicle observing a time-varying phenomenon, such as coastal waves, but the ability to generate a spatiotemporal map would have immediate scientific and engineering applications. In this paper, an uncrewed surface vessel (USV) was used to measure waves on the coast of Lake Ontario, Canada. Data were collected from a low-cost inertial measurement system onboard the USV and processed in an offline Gaussian process regression (GPR) workflow to create a spatiotemporal wave model. Frequency analysis of raw sensor data was used to best select and design kernel functions, and to initialize hyperparameters. The relative speed of the waves limited the ability to make complete wave reconstructions, but GPR captured the dominant periodic components of the waves despite irregularities in the signals. After optimization, the hyperparameters indicate a dominant signal with a wave period of 0.87 \mathbf{s}\mathbf{s}, which concurs with ground truth estimates.

This paper introduces a novel class of hyperredun-dant robots comprised of chains of permanently magnetized spheres enclosed in a cylindrical polymer skin. With their shape controlled using an externally-applied magnetic field, the spherical joints of these robots enable them to bend to very small radii of curvature. These robots can be used as steerable tips for endoluminal instruments. A kinematic model is derived based on minimizing magnetic and elastic potential energy. Simulation is used to demonstrate the enhanced steerability of these robots in comparison to magnetic soft continuum robots designed using either distributed or lumped magnetic material. Experiments are included to validate the model and to demonstrate the steering capability of ball chain robots in bifurcating channels.

We develop a novel hall-effect-based six-axis force/torque (F/T) tactile sensor integrated into the fingertip of robotic hands. When the robotic hands performs the grasping tasks in an unstructured environment, the visual information plays a main role in sensing the external properties of the objects. However, the various intrinsic properties of the objects such as softness, roughness, mass distribution, and weight cannot be measured properly only with the visual information. To detect the various force information in performing diverse tasks, we aim to implement the six-axis F/T fingertip tactile sensor with hall-effect-based principle. The experimental results demonstrate that the proposed sensor can measure the six-axis F/T with average errors of about 3.3%. In addition, it is observed that the effect of stray field can be shielded by applying a soft magnetic shielding film to the sensor.

Transparent object perception is a crucial skill for applications such as robot manipulation in household and laboratory settings. Existing methods utilize RGB-D or stereo inputs to handle a subset of perception tasks including depth and pose estimation. However transparent object perception remains to be an open problem. In this paper, we forgo the unreliable depth map from RGB-D sensors and extend the stereo based method. Our proposed method, MVTrans, is an end-to-end multi-view architecture with multiple perception capabilities, including depth estimation, segmentation, and pose estimation. Additionally, we establish a novel procedural photo-realistic dataset generation pipeline and create a large-scale transparent object detection dataset, Syn-TODD, which is suitable for training networks with all three modalities, RGB-D, stereo and multi-view RGB. https://ac-rad.github.io/MVTrans/

Multi-view radar-camera fused 3D object detection provides a farther detection range and more helpful features for autonomous driving, especially under adverse weather. The current radar-camera fusion methods deliver kinds of designs to fuse radar information with camera data. However, these fusion approaches usually adopt the straightforward concatenation operation between multi-modal features, which ignores the semantic alignment with radar features and sufficient correlations across modals. In this paper, we present MVFusion, a novel Multi-View radar-camera Fusion method to achieve semantic-aligned radar features and enhance the cross-modal information interaction. To achieve so, we inject the semantic alignment into the radar features via the semantic-aligned radar encoder (SARE) to produce image-guided radar features. Then, we propose the radar-guided fusion transformer (RGFT) to fuse our radar and image features to strengthen the two modals' correlation from the global scope via the cross-attention mechanism. Extensive experiments show that MVFusion achieves state-of-the-art performance (51.7% NDS and 45.3% mAP) on the nuScenes dataset. We shall release our code and trained networks upon publication.

Magnetic resonance imaging (MRI)-guided robotic systems offer great potential for new minimally invasive medical tools, including MRI-powered miniature robots. By re-purposing the imaging hardware of an MRI scanner, the magnetic miniature robot could be navigated into the remote part of the patient's body without needing tethered endoscopic tools. However, state-of-art MRI-powered magnetic miniature robots have limited functionality besides navigation. Here, we propose an MRI-powered magnetic miniature capsule robot benefiting from acoustic streaming forces generated by MRI-guided high-intensity focus ultrasound (HIFU) for controlled drug release. Our design comprises a polymer capsule shell with a submillimeter-diameter drug-release hole that captures an air bubble functioning as a stopper. We use the HIFU pulse to initiate drug release by removing the air bubble once the capsule robot reaches the target location. By controlling acoustic pressure, we also regulate the drug release rate for multiple locations targeting during navigation. We demonstrated that the proposed magnetic capsule robot could travel at high speed, up to 1.13 cm/s in ex vivo porcine small intestine, and release drug to multiple target sites in a single operation, using a combination of MRI-powered actuation and HIFU-controlled release. The proposed MRI-guided microrobotic drug release system will greatly impact minimally invasive medical procedures by allowing on-demand targeted drug delivery.

Game theoretic methods have become popular for planning and prediction in situations involving rich multi-agent interactions. However, these methods often assume the existence of a single local Nash equilibria and are hence unable to handle uncertainty in the intentions of different agents. While maximum entropy (MaxEnt) dynamic games try to address this issue, practical approaches solve for MaxEnt Nash equilibria using linear-quadratic approximations which are restricted to unimodal responses and unsuitable for scenarios with multiple local Nash equilibria. By reformulating the problem as a POMDP, we propose MPOGames, a method for efficiently solving MaxEnt dynamic games that captures the interactions between local Nash equilibria. We show the importance of uncertainty-aware game theoretic methods via a two-agent merge case study. Finally, we prove the real-time capabilities of our approach with hardware experiments on a 1/10th scale car platform.

Model predictive control is a powerful tool to generate complex motions for robots. However, it often requires solving non-convex problems online to produce rich behaviors, which is computationally expensive and not always practical in real time. Additionally, direct integration of high dimensional sensor data (e.g. RGB-D images) in the feedback loop is challenging with current state-space methods. This paper aims to address both issues. It introduces a model predictive control scheme, where a neural network constantly updates the cost function of a quadratic program based on sensory inputs, aiming to minimize a general non-convex task loss without solving a non-convex problem online. By updating the cost, the robot is able to adapt to changes in the environment directly from sensor measurement without requiring a new cost design. Furthermore, since the quadratic program can be solved efficiently with hard constraints, a safe deployment on the robot is ensured. Experiments with a wide variety of reaching tasks on an industrial robot manipulator demonstrate that our method can efficiently solve complex non-convex problems with high-dimensional visual sensory inputs, while still being robust to external disturbances.

Autonomous localization in unknown environments is a fundamental problem in many emerging fields and the monocular visual approach offers many advantages, due to being a rich source of information and avoiding comparatively more complicated setups and multisensor calibration. Deep learning opened new venues for monocular odometry yielding not only end-to-end approaches but also hybrid methods combining the well studied geometry with specific deep components. In this paper we propose a monocular odometry that leverages deep depth within a feature based geometrical framework yielding a lightweight frame-to-frame approach with metrically scaled trajectories and state-of-the-art accuracy. The front-end is based on a multihypothesis matcher with perspective correction coupled with deep depth predictions that enables careful feature selection and tracking; especially of ground plane features that are suitable for translation estimation. The back-end is based on point-to-epipolar line minimization for rotation and unit translation estimation, followed by deep depth aided reprojection error minimization for metrically correct translation estimation. Furthermore, we also present a domain shift adaptation approach that allows for generalization over different camera intrinsic and extrinsic setups. The proposed approach is evaluated on the KITTI and KITTI-360 datasets, showing competitive results and in most cases outperforming other state-of-the-art stereo and monocular methods.

Manipulation relationship detection (MRD) aims to guide the robot to grasp objects in the right order, which is important to ensure the safety and reliability of grasping in object stacked scenes. Previous works infer manipulation relationship by deep neural network trained with data collected from a predefined view, which has limitation in visual dislocation in unstructured environments. Multi-view data provide more comprehensive information in space, while a challenge of multi-view MRD is domain shift. In this paper, we propose a novel multi-view fusion framework, namely multi-view MRD network (MMRDN), which is trained by 2D and 3D multi-view data. We project the 2D data from different views into a common hidden space and fit the embeddings with a set of Von-Mises-Fisher distributions to learn the consistent representations. Besides, taking advantage of position information within the 3D data, we select a set of KK Maximum Vertical Neighbors (KMVN) points from the point cloud of each object pair, which encodes the relative position of these two objects. Finally, the features of multi-view 2D and 3D data are concatenated to predict the pairwise relationship of objects. Experimental results on the challenging REGRAD dataset show that MMRDN outperforms the state-of-the-art methods in multi-view MRD tasks. The results also demonstrate that our model trained by synthetic data is capable to transfer to real-world scenarios.

In-space assembly is crucial to creating large-scale space structures and enabling long term space missions. Natural limitations in the size of transportation vehicles and ISRU production facilities necessitate an additive strategy with the size of the typical structural unit being essentially fixed and inversely proportional to the final assembly size. In prior robotic and space assembly examples, reversible mechanical integration of structural modules is typically achieved with actuated alignment and fastening mechanisms onboard every structural module. Additive assembly or manufacturing planning approaches often feature a “build front” that receives new materials or parts and progresses gradually across the target geometry. The system we describe here places much of the alignment and fastener actuation systems onboard a mobile robot that can operate at a build front while companion robots (Scaling Omni-directional Lattice Locomoting Explorer, SOLL-E) provide part or material transportation. The design and evaluation of this Mobile Meta-Material Interior Co-Integrator (MMIC-I), an inchworm-style locomoting robotic assembler, is described here with an emphasis on ease of assembly and a low number of unique parts for a simple design. It is designed to assist in alignment of cuboctahedron structural unit cells with captive fasteners, defining the build front in operation. Adjacent structural unit cells are locked together with specified axial and rotational actuation of the fasteners. Hardware prototypes show that the robot is able to successfully locomote to any indexed location within a lattice structure and bolt together each set of fasteners on any interface.

We present a new traffic dataset, Meteor, which captures traffic patterns and multi-agent driving behaviors in unstructured scenarios. Meteor consists of more than 1000 one-minute videos, over 2 million annotated frames with bounding boxes and GPS trajectories for 16 unique agent categories, and more than 13 million bounding boxes for traffic agents. Meteor is a dataset for rare and interesting, multi-agent driving behaviors that are grouped into traffic violations, atypical interactions, and diverse scenarios. Every video in Meteor is tagged using a diverse range of factors corresponding to weather, time of the day, road conditions, and traffic density. We use Meteor to benchmark perception methods for object detection and multi-agent behavior prediction. Our key finding is that state-of-the-art models for object detection and behavior prediction, which otherwise succeed on existing datasets such as Waymo, fail on the Meteor dataset. Meteor is a step towards developing more sophisticated perception models for dense, heterogeneous, and unstructured scenarios.

In this paper, we propose a novel method to create visuomotor mobile manipulation solutions to long-horizon activities. We propose to leverage the recent advances in robot simulation to train robust visual solutions in simulation that can transfer to the real world. While previous works have shown success applying this procedure to autonomous visual navigation and stationary manipulation, applying it to long-horizon visuomotor mobile manipulation is still an open challenge that demands both perceptual and compositional generalization of multiple skills. In this work, we develop Mobile-EMBER, or M-EMBER, a factorized method that decomposes a long-horizon mobile manipulation activity into a repertoire of primitive visual skills, reinforcement-learns each skill in simulation, and composes these skills to a long-horizon mobile manipulation activity. On a real mobile manipulation robot, we find that M-EMBER completes a long-horizon mobile manipulation activity, cleaning_kitchen, achieving over 50% success rate. This requires successfully planning and executing five factorized, learned visual skills, in sequences of up to 48 skills long.

The dynamics of all real quadrotors inevitably differ even if they are the same product. In particular, the dynamics can change significantly during the flight due to additional device attachments or overheating motors. In this study, we focus on training a low-level controller, which operates in response to dynamics-changes without prior knowledge or fine-tuning of the parameters, using reinforcement learning. We randomize the dynamics of quadrotors in the simulator and train the policy based on dynamics information extracted from the state-action history through recurrent neural networks (RNNs). In addition, our experiment demonstrates the difficulties in applying existing actor-critic structures that extract dynamics information using end-to-end RNNs for unstable quadrotors; hence, we propose a novel structure with better performance. Finally, the excellent performance of the proposed controller is verified by testing experiments that stabilize quadrotors with different dynamics. The experiment videos and the code can be found at https://github.com/jackyoung96/RNN-Quadrotor-controller.

As LiDAR sensors have become ubiquitous, the need for an efficient LiDAR data compression algorithm has increased. Modern LiDARs produce gigabytes of scan data per hour (Fig. 1) and are often used in applications with limited compute, bandwidth, and storage resources. We present a fast, lossless compression algorithm for Li-DAR range and attribute scan sequences including multiple-return range, signal, reflectivity, and ambient infrared. Our algorithm—dubbed “Jiffy”—achieves substantial compression by exploiting spatiotemporal redundancy and sparsity. Speed is accomplished by maximizing use of single-instruction-multiple-data (SIMD) instructions. In autonomous driving, infrastructure monitoring, drone inspection, and handheld mapping benchmarks, the Jiffy algorithm consistently outcompresses competing lossless codecs while operating at speeds in excess of 65M points/sec on a single core. In a typical autonomous vehicle use case, single-threaded Jiffy achieves 6x compression of centimeter-precision range scans at 500+ scans per second. To ensure reproducibility and enable adoption, the software is freely available as an open source library33Software is available here: http://github.com/jsford64/jiffy-compression.

The localization system is an essential element in robotics, which can provide accurate position information. Multiple localization systems can be integrated for reliable localization operations because there are various methods for measuring the position or processing algorithms. Significantly, the track-to-track (T2T) fusion method can fuse multiple localization systems using each system's estimate without accessing the sensor's low data. However, most T2T fusion-based localization systems ignore slowly varying biases, such as drift errors, odometry errors, and offsets among multiple maps. This can degrade the localization performance because a slowly varying bias is directly reflected in the localization estimate. Therefore, a slowly varying bias must be considered in the fusion process to derive reliable estimates. This study proposes a T2T fusion-based localization system that considers a slowly varying bias. First, the slow-varying bias difference between the systems was estimated. Because each localization system can have a different bias, the estimated bias difference was used to align it with the reference system. Second, a fused estimate can be obtained by T2T fusion using biasaligned estimates. The proposed fusion system can also be used without limiting the number of inputs to the localization system. The proposed system was compared with various T2T-based localization fusion algorithms for verification in a simulation environment, and it exhibited the best performance in RMSE error comparison.

Investigations of unmanned aerial vehicles (UAV s) for planetary exploration and payload manipulation have become a strong focus of research within space robotics. Among possible solutions, balloon-based systems possess merits that make them extremely attractive, such as their simple operation mechanism and endured operation time. However, there are many hurdles to overcome to achieve robust trajectory tracking performance for balloon-based applications. In this work, in order to facilitate the control and versatile use of balloons for near-surface planetary payload manipulation, a novel robotic platform and control strategy featuring the coordinated servoing of multiple unmanned ground vehicles (UGVs) to actuate a cable-driven balloon and the suspended payload is proposed. An earthbound prototype and dynamic model of this system are designed to allow for the investigation of payload trajectory tracking performance using a tailored Model Predictive Controller in simulation and experiment.