ICRA 2021 论文列表

Can we use reinforcement learning to learn general-purpose policies that can perform a wide range of different tasks, resulting in flexible and reusable skills? Contextual policies provide this capability in principle, but the representation of the context determines the degree of generalization and expressivity. Categorical contexts preclude generalization to entirely new tasks. Goal-conditioned policies may enable some generalization, but cannot capture all tasks that might be desired. In this paper, we propose goal distributions as a general and broadly applicable task representation suitable for contextual policies. Goal distributions are general in the sense that they can represent any state-based reward function when equipped with an appropriate distribution class, while the particular choice of distribution class allows us to trade off expressivity and learnability. We develop an off-policy algorithm called distribution-conditioned reinforcement learning (DisCo RL) to efficiently learn these policies. We evaluate DisCo RL on a variety of robot manipulation tasks and find that it significantly outperforms prior methods on tasks that require generalization to new goal distributions.

Human motion prediction is essential in human-robot interaction. Current research mostly considers the joint dependencies but ignores the bone dependencies and their relationship in the human skeleton, thus limiting the prediction accuracy. To address this issue, we represent the human skeleton as a directed acyclic graph with joints as vertexes and bones as directed edges. Then, we propose a novel directed acyclic graph neural network (DA-GNN) that follows the encoder-decoder structure. The encoder is stacked by multiple encoder blocks, each of which includes a directed acyclic graph computational operator (DA-GCO) to update joint and bone attributes based on the relationship between joint and bone dependencies in the observed human states, and a temporal update operator (TUO) to update the temporal dynamics of joints and bones in the same observation. After progressively implementing the above update process, the encoder outputs the final update result, fed into the decoder. The decoder includes a directed acyclic graph-based gated recurrent unit (DAG-GRU) and a multi-layered perceptron (MLP) to predict future human states sequentially. To the best of our knowledge, this is the first time to introduce the relationship between bone and joint dependencies in human motion prediction. Our experimental evaluations on two datasets, CMU Mocap and Human 3.6m, prove that DA-GNN outperforms current models. Finally, we showcase the efficacy of DA-GNN in a realistic HRI scenario.

Robust and accurate localization plays a key role in autonomous driving and robot applications. To utilize the complementary properties of different sensors, we present a novel tightly-coupled approach to combine the local (stereo cameras, IMU) and global sensors (magnetometer, GNSS). We jointly optimize all the model parameters through one active window. The visual part integrates constraints from static stereo into the photometric bundle adjustment pipeline of dynamic multiview stereo. Accumulating IMU information between keyframes, magnetometer and GNSS measurements are all inserted into the active window as additional constrains among all the keyframes. Through these, our method can realize globally drift-free and locally accurate state estimation. We evaluate the effectiveness of our system on public datasets under with real-world experiments.

In this paper, we present a model predictive controller for a fully actuated aerial manipulator to track a hybrid force and pose trajectory at the end-effector in an aerial interaction task. A force sensor at the end-effector is used to detect contact and to directly control the interaction force. We propose an approach for automatic transition between three operation modes which reflect the state of contact constraints, including free flight and two modes for force control based on static or dynamic friction at the end-effector. This division into three modes allows for different mode-specific controller tunings to optimize the desired performance throughout an interaction task. Results from flight experiments which combine force, position, and attitude tracking, show the performance of the controller in terms of accuracy and precision. The performance is further benchmarked against a hybrid force/impedance controller.

Semantic 3D scene understanding is a fundamental problem in computer vision and robotics. Despite recent advances in deep learning, its application to multi-domain 3D semantic segmentation typically suffers from the lack of extensive enough annotated 3D datasets. On the contrary, 2D neural networks benefit from existing large amounts of training data and can be applied to a wider variety of environments, sometimes even without need for retraining. In this paper, we present ‘Diffuser’, a novel and efficient multi-view fusion framework that leverages 2D semantic segmentation of multiple image views of a scene to produce a consistent and refined 3D segmentation. We formulate the 3D segmentation task as a transductive label diffusion problem on a graph, where multi-view and 3D geometric properties are used to propagate semantic labels from the 2D image space to the 3D map. Experiments conducted on indoor and outdoor challenging datasets demonstrate the versatility of our approach, as well as its effectiveness for both global 3D scene labeling and single RGB-D frame segmentation. Furthermore, we show a significant increase in 3D segmentation accuracy compared to probabilistic fusion methods employed in several state-of-the-art multi-view approaches, with little computational overhead.

A novel 3-dimensional (3-D) alignment method for point-cloud registration is proposed where the time-differential information of the measured points is employed. The new problem turns out to be a novel multi-dimensional optimization. Analytical solution to this optimization is then obtained, which sets the ground of further correspondence matching using k-D trees. Finally, via many examples, we show that the new method owns better registration accuracy in real-world experiments.

A limitation of model-based reinforcement learning (MBRL) is the exploitation of errors in the learned models. Blackbox models can fit complex dynamics with high fidelity, but their behavior is undefined outside of the data distribution. Physics-based models are better at extrapolating, due to the general validity of their informed structure, but underfit in the real world due to the presence of unmodeled phenomena. In this work, we demonstrate experimentally that for the offline model-based reinforcement learning setting, physics-based models can be beneficial compared to high-capacity function approximators if the mechanical structure is known. Physics-based models can learn to perform the ball in a cup (BiC) task on a physical manipulator using only 4 minutes of sampled data using offline MBRL. We find that black-box models consistently produce unviable policies for BiC as all predicted trajectories diverge to physically impossible state, despite having access to more data than the physics-based model. In addition, we generalize the approach of physics parameter identification from modeling holonomic multi-body systems to systems with nonholonomic dynamics using end-to-end automatic differentiation.Videos: https://sites.google.com/view/ball-in-a-cup-in-4-minutes/

Manipulation in a densely cluttered environment creates complex challenges in perception to close the control loop, many of which are due to the sophisticated physical interaction between the environment and the manipulator. Drawing from biological sensory-motor control, to handle the task in such a scenario, tactile sensing can be used to provide an additional dimension of the rich contact information from the interaction for decision making and action selection to manoeuvre towards a target. In this paper, a new tactile-based motion planning and control framework based on bioinspiration is proposed and developed for a robot manipulator to manoeuvre in a cluttered environment. An iterative two-stage machine learning approach is used in this framework: an autoencoder is used to extract important cues from tactile sensory readings while a reinforcement learning technique is used to generate optimal motion sequence to efficiently reach the given target. The framework is implemented on a KUKA LBR iiwa robot mounted with a SynTouch BioTac tactile sensor and tested with real-life experiments. The results show that the system is able to move the end-effector through the cluttered environment to reach the target effectively.

To address the communication and ranging challenges caused by underwater environment, we design dual modal devices for autonomous underwater vehicles (AUVs). The dual-modal design builds upon a co-axial ultrasonic and green laser beams which leverage different signal diverging patterns and different responses in the underwater environment by each modality to achieve robust adaptability. Here we report our recent progress in improving scanning and aiming capabilities for dual-modal beam steering. The core part is our Two-Axis Water-immersible Micro Scanning Mirror (WIMSM). We improve hinge design of WIMSM for larger scanning range. We incorporate high speed Hall effect sensor-based pose feedback channel to enable closed-loop scanning and aiming control. We design ultrasonic-assisted laser handshaking method to help AUVs to acquire optical underwater communication. We have prototyped our devices and tested them in a water tank. The initial results are promising.

This paper describes the development of a 1-DOF kinesthetic force display device in the form of an arm training simulator that replicates the haptic feeling of lead-pipe rigidity in the elbow joint. Patients with lead-pipe rigidity have uniformly elevated muscle tone throughout the range of motion, which is an important clinical sign for diagnosing Parkinson’s disease during a neurological examination. The simulator could provide training opportunities for healthcare trainees to learn and practice the assessment technique for lead-pipe rigidity. The simulator was driven by a series elastic actuator in order to have more accurate joint torque control in a safe and cost-effective manner for rendering abnormal muscle resistance. A mathematical model of lead-pipe rigidity based on hyperbolic tangent was proposed to recreate the elevated muscle resistance at different Unified Parkinson’s Disease Rating Scale (UPDRS) 0-3. Performance of the simulator was evaluated through benchtop tests and rigidity simulation tests. Preliminary results suggested the simulator had good torque control accuracy (an average RMSE < 0.27 Nm) and good fidelity in mimicking clinically-measured lead-pipe rigidity at UPDRS 0-3.

This paper describes a set of software modules and algorithms for maritime object detection and tracking. The approach described here is designed to work in conjunction with various sensors from a maritime surface vessel (e.g. marine RADAR, LIDAR, camera). The described system identifies obstacles from the input sensors, estimates their state, and fuses the obstacle data into a consolidated report. The system is verified using experiments conducted on a live system and successfully demonstrates the ability to detect and track obstacles up to 450m away while operating at 7 fps. The software is open source and available at https://github.com/uml-marine-robotics/asv_perception.

Birds are agile in locomotion and able to move quickly and easily from one place to another. When a bird is on the ground, and a threat approaches, the bird will fly away and escape. An ornithopter robot provides advantages in energy saving, maneuverability, and crash safety. Most flapping robots require an operator or assistance to take off. The goal of this study is to enable self-takeoff from the ground. The developed robot can generate thrust to its body by exceeding its own weight using a simple flapping mechanism and lightweight design. The result of the takeoff experiment showed that the ornithopter robot was able to self-takeoff from the ground without assistance.

Robotic hand exoskeletons can provide assistance to people who suffer from hand functional disability or spinal cord injury (SCI). However, the current hand exoskeletons remain challenging with respect to having a user-friendly design that satisfies human motion with a lightweight structure. Here we propose a method of using topology optimization in the design of finger exoskeletons, which is a lightweight and integrate manufactured exoskeleton. The exoskeleton is designed by generating the topology configuration according to the bending state of the finger. The objective function in the optimization is set to be the maximization of output displacement of the design domain. After obtaining the topology configuration, a conjugate surface flexure hinge is developed as an elastic joint to replace the compliant part of the topologies. Finally, a rigid-compliant parallel exoskeleton is fabricated by a 3D printer and the performance of the mechanism is verified.

Optical colonoscopy is the gold standard for colorectal cancer screening, however even under optimal conditions only 81% of the internal tissue is inspected, in part causing up to 22% of early adenomas to be missed. Blindspots commonly occur at acute bends where the camera’s view is blocked. 3D reconstruction alone is insufficient to assess screening completeness as predictions of both the seen and unseen mucosa are required. Existing works in blindspot detection nevertheless use a highly detailed 3D reconstruction as the first step, the complexity of which degrades processing speed and reliability. We demonstrate that this complexity is not needed to predict whether acute bends have been adequately inspected. We propose a parametric model of the colon with only 2 variables: radius and curvature. By incorporating curvature, our method can predict the occlusion which acute bends cause. We use CT scans from 12 patients which on average contain 20 bends. We use a custom colonoscopy simulator and, assuming known geometry, show that a curved model reliably predicts these blindspots while a non-curved model always misses them. From our frame-by-frame predictions we build a panoramic map of the tissue inspected over the procedure. We show that this correctly identifies all 10 blindspots during 3 minutes of colonoscopy. We envisage that by alerting clinicians to mucosa which they have missed in real time, they can revisit these areas, improving the detection of polyps and cancers.

We present a method for detecting and mapping trees in noisy stereo camera point clouds, using a learned 3D object detector. Inspired by recent advancements in 3-D object detection using a pseudo-lidar representation for stereo data, we train a PointRCNN detector to recognize trees in forest-like environments. We generate detector training data with a novel automatic labeling process that clusters a fused global point cloud. This process annotates large stereo point cloud training data sets with minimal user supervision, and unlike previous pseudo-lidar detection pipelines, requires no 3D ground truth from other sensors such as lidar. Our mapping system additionally uses a Kalman filter to associate detections and consistently estimate the positions and sizes of trees. We collect a data set for tree detection consisting of 8680 stereo point clouds, and validate our method on an outdoors test sequence. Our results demonstrate robust tree recognition in noisy stereo data at ranges of up to 7 meters, on 720p resolution images from a Stereolabs ZED 2 camera. Code and data are available at https://github.com/brian-h-wang/pseudolidar-tree-detection.

Oysters are an essential species in the Chesapeake Bay living ecosystem. Oysters are filter feeders and considered the vacuum cleaners of the Chesapeake Bay that can considerably improve the Bay's water quality. Many oyster restoration programs have been initiated in the past decades and continued to date. Advancements in robotics and artificial intelligence have opened new opportunities for aquaculture. Drone-like ROVs with high maneuverability are getting more affordable and, if equipped with proper sensory devices, can monitor the oysters. This work presents our efforts for videography of the Chesapeake bay bottom using an ROV, constructing a database of oysters, implementing Mask R-CNN for detecting oysters, and counting their number in a video by tracking them.

Using sensor data from multiple modalities presents an opportunity to encode redundant and complementary features that can be useful when one modality is corrupted or noisy. Humans do this everyday, relying on touch and proprioceptive feedback in visually-challenging environments. However, robots might not always know when their sensors are corrupted, as even broken sensors can return valid values. In this work, we introduce the Crossmodal Compensation Model (CCM), which can detect corrupted sensor modalities and compensate for them. CMM is a representation model learned with self-supervision that leverages unimodal reconstruction loss for corruption detection. CCM then discards the corrupted modality and compensates for it with information from the remaining sensors. We show that CCM learns rich state representations that can be used for manipulation policies learned with reinforcement learning, even when input modalities are corrupted during policy rollout in ways not seen during training.

For multi-limbed robots, motion planning with posture and force constraints tends to be a difficult optimization problem due to nonlinearities, which also present extended solve times. We propose a multi-stage optimization framework with data-driven inter-stage coupling constraints to address the nonlinearity. Both clustering and evolutionary approaches to find the McCormick envelope relaxations are used to find the problem-specific parameters. The learned constraints are then used in the prior stages, which provides advanced knowledge of the following stages. This leads to improved solve times and interpretability of the results. The planner is validated through multiple walking and climbing tasks on a 10 kg hexapod robot.

Safety can be listed as one of the most important aspects of every working environment, from the low risky to the most dangerous one. Construction sites can be clearly identified among the riskiest working fields, mainly because several complex and fast maneuvers are executed in a very crowded, dynamic and uncertain scenario. Specifically referring to construction machines, typical operations such as digging, earthmoving, heavy weights loading/unloading, etc. become incredibly risky if the machine stability is not guaranteed by the operator. To this purpose, this paper proposes a practical implementation of a dynamic fall prediction system for excavators based on the concept of the Zero Moment Point (ZMP). A commercially available excavator has been dynamically modelled and equipped with a set of sensors to reconstruct the state of the machine. When the proposed system is running, an alarm rings if a critical condition is reached, thus allowing the operator to avoid risky and potentially harmful situations.