IROS 2020 论文列表

Autonomous vehicles often rely on high-definition (HD) maps to navigate around. However, lane markings (LMs) are not necessarily static objects due to wear & tear from usage and road reconstruction & maintenance. Therefore, the wrong matching between LMs in the HD map and sensor readings may lead to erroneous localization or even cause traffic accidents. It is imperative to keep LMs up-to-date. However, frequently recollecting data with dedicated hardware and specialists to update HD maps is not only cost-prohibitive but also unviable. Here we propose to utilize crowdsourced images from multiple vehicles at different times to help verify LMs for HD map maintenance. We obtain the LM distribution in the image space by considering the camera pose uncertainty in perspective projection. Both LMs in HD map and LMs in the image are treated as observations of LM distributions which allow us to construct posterior conditional distribution (a.k.a Bayesian belief functions) of LMs from either sources. An LM is consistent if belief functions from the map and the image satisfy statistical hypothesis testing. We further extend the Bayesian belief model into a sequential belief update using crowdsourced images. LMs with a higher probability of existence are kept in the HD map whereas those with a lower probability of existence are removed from the HD map. We verify our approach using real data. Experimental results show that our method is capable of verifying and updating LMs in the HD map.

Although soft robots are a good alternative to rigid, traditional robots due to their intrinsic compliance and environmental adaptability, there are several drawbacks that limit their impact, such as low force exertion capability and low resistance to deformation. For this reason, soft structures of variable stiffness have become a popular solution in the field to combine the benefits of both soft and rigid designs. In this paper, we develop laminar jamming flexure joints that facilitate the development of adaptive robot grippers with variable stiffness. Initially, we propose a mathematical model of the laminar jamming structures. Then, the model is experimentally validated through bending tests using different materials, pressures, and number of layers. Finally, the soft, laminar jamming structured are employed to develop variable stiffness flexure joints for two different adaptive robot grippers. Bending profile analysis and grasping tests have demonstrated the benefits of the proposed jamming structures and the capabilities of the designed grippers.

It is well known that semantic segmentation can be used as an effective intermediate representation for learning driving policies. However, the task of street scene semantic segmentation requires expensive annotations. Furthermore, segmentation algorithms are often trained irrespective of the actual driving task, using auxiliary image-space loss functions which are not guaranteed to maximize driving metrics such as safety or distance traveled per intervention. In this work, we seek to quantify the impact of reducing segmentation annotation costs on learned behavior cloning agents. We analyze several segmentation-based intermediate representations. We use these visual abstractions to systematically study the trade-off between annotation efficiency and driving performance, i.e., the types of classes labeled, the number of image samples used to learn the visual abstraction model, and their granularity (e.g., object masks vs. 2D bounding boxes). Our analysis uncovers several practical insights into how segmentation-based visual abstractions can be exploited in a more label efficient manner. Surprisingly, we find that state-of-the-art driving performance can be achieved with orders of magnitude reduction in annotation cost. Beyond label efficiency, we find several additional training benefits when leveraging visual abstractions, such as a significant reduction in the variance of the learned policy when compared to state-of-the-art end-to-end driving models.

As a critical sensor for high-level autonomous vehicles, LiDAR's limitations in adverse weather (e.g. rain, fog, snow, etc.) impede the deployment of self-driving cars in all weather conditions. In this paper, we model the performance of a popular 903nm ToF LiDAR under various fog conditions based on a LiDAR dataset collected in a well-controlled artificial fog chamber. Specifically, a two-stage data-driven method, called LaNoising (la for laser), is proposed for generating LiDAR measurements under fog conditions. In the first stage, the Gaussian Process Regression (GPR) model is established to predict whether a laser can successfully output a true detection range or not, given certain fog visibility values. If not, then in the second stage, the Mixture Density Network (MDN) is used to provide a probability prediction of the noisy measurement range. The performance of the proposed method has been quantitatively and qualitatively evaluated. Experimental results show that our approach can provide a promising description of 903nm ToF LiDAR performance under fog.

This paper describes the design, control and initial experimental results of the quadruped robot LLAMA. Designed to operate in a human-scale world, this 67kg-class, all-electric robot is capable of rapid motion over a variety of terrains. Thanks to a unique leg configuration and custom high-torque, low gear-ratio motors, it can move omnidirectionally at speeds over 1 m/s. A hierarchical reactive control scheme allows for robust and efficient motion even under variable payloads. This paper describes the structure of the controller and outlines simulation results that probe the performance envelope of the robot suggesting payload capacities up to one third of its body weight. Initial testing shows robust motion over loose debris and a variety of ground slopes. Videos of the robot may be seen at https://tinyurl.com/llama-robot.

We propose a framework for tightly-coupled lidar inertial odometry via smoothing and mapping, LIO-SAM, that achieves highly accurate, real-time mobile robot trajectory estimation and map-building. LIO-SAM formulates lidar-inertial odometry atop a factor graph, allowing a multitude of relative and absolute measurements, including loop closures, to be incorporated from different sources as factors into the system. The estimated motion from inertial measurement unit (IMU) pre-integration de-skews point clouds and produces an initial guess for lidar odometry optimization. The obtained lidar odometry solution is used to estimate the bias of the IMU. To ensure high performance in real-time, we marginalize old lidar scans for pose optimization, rather than matching lidar scans to a global map. Scan-matching at a local scale instead of a global scale significantly improves the real-time performance of the system, as does the selective introduction of keyframes, and an efficient sliding window approach that registers a new keyframe to a fixed-size set of prior "sub-keyframes." The proposed method is extensively evaluated on datasets gathered from three platforms over various scales and environments.

Multi-sensor fusion of multi-modal measurements from commodity inertial, visual and LiDAR sensors to provide robust and accurate 6DOF pose estimation holds great potential in robotics and beyond. In this paper, building upon our prior work (i.e., LIC-Fusion), we develop a sliding-window filter based LiDAR-Inertial-Camera odometry with online spatiotemporal calibration (i.e., LIC-Fusion 2.0), which introduces a novel sliding-window plane-feature tracking for efficiently processing 3D LiDAR point clouds. In particular, after motion compensation for LiDAR points by leveraging IMU data, low-curvature planar points are extracted and tracked across the sliding window. A novel outlier rejection criteria is proposed in the plane-feature tracking for high quality data association. Only the tracked planar points belonging to the same plane will be used for plane initialization, which makes the plane extraction efficient and robust. Moreover, we perform the observability analysis for the IMU-LiDAR subsystem under consideration and report the degenerate cases for spatiotemporal calibration using plane features. While the estimation consistency and identified degenerate motions are validated in Monte-Carlo simulations, different real-world experiments are also conducted to show that the proposed LIC-Fusion 2.0 outperforms its predecessor and other state-of-the-art methods.

Intelligent vision is appealing in computer-assisted and robotic surgeries. Vision-based analysis with deep learning usually requires large labeled datasets, but manual data labeling is expensive and time-consuming in medical problems. We investigate a novel cross-domain strategy to reduce the need for manual data labeling by proposing an image-to-image translation model live-cadaver GAN (LC-GAN) based on generative adversarial networks (GANs). We consider a situation when a labeled cadaveric surgery dataset is available while the task is instrument segmentation on an unlabeled live surgery dataset. We train LC-GAN to learn the mappings between the cadaveric and live images. For live image segmentation, we first translate the live images to fake-cadaveric images with LC-GAN and then perform segmentation on the fake-cadaveric images with models trained on the real cadaveric dataset. The proposed method fully makes use of the labeled cadaveric dataset for live image segmentation without the need to label the live dataset. LC-GAN has two generators with different architectures that leverage the deep feature representation learned from the cadaveric image based segmentation task. Moreover, we propose the structural similarity loss and segmentation consistency loss to improve the semantic consistency during translation. Our model achieves better image-to-image translation and leads to improved segmentation performance in the proposed cross-domain segmentation task.

Tracking and locating radio-tagged wildlife is a labor-intensive and time-consuming task necessary in wildlife conservation. In this article, we focus on the problem of achieving embedded autonomy for a resource-limited aerial robot for the task capable of avoiding undesirable disturbances to wildlife. We employ a lightweight sensor system capable of simultaneous (noisy) measurements of radio signal strength information from multiple tags for estimating object locations. We formulate a new lightweight task-based trajectory planning method-LAVAPilot-with a greedy evaluation strategy and a void functional formulation to achieve situational awareness to maintain a safe distance from objects of interest. Conceptually, we embed our intuition of moving closer to reduce the uncertainty of measurements into LAVAPilot instead of employing a computationally intensive information gain based planning strategy. We employ LAVAPilot and the sensor to build a lightweight aerial robot platform with fully embedded autonomy for jointly tracking and planning to track and locate multiple VHF radio collar tags used by conservation biologists. Using extensive Monte Carlo simulation-based experiments, implementations on a single board compute module, and field experiments using an aerial robot platform with multiple VHF radio collar tags, we evaluate our joint planning and tracking algorithms. Further, we compare our method with other information-based planning methods with and without situational awareness to demonstrate the effectiveness of our robot executing LAVAPilot. Our experiments demonstrate that LAVAPilot significantly reduces (by 98.5%) the computational cost of planning to enable real-time planning decisions whilst achieving similar localization accuracy of objects compared to information gain based planning methods, albeit taking a slightly longer time to complete a mission. To support research in the field, and conservation biology, we also open source the complete project. I... Show More

This work presents a deep reinforcement learning framework for interactive navigation in a crowded place. Our proposed Learning to Balance (L2B) framework enables mobile robot agents to steer safely towards their destinations by avoiding collisions with a crowd, while actively clearing a path by asking nearby pedestrians to make room, if necessary, to keep their travel efficient. We observe that the safety and efficiency requirements in crowd-aware navigation have a trade-off in the presence of social dilemmas between the agent and the crowd. On the one hand, intervening in pedestrian paths too much to achieve instant efficiency will result in collapsing a natural crowd flow and may eventually put everyone, including the self, at risk of collisions. On the other hand, keeping in silence to avoid every single collision will lead to the agent's inefficient travel. With this observation, our L2B framework augments the reward function used in learning an interactive navigation policy to penalize frequent active path clearing and passive collision avoidance, which substantially improves the balance of the safety-efficiency trade-off. We evaluate our L2B framework in a challenging crowd simulation and demonstrate its superiority, in terms of both navigation success and collision rate, over a state-of-the-art navigation approach.

Underactuated designs of robot limbs can enable these systems to passively adapt their joint configuration in response to external forces. Passive adaptation and reconfiguration can be extremely beneficial in situations where manipulation or locomotion with complex substrates is required. A common design for underactuated systems often involves a single tendon that actuates multiple rotational joints, each with a torsional elastic spring resisting bending. However, a challenge of using those joints for legged locomotion is that limbs typically need to follow a cyclical trajectory so that feet can alternately be engaged in stance and swing phases. Such trajectories present challenges for linearly elastic underactuated limbs. In this paper, we present a new method of underactuated limb design which incorporates hysteretic joints that change their torque response during loading and unloading. A double-jointed underactuated limb with both linear and hysteretic joints can thus be tuned to create a variety of looped trajectories. We fabricate these joints inside a flexible legged robot using a modified laminate based 3D printing method, and the result shows that with passive compliance and a mechanically determined joint sequence, a 2-legged minimalist robot can successfully walk through a confined channel over uneven substrates.

Category-based methods for task-specified grasp planning have recently been proposed in the literature. Such methods, however, are normally time consuming in both training and grasp determination process and lack capabilities to improve grasping skills due to the fixed training data set. This paper presents an improved approach for knowledge-based grasp planning by developing a multi-layer network using self-organizing map. A number of grasp candidates are learned in the experiments and the information that is associated with these grasp candidates is clustered based on different criteria on each network layer. A codebook which is composed of a small number of generalized models and the corresponding task-oriented grasps is generated from the network. In addition, the proposed network is capable of automatically adjusting its size so that the codebook can be continuously updated from each interaction with the novel objects. In order to increase the accuracy and convergence rate of the clustering process, a new initialization method is also proposed. Simulation results present the advantages of the proposed initialization method and the auto-growing algorithm in terms of accuracy and efficiency over some conventional methods. Experimental results demonstrate that novel objects can be successfully grasped in accordance with desired tasks using the proposed approach.

This paper proposes a kinodynamic motion plan-ning framework for multi-legged robot jumping based on the mixed-integer convex program (MICP), which simultaneously reasons about centroidal motion, contact points, wrench, and gait sequences. This method uniquely combines configuration space discretization and the construction of feasible wrench polytope (FWP) to encode kinematic constraints, actuator limit, friction cone constraint, and gait sequencing into a single MICP. The MICP could be efficiently solved to the global optimum by off-the-shelf numerical solvers and provide highly dynamic jumping motions without requiring initial guesses. Simulation and experimental results demonstrate that the proposed method could find novel and dexterous maneuvers that are directly deployable on the two-legged robot platform to traverse through challenging terrains.

The human hand serves as an inspiration for robotic grippers. However, the dimensions of the human hand evolved under a different set of constraints and requirements than that of robots today. This paper discusses a method of kinematically optimizing the design of an anthropomorphic robotic hand. We focus on maximizing the workspace intersection of the thumb and the other fingers as well as maximizing the size of the largest graspable object. We perform this optimization and use the resulting dimensions to construct a flexible, underactuated 3D printed prototype. We verify the results of the optimization through experimentation, demonstrating that the optimized hand is capable of grasping objects ranging from less than 1 mm to 12.8 cm in diameter with a high degree of reliability. The hand is lightweight and inexpensive, weighing 333 g and costing less than 175 USD, and strong enough to lift over 1.1 lb (500 g). We demonstrate that the optimized hand outperforms an open-source 3D printed anthropomorphic hand on multiple tasks. Finally, we demonstrate the performance of our hand by employing a classification-based user intent decision system which predicts the grasp type using real-time electromyographic (EMG) activity patterns.

Control and localization of deformable linear objects (DLOs) require models to handle their deformation. This paper proposes an approach to automatically generate a model from available visual sensor information. Based on point cloud data obtained from a 3D stereo camera, the kinematics of a multibody model formulation are derived. The approach aims to balance the tradeoff between computational complexity and model accuracy. This is achieved with a geometric error criterion that reduces the introduced degrees of freedom (DOF) of the model to a necessary minimum, representing the continuous shape with as few bodies as possible. The approach is evaluated analytically and validated with an experimental scenario of DLO manipulation.

This paper presents the development of a Kalman filter-based range estimation technique to precisely calculate the inter-node ranges of Ultra Wide Band (UWB) modules. Relative clock tracking filters running between every anchor pair tracks relative clock dynamics while estimating the time of flight as a filter state. Both inbound and outbound message timestamps are used to update the filter to make the time of flight observable in the chosen state space design. A faster relative clock filter convergence has been achieved with the inclusion of the clock offset ratio as a measurement additional to the timestamps. Furthermore, a modified gradient clock synchronization algorithm is used to achieve global clock synchronization throughout the network. A correction term is used in the gradient clock synchronization algorithm to enforce the global clock rate to converge at the average of individual clock rates while achieving asymptotic stability in clock rate error state. Experiments are conducted to evaluate synchronization and ranging accuracy of the proposed range estimation approach.

In this paper, we propose a dependable visual kidnap recovery (KR) framework that pinpoints a unique pose in a given 3D map when a device is turned on. For this framework, we first develop indoor-GeM (i-GeM), which is an extension of GeM [1] but considerably more robust than other global descriptors [2]-[4], including GeM itself. Then, we propose a convolutional neural network (CNN)-based system called KR-Net, which is based on a coarse-to-fine paradigm as in [5] and [6]. To our knowledge, KR-Net is the first network that can pinpoint a wake-up pose with a confidence level near 100% within a 1.0 m translational error boundary. This dependable success rate is enabled not only by i-GeM, but also by a combinatorial pooling approach that uses multiple images around the wake-up spot, whereas previous implementations [5], [6] were constrained to a single image. Experiments were conducted in two challenging datasets: a large-scale (12,557 m2) area with frequent featureless or repetitive places and a place with significant view changes due to a one-year gap between prior modeling and query acquisition. Given 59 test query sets (eight images per pose), KR-Net successfully found all wake-up poses, with average and maximum errors of 0.246 m and 0.983 m, respectively.

We present KOVIS, a novel learning-based, calibration-free visual servoing method for fine robotic manipulation tasks with eye-in-hand stereo camera system. We train the deep neural network only in the simulated environment; and the trained model could be directly used for real-world visual servoing tasks. KOVIS consists of two networks. The first keypoint network learns the keypoint representation from the image using with an autoencoder. Then the visual servoing network learns the motion based on keypoints extracted from the camera image. The two networks are trained end-to-end in the simulated environment by self-supervised learning without manual data labeling. After training with data augmentation, domain randomization, and adversarial examples, we are able to achieve zero-shot sim-to-real transfer to real-world robotic manipulation tasks. We demonstrate the effectiveness of the proposed method in both simulated environment and real-world experiment with different robotic manipulation tasks, including grasping, peg-in-hole insertion with 4mm clearance, and M13 screw insertion. The demo video is available at: http://youtube/gfBJBR2tDzA.

Synthetic data has been applied in many deep learning based computer vision tasks. Limited performance of algorithms trained solely on synthetic data has been approached with domain adaptation techniques such as the ones based on generative adversarial framework. We demonstrate how adversarial training alone can introduce semantic inconsistencies in translated images. To tackle this issue we propose density prematching strategy using KLIEP-based density ratio estimation procedure. Finally, we show that aforementioned strategy improves quality of translated images of underlying method and their usability for the semantic segmentation task in the context of autonomous driving.