IROS 2021 论文列表

Recent studies on deep-learning-based small defection segmentation approaches are trained in specific settings and tend to be limited by fixed context. Throughout the training, the network inevitably learns the representation of the background of the training data before figuring out the defection. They underperform in the inference stage once the context changed and can only be solved by training in every new settings. This eventually leads to the limitation in practical robotic applications where contexts keep varying. To cope with this, instead of training a network context by context and hoping it to generalize, why not stop misleading it with any limited context and start training it with pure simulation? In this paper, we propose the network SSDS that learns a way of distinguishing small defections between two images regardless of the context, so that the network can be trained once for all. A small defection detection layer utilizing the pose sensitivity of phase correlation between images is introduced and is followed by an outlier masking layer. The network is trained on randomly generated simulated data with simple shapes and is generalized across the real world. Finally, SSDS is validated on real-world collected data and demonstrates the ability that even when trained in cheap simulation, SSDS can still find small defections in the real world showing the effectiveness and its potential for practical applications. Code is available here

Model-free reinforcement learning has become a viable approach for vision-based robot control. However, sample complexity and adaptability to domain shifts remain persistent challenges when operating in high-dimensional observation spaces (images, LiDAR), such as those that are involved in autonomous driving. In this paper, we propose a flexible framework by which a policy’s observations are augmented with robust attention representations in the latent space to guide the agent’s attention during training. Our method encodes local and global descriptors of the augmented state representations into a compact latent vector, and scene dynamics are approximated by a recurrent network that processes the latent vectors in sequence. We outline two approaches for constructing attention maps; a supervised pipeline leveraging semantic segmentation networks, and an unsupervised pipeline relying only on classical image processing techniques. We conduct our experiments in simulation and test the learned policy against varying seasonal effects and weather conditions. Our design decisions are supported in a series of ablation studies. The results demonstrate that our state augmentation method both improves learning efficiency and encourages robust domain adaptation when compared to common end-to-end frameworks and methods that learn directly from intermediate representations.

A mobile robot that follows behind humans in structured environments has to face the challenge of full occlusion caused by the walls when the target person makes a turn at the corridor intersections. This may result in short-term, even a permanent loss of the target from the field of view of the Human-Following Robots (HFRs). Concerning this issue, a novel side-by-side following method for HFRs is addressed. In this paper, HFRs detect the legs of target person and different types of corridor intersections using the onboard laser scanner at first. Then, we provide a corridor detector method to cluster the geometric structure constraint between the target and corridor intersections. At last, a Side-by-side Following Leg Tracker (SFLT) is designed by integrating the laser information, in order to increase the visible time of the target person, while the target is turning at the corridor intersections. The corridor detector method and SFLT method have been simulated in MATLAB. Moreover, the approach of side-by-side following has been implemented in the Robot Operating System (ROS) of real-life robots in the corridor environment. The results from simulation and practical experiment show that, by using our method, HFRs were able to successfully follow the human92.0% while a mobile robot meeting potential occlusions at corridor intersections.

In this paper, we present a capacitance-based sensor array for physical human-robot interaction (pHRI) applications that can measure the proximity, near-zero-force (NZF) contacts, and pressure between a robot and human body. The top segment including the electrodes is made of soft, stretchable materials, while the bottom segment consists of electrodes patterned from a thin copper film. The resulting device is soft and conformable to smooth curved surfaces of robot links while ensuring high signal integrity. It can be fabricated in different sizes from fingertips to torso because the fabrication process employs conventional, scalable methods. Using this sensor, we investigate the problem of recognizing gentle contact gestures often seen in affectionate physical interactions. The output of this multi-modal sensor is a 2D array compatible with machine learning algorithms used for pressure and image-based recognition problems. We utilize the spatio-temporal information of the 2D capacitance data by applying two existing deep neural network architectures. The highest accuracy achieved is over 99% in 7-class recognition of contact gestures involving proximity, NZF contacts, and medium pressure.

Forecasting the future behaviors of dynamic actors is an important task in many robotics applications such as self-driving. It is extremely challenging as actors have latent intentions and their trajectories are governed by complex interactions between the other actors, themselves, and the map. In this paper, we propose LaneRCNN, a graph-centric motion forecasting model that captures the actor-to-actor and the actor-to-map relations in a distributed and structured manner. Relying on a specially designed graph encoder, we learn a local graph representation per actor (LaneRoI) to encode its past motions and the local map topology. We further develop an interaction module which permits efficient message passing among local graph representations within a shared global lane graph. Moreover, we parameterize the output trajectories based on lane graphs, a more amenable prediction parameterization. We demonstrate the effectiveness of our approach on the challenging Argoverse [1] motion forecasting benchmark and achieve state-of-the-art performance.

In this paper, we revisit the long-standing Traveling Salesman Problem (TSP) and focus on the challenging, yet practical route planning problem with limited computational resources. We make contributions to TSP, one of the most famous NP-hard problems by providing a new improved approximate solution, which we term TOpology Preserving Self-Organizing Map (TOPSOM). TOPSOM well preserves the topology of the node map to be traversed by maintaining the continuity of nodes and the distances between them. In addition, to satisfy the requirements of convex hull, we design an elastic competitive Hebbian learning rule. TOPSOM can solve large-scale TSPs with high precision and high efficiency with limited computational costs. Extensive experimental results on mainstream route planning benchmarks including TSPLIB and National TSP’s show that our method consistently outperforms baseline methods, by up to 7.7% in terms of the Percent Deviation of Mean solution to best known solution.

Enabling mobile robots for solving challenging and diverse shape, texture, and motion related tasks with high fidelity vision requires the integration of novel multimodal imaging sensors and advanced fusion techniques. However, it is associated with high cost, power, hardware modification, and computing requirements which limit its scalability. In this paper, we propose a novel Simultaneously Learned Auto Encoder Domain Adaptation (SAEDA)-based transfer learning technique to empower noisy sensing with advanced sensor suite capabilities. In this regard, SAEDA trains both source and target auto-encoders together on a single graph to obtain the domain invariant feature space between the source and target domains on simultaneously collected data. Then, it uses the domain invariant feature space to transfer knowledge between different signal modalities. The evaluation has been done on two collected datasets (LiDAR and Radar) and one existing dataset (LiDAR, Radar and Video) which provides a significant improvement in quadruped robot-based classification (home floor and human activity recognition) and regression (surface roughness estimation) problems. We also integrate our sensor suite and SAEDA framework on two real-time systems (vacuum cleaning and Mini-Cheetah quadruped robots) for studying the feasibility and usability.

This paper presents a method for biped gait generation along a predefined curve with fully stretched knees. First, we design a spatial gait pattern as a function of the traveled distance on the path without considering dynamics. Then, a consistent dynamic walking motion is obtained by optimization that minimizes the zero-moment point and the speed errors while considering the trade-off between kinematics and dynamics. This method generalizes the spatially quantized dynamics-based gait generation, which is our former method restricted to straight paths, to walk on arbitrary curves. The generated gaits are validated by dynamic simulation.

Moths use hundreds of strain sensors (campaniform sensilla) on each wing to quickly respond to perturbations that may otherwise destabilize the moth during flight. A similar sensing approach could help stabilize micro-aerial vehicles (MAVs), but large sensor arrays are challenging due to the wiring and large latency that exists when capturing data from many traditional strain sensors. This work introduces a simplified bio-inspired strain sensor; the sensor interface and kinematics were inspired by campaniform sensilla that output a spike only in response to signals of interest. The engineered sensor outputs a discrete analog signal representing strain thresholds. A kinematic model of the sensor design is developed and describes the measured strain in terms of the sensor’s geometric parameters. This model is used to understand trade-offs between sensor resolution and range, and is validated using a finite element model (FEM) of the sensor. The sensor was designed with ease of fabrication in mind, using simple techniques and commercially available components. Fabricated sensors were tested in a four-point flexural test, and the data from the analytical and FEM model show good agreement with the experimental results. The sensors demonstrate resolutions of 83, 158, and 281 microstrain for the different designs tested. A sensor is placed on a model wing to illustrate future applications to MAVs as well as the sensor’s ability to sense both compressive and tensile strains.

Robot modelling is an essential part to properly understand how a robotic system moves and how to control it. The kinematic model of a robot is usually obtained by using Denavit-Hartenberg convention, which relies on a set of parameters to describe the end-effector pose in a Cartesian space. These parameters are assigned based on geometrical considerations of the robotic structure, however, the assigned values may be inaccurate. The purpose of robot kinematic calibration is therefore to find optimal parameters which improve the accuracy of the robot model. In this work we present Kalibrot, an open source Matlab package for robot kinematic calibration. Kalibrot has been designed to simplify robot calibration and easily assess the calibration results. Beside computing the optimal parameters, Kalibrot provides a visualization layer showing the values of the calibrated parameters, what parameters can be identified, and the calibrated robotic structure. The capabilities of the package are here shown through simulated and real world experiments.

We present KDFNet, a novel method for 6D object pose estimation from RGB images. To handle occlusion, many recent works have proposed to localize 2D keypoints through pixel-wise voting and solve a Perspective-n-Point (PnP) problem for pose estimation, which achieves leading performance. However, such voting process is direction-based and cannot handle long and thin objects where the direction intersections cannot be robustly found. To address this problem, we propose a novel continuous representation called Keypoint Distance Field (KDF) for projected 2D keypoint locations. Formulated as a 2D array, each element of the KDF stores the 2D Euclidean distance between the corresponding image pixel and a specified projected 2D keypoint. We use a fully convolutional neural network to regress the KDF for each keypoint. Using this KDF encoding of projected object keypoint locations, we propose to use a distance-based voting scheme to localize the keypoints by calculating circle intersections in a RANSAC fashion. We validate the design choices of our framework by extensive ablation experiments. Our proposed method achieves state-of-the-art performance on Occlusion LINEMOD dataset with an average ADD(-S) accuracy of 50.3% and TOD dataset mug subset with an average ADD accuracy of 75.72%. Extensive experiments and visualizations demonstrate that the proposed method is able to robustly estimate the 6D pose in challenging scenarios including occlusion.

In this paper, we present a novel learnable and continuous Monte-Carlo Tree Search method, named as KB-Tree, for motion planning in autonomous driving. The proposed method utilizes an asymptotical PUCB based on Kernel Regression (KR-AUCB) as a novel UCB variant, to improve the exploitation and exploration performance. In addition, we further optimize the sampling in continuous space by adapting Bayesian Optimization (BO) in the selection process of MCTS. Moreover, we use a customized Graph Neural Network (GNN) as our feature extractor to improve the learning performance. To the best of our knowledge, we are the first to apply the continuous MCTS method in autonomous driving. To validate our method, we conduct extensive experiments under several weakly and strongly interactive scenarios. The results show that our proposed method performs well in all tasks, and outperforms the learning-based continuous MCTS method and the state-of-the-art Reinforcement Learning (RL) baseline.

The dominant way to control a robot manipulator uses hand-crafted differential equations leveraging some form of inverse kinematics / dynamics. We propose a simple, versatile joint-level controller that dispenses with differential equations entirely. A deep neural network, trained via model-free reinforcement learning, is used to map from task space to joint space. Experiments show the method capable of achieving similar error to traditional methods, while greatly simplifying the process by automatically handling redundancy, joint limits, and acceleration / deceleration profiles. The basic technique is extended to avoid obstacles by augmenting the input to the network with information about the nearest obstacles. Results are shown both in simulation and on a real robot via sim-to-real transfer of the learned policy. We show that it is possible to achieve sub-centimeter accuracy, both in simulation and the real world, with a moderate amount of training.

Among the most prevalent motion planning techniques, sampling and trajectory optimization have emerged successful due to their ability to handle tight constraints and high-dimensional systems, respectively. However, limitations in sampling in higher dimensions and local minima issues in optimization have hindered their ability to excel beyond static scenes in offline settings. Here we consider highly dynamic environments with long horizons that necessitate a fast on-line solution. We present a unified approach that leverages the complementary strengths of sampling and optimization, and interleaves them both in a manner that is well suited to this challenging problem. With benchmarks in multiple synthetic and realistic simulated environments, we show that our approach performs significantly better on various metrics against baselines that employ either only sampling or only optimization. Project page: https://sites.google.com/view/jistplanner

Multi-object tracking (MOT) with camera-LiDAR fusion demands accurate results of object detection, affinity computation and data association in real time. This paper presents an efficient multi-modal MOT framework with online joint detection and tracking schemes and robust data association for autonomous driving applications. The novelty of this work includes: (1) development of an end-to-end deep neural network for joint object detection and correlation using 2D and 3D measurements; (2) development of a robust affinity computation module to compute occlusion-aware appearance and motion affinities in 3D space; (3) development of a comprehensive data association module for joint optimization among detection confidences, affinities and start-end probabilities. The experiment results on the KITTI tracking benchmark demonstrate the superior performance of the proposed method in terms of both tracking accuracy and processing speed.

Although autonomous driving technology has made tremendous progress in recent years, it is still challenging to predict the intentions and trajectories of pedestrians. The state-of-the-art methods suffer from two problems. (1) Existing works consider these two tasks separately, ignoring the connection between them. (2) The selection and integration of inputs for these tasks are not well designed. In this paper, these two tasks are taken into consideration in a unified model. In this way, the information provided by the labels of each other is shared, improving the performance of both tasks. Besides, in addition to the bounding boxes and speeds, orientation and road semantic segmentation features are taken into consideration to show the potential intention and road context of the pedestrian. And all the inputs are weighted by an attention module before integration. Meanwhile, a Transformer encoder is applied in our method to extract the temporal information from the fused feature sequence. Our method outperforms all previous models for both trajectory prediction and intention prediction tasks on the JAAD dataset and PIE dataset.

We present a novel approach to joint depth and normal estimation for time-of-flight (ToF) sensors. Our model learns to predict the high-quality depth and normal maps jointly from ToF raw sensor data. To achieve this, we meticulously constructed the first large-scale dataset (named ToF-100) with paired raw ToF data and ground-truth high-resolution depth maps provided by an industrial depth camera. In addition, we also design a simple but effective framework for joint depth and normal estimation, applying a robust Chamfer loss via jittering to improve the performance of our model. Our experiments demonstrate that our proposed method can efficiently reconstruct high-resolution depth and normal maps and significantly outperforms state-of-the-art approaches.

UAVs are deployed in various applications including disaster search-and-rescue, precision agriculture, law enforcement and first response. As UAV software systems grow more complex, the drawbacks of developing them in low-level languages become more pronounced. For example, the lack of memory safety in C implies poor isolation between the UAV autopilot and other concurrent tasks. As a result, the most crucial aspect of UAV reliability-timely control of the flight-could be adversely impacted by other tasks such as perception or planning. We introduce JCopter, an autopilot framework for UAVs developed in a managed language, i.e., a high-level language with built-in safe memory and timing management. Through detailed simulation as well as flight testing, we demonstrate how JCopter retains the timeliness of C-based autopilots while also providing the reliability of managed languages.

We study the iterative refinement of path planning for multiple robots, known as multi-agent pathfinding (MAPF). Given a graph, agents, their initial locations, and destinations, a solution of MAPF is a set of paths without collisions. Iterative refinement for MAPF is desirable for three reasons: 1) optimization is intractable, 2) sub-optimal solutions can be obtained instantly, and 3) it is anytime planning, desired in online scenarios where time for deliberation is limited. Despite the high demand, this is under-explored in MAPF because finding good neighborhoods has been unclear so far. Our proposal uses a sub-optimal MAPF solver to obtain an initial solution quickly, then iterates the two procedures: 1) select a subset of agents, 2) use an optimal MAPF solver to refine paths of selected agents while keeping other paths unchanged. Since the optimal solvers are used on small instances of the problem, this scheme yields efficient-enough solutions rapidly while providing high scalability. We also present reasonable candidates on how to select a subset of agents. Evaluations in various scenarios show that the proposal is promising; the convergence is fast, scalable, and with reasonable quality.