IROS 2021 论文列表

Autonomous inspection of powerlines with quadrotors is challenging. Flights require persistent perception to keep a close look at the lines. We propose a method that uses event cameras to robustly track powerlines. Event cameras are inherently robust to motion blur, have low latency, and high dynamic range. Such properties are advantageous for autonomous inspection of powerlines with drones, where fast motions and challenging illumination conditions are ordinary. Our method identifies lines in the stream of events by detecting planes in the spatio-temporal signal, and tracks them through time. The implementation runs onboard and is capable of detecting multiple distinct lines in real time with rates of up to 320 thousand events per second. The performance is evaluated in real-world flights along a powerline. The tracker is able to persistently track the powerlines, with a mean lifetime of the line 10× longer than existing approaches.

Soft pneumatic legged robots show promise in their ability to traverse a range of different types of terrain, including natural unstructured terrain met in applications like precision agriculture. They can adapt their body morphology to the intricacies of the terrain at hand, thus enabling robust and resilient locomotion. In this paper we capitalize upon recent developments on soft pneumatic legged robots to introduce a closed-loop trajectory tracking control scheme for operation over flat ground. Closed-loop pneumatic actuation feedback is achieved via a compact and portable pneumatic regulation board. Experimental results reveal that our soft legged robot can precisely control its body height and orientation while in quasi-static operation based on a geometric model. The robot can track both straight line and curved trajectories as well as variable-height trajectories. This work lays the basis to enable autonomous navigation for soft legged robots.

Dynamic environments that include unstructured moving objects pose a hard problem for Simultaneous Localization and Mapping (SLAM) performance. The motion of rigid objects can be typically tracked by exploiting their texture and geometric features. However, humans moving in the scene are often one of the most important, interactive targets – they are very hard to track and reconstruct robustly due to non-rigid shapes. In this work, we present a fast, learning-based human object detector to isolate the dynamic human objects and realise a real-time dense background reconstruction framework. We go further by estimating and reconstructing the human pose and shape. The final output environment maps not only provide the dense static backgrounds but also contain the dynamic human meshes and their trajectories. Our Dynamic SLAM system runs at around 26 frames per second (fps) on GPUs, while additionally turning on accurate human pose estimation can be executed at up to 10 fps.

We propose a novel multi-pose loss function to train a neural network for 6D pose estimation, using synthetic data and evaluating it on real images. Our loss is inspired by the VSD (Visible Surface Discrepancy) metric and relies on a differentiable renderer and CAD models. This novel multi-pose approach produces multiple weighted pose estimates to avoid getting stuck in local minima. Our method resolves pose ambiguities without using predefined symmetries. It is trained only on synthetic data. We test on real-world RGB images from the T-LESS dataset, containing highly symmetric objects common in industrial settings. We show that our solution can be used to replace the codebook in a state-of-the-art approach. So far, the codebook approach has had the shortest inference time in the field. Our approach reduces inference time further while a) avoiding discretization, b) requiring a much smaller memory footprint and c) improving pose recall.3

Recent work on robot learning with visual observations has shown great success in solving many manipulation tasks. While visual observations contain rich information about the environment and the robot, they can be unreliable in the presence of visual noise or occlusions. In these cases, we can leverage tactile observations generated by the interaction between the robot and the environment. In this paper, we propose a framework for learning manipulation policies that fuse visual and tactile feedback. The control problems considered in this work are to localize a gripper with respect to the environment image and navigate to desired states. Our method uses a learned Bayes filter to estimate the state of a gripper by conditioning the tactile observations on the environment image. We use deep reinforcement learning for solving the localization and navigation problems provided with the belief of the gripper’s state and the environment image. We compare our method against two baselines where the agent uses tactile observation directly with a recurrent neural network or uses a point estimate of the state instead of the full belief state. We also transfer the policies to the real world and validate them on a physical robot.

Currently, there have been many kinds of pointbased 3D trackers, while voxel-based methods are still underexplored. In this paper, we first propose a voxel-based tracker, named PointSiamRCNN, improving tracking performance by embedding target information into the search region. Our framework is composed of two parts for achieving proposal generation and proposal refinement, which fully releases the potential of the two-stage object tracking. Specifically, it takes advantage of efficient feature learning of the voxel-based Siamese network and high-quality proposal generation of the Siamese region proposal network head. In the search region, the groundtruth annotations are utilized to realize semantic segmentation, which leads to more discriminative feature learning with pointwise supervisions. Furthermore, we propose the Self and Cross Attention Module for embedding target information into the search region. Finally, the multi-scale RoI pooling module is proposed to obtain compact representations from target-aware features for proposal refinement. Exhaustive experiments on the KITTI tracking dataset demonstrate that our framework reaches the competitive performance with the state-of-the-art 3D tracking methods and achieves the state-of-the-art in terms of BEV tracking.

Provides an abstract for each of the invited presentations and may include a brief professional biography of each presenter. The complete presentations were not made available for publication as part of the conference proceedings.

This work presents an efficient and implementable solution to the problem of joint task allocation and path planning in a multi-UAV platform. The sensing requirement associated with the task gives rise to an uncanny variant of the traditional vehicle routing problem with coverage/sensing constraints. As is the case in several multi-robot path-planning problems, our problem reduces to an mTSP problem. In order to tame the computational challenges associated with the problem, we propose a hierarchical solution that decouples the vehicle routing problem from the target allocation problem. As a tangible solution to the allocation problem, we use a clustering-based technique that incorporates temporal uncertainty in the cardinality and position of the robots. Finally, we implement the proposed techniques on our multi-quadcopter platforms.

In many real-world manipulation problems, the constraints imposed by the environment on an object are tight. In these cases, most state-of-the-art planners struggle to fit satisfactorily in low dimensional sub-manifolds, while still ensuring geometric and force feasibility. On the other hand, humans are at ease with such situations and indeed exploit constraints to manipulate objects proficiently.To face this challenge, we propose to merge state-of-art randomized grasp planning methods with model-based grasp analysis. We use the partial form-closure analysis framework to find the geometrically feasible motions of the object. Then, to ensure that the desired motions are physically realizable by the robot, we resort to an extension of the force-closure analysis framework accounting also for dynamic friction. We use these instruments to construct a random tree in a simplified planning space containing only object-robot configurations that are reachable through effectively applicable contact forces. The algorithm, validated in simulation and in preliminary experiments with a collaborative robot, features the ability to compute solutions for heavily constrained real-world manipulation problems.

Sampling-based motion planning is the predominant paradigm in many real-world robotic applications, but its performance is immensely dependent on the quality of the samples. The majority of traditional planners are inefficient as they use uninformative sampling distributions instead of exploiting structures and patterns in the problem to guide better sampling strategies. Moreover, most current learning-based planners are susceptible to posterior collapse or mode collapse due to the sparsity and highly varying nature of C-Space and motion plan configurations. This work introduces a conditional normalising flow-based distribution learned through previous experiences, which improves existing methods’ sampling scheme. Our distribution can be conditioned on the current problem instance to provide informative prior to sample configurations within promising regions. When we train our sampler with an expert planner, the resulting distribution is often near-optimal, and the planner can find a solution faster, with less invalid samples and less initial cost. The normalising flow-based distribution uses simple invertible transformations that are very computationally efficient, and our optimisation formulation explicitly avoids mode collapse in contrast to other existing learning-based sampler. Finally, we provide a formulation and theoretical foundation to sample from the distribution efficiently. Experimentally we demonstrate utilising the flow-based distribution in a sampling-based motion planner allows a solution to be found faster, with fewer samples and better overall runtime performance.

The detection of planar surfaces in a point cloud is a popular technique for the extraction of drivable or walkable surfaces and for tabletop segmentation. Unfortunately, RGB-D sensors are quite noisy and provide incomplete data, which makes the extraction of surfaces more challenging. Also, it is desirable to process the point cloud data in real time, which at a rate of approximately 30 Hz, leaves only a small amount of computation time per frame. We have already developed a real time-capable plane segmentation method [1] that exploits the organized structure of RGB-D point clouds in order to implement a computationally efficient region growing algorithm. It uses the point-plane distance to assign points to their segments rather than inherently unreliable surface normals. Now we are presenting an improvement where we adapt thresholds and other parameters of our algorithm to the measured depth in order to account for an increasing scatter of the points at larger distances from the camera. We estimate a minimum detectable plane size in pixels dependent on the measured depth. This enables us to stride in pixel coordinates with larger steps that are adaptive to the measured depth and to implement more robust sanity checks of depth-dependent size. Apart from a speed-up of the runtime of our algorithm, the segmentation quality also increased. We show a comparison between our improvement, our previous version, and other state-of-the-art methods evaluated on multiple commonly available datasets.

Autonomous robots need to plan the tasks they carry out to fulfill their missions. The missions’ increasing complexity does not let human designers anticipate all the possible situations, so traditional control systems based on state machines are not enough. This paper contains a description of the ROS2 Planning System (PlanSys2 in short), a framework for symbolic planning that incorporates novel approaches for execution on robots working in demanding environments. PlanSys2 aims to be the reference task planning framework in ROS2, the latest version of the de facto standard in robotics software development. Among its main features, it can be highlighted the optimized execution, based on Behavior Trees, of plans through a new actions auction protocol and its multi-robot planning capabilities. It already has a small but growing community of users and developers, and this document is a summary of the design and capabilities of this project.

Metastatic involvement of lymph nodes is one of the most important prognostic variables for many cancers. Several deep learning based algorithms have been developed to segment metastatic regions in pathological images to help predict prognosis. However, the training of these methods requires a large amount of annotated data, and the labeling task is an extremely time-consuming process for human annotators. In order to reduce the annotation burden, we for the first time propose a semi-automatic annotation method (PiPo-Net) for the labeling of pathological images. The method is comprised of two subnetworks, a pixel-wise segmentation network (Pi-Net) and a polygon-based annotation network (Po-Net). The Pi-Net adopts an improved encoder-decoder architecture and can effectively aggregate multi-scale image features. The Po-Net is built on the Pi-Net and leverages a two-layer recurrent neural network to generate tight-bounded polygons for the metastatic regions. Corresponding to the proposed network architecture, a loss function called PiPo-loss is introduced to help optimize the whole network. The main advantage of our method is that it integrates human annotators into the prediction loop, allowing to iteratively refine the predictions according to the suggestions from human annotators. We evaluate our method on Camelyon16 database and achieve a Dice score of 91% in the initial annotation attempt. We also demonstrate the effectiveness of the human-network collaborative annotation, which achieves promising labeling results, verifying the advantages of our proposed method.

Most controllers for lower-limb robotic prostheses require individually tuned parameter sets for every combination of speed and incline that the device is designed for. Because ambulation occurs over a continuum of speeds and inclines, this design paradigm requires tuning of a potentially prohibitively large number of parameters. This limitation motivates an alternative control framework that enables walking over a range of speeds and inclines while requiring only a limited number of tunable parameters. In this work, we present the implementation of a continuously varying kinematic controller on a custom powered knee-ankle prosthesis. The controller uses a phase variable derived from the residual thigh angle, along with real-time estimates of ground inclination and walking speed, to compute the appropriate knee and ankle joint angles from a continuous model of able-bodied kinematic data. We modify an existing phase variable architecture to allow for changes in speeds and inclines, quantify the closed-loop accuracy of the speed and incline estimation algorithms for various references, and experimentally validate the controller by observing that it replicates kinematic trends seen in able-bodied gait as speed and incline vary.

Full body scanning plays an important role in automated industrial manufacture and inspection. It requires the fusion of multi-view point cloud data and consumes large computational resources when the geometry of corresponding point clouds are unknown. Structured Light Illumination (SLI) is one of the most promising indoor 3D imaging techniques, but also has the same weakness for fusing the multi-view point clouds. This work proposes a mobile SLI system to alleviate this problem for full body 3D scanning. We derive the geometric relation between the phase and the motion of the mobile system, and develop an optimization approach to estimate the pose by comparing the phase image pair before and after the motion. Further more, a graph-based Simultaneous Localization And Mapping (SLAM) framework is built to improve the global accuracy of the pose estimation. By using the 2D phase comparison, the proposed method is more accurate than the normal 2D image feature point matching, and has lower computational complexity than the 3D point registration. The proposed system is experimented in both 3D Simulator and real environment. The yielded full body scan results demonstrated its higher accuracy and more efficiency than the current methods.

Social robots are used in daily life. One of the applications of social robots is as recommendation systems. Previous research has mainly investigated how persuasive recommendations can be improved by focusing on the non-verbal/verbal behavior of robots. However, to use robots as recommendation systems every day, it is extremely important to examine the persistence of repeated persuasion over a long term, rather than the effect of one-time persuasion. Therefore, the objective of this study was to investigate the persistence of repeated persuasive of robots. For this purpose, robots with three types of behavior (Expert Behavior, Local Behavior, and Growth Behavior) recommended nutrition bars in a situation of daily consumption behavior for two weeks. We could confirm significant differences in the persistent persuasiveness in each behavior. The results suggested that the combination of value co-creation using local information and meta-trust expression had a significant impact on the persistence of the repeated persuasiveness of the robots in the longitudinal period. However, the acceptance of the recommendation robot system decreased due to the increase in the amount of information during recommendation; therefore, a new recommend system to solve this problem is desired.

In this paper we develop a gestural communication perception system for a social robot companion that is able to autonomously learn novel gestures on-the-fly. The system constantly tracks human gestural activities with a camera and evaluates the performed gestures under an open-set assumption. This allows for the identification of unknown gestures. Once detected, the system stores motion sequences of the novel gesture class and employs a dialogue interaction with the human to automatically label the unknown gesture. Subsequently, the gestural model is updated, grounding the unknown gesture through dialog interaction. In our experiment, we evaluate a neural network with varying threshold values for the open gesture recognition with unknown detection. Results show that the general classifier reaches an accuracy of more than 83%, and an f1-score of 0.79 in an open-ended scenario. The method is furthermore tested in a first in-lab interaction setting, which shows the system usability and its potential for future personalized human-robot gestural communication.

This paper studies autonomous stair climbing for quadrupedal robots with perception. Enabling quadrupeds to reliably climb staircases greatly expands their applicability in practical scenarios. For this structured task, we develop a simple yet effective perception and control framework for autonomous quadrupedal stair climbing. By exploiting the structural knowledge about the staircases, the proposed framework first extracts the geometric information about the staircase from measurements of the perception system. Then, the climbing velocity and associated foothold references during stair climbing are generated via simple optimization algorithms based on the geometric information about the staircase. Given these references, we use model predictive control based approach to generate input joint torques for controlling the quadruped to complete the whole stair climbing task. Simulation validations using the full dynamic model of the Unitree’s Aliengo quadruped with the MuJoCo simulator are performed, which demonstrate successful autonomous climbing of various staircases with different geometries. Effectiveness of the proposed strategy is further validated through hardware experiments on the real Aliengo robot with different real-world staircases.

In this paper, we develop a novel path-constrained and collision-free optimal trajectory planning algorithm for robot manipulators in the presence of obstacles for the following problem: Given a desired sequence of discrete waypoints of robot configurations, a set of robot kinematic and dynamic constraints, and a set of obstacles, determine a time and jerk optimal and collision-free trajectory for the robot passing through the given waypoints with constant speed. Our approach in developing the robot path through the waypoints relies on the orthogonal collocation method where the states are represented with Legendre polynomials in the Barycentric form; the transcription process efficiently converts the continuous-time formulation of the optimal control problem (both time and jerk optimal) into a discrete non-linear program. In addition, we provide an efficient method for avoiding robot self-collisions (of joints and links) and collisions with workspace obstacles by modeling them as the union of spheres and cylinders in the workspace. The resulting collision free optimal trajectory provides smooth and constrained motion for the robot passing through all the waypoints in the given prescribed sequence. The proposed method is validated using numerical simulations and experiments on a six degree-of-freedom robot.