ICRA 2021 论文列表

Event cameras are promising devices for low latency tracking and high-dynamic range imaging. In this paper, we propose a novel approach for 6 degree-of-freedom (6-DoF) object motion tracking that combines measurements of event and frame-based cameras. We formulate tracking from high rate events with a probabilistic generative model of the event measurement process of the object. On a second layer, we refine the object trajectory in slower rate image frames through direct image alignment. We evaluate the accuracy of our approach in several object tracking scenarios with synthetic data, and also perform experiments with real data.

Robotic hopping requires high performance and precision, due to its extreme interactions with the environment. Designing a system that will perform optimally, or even stably, for this motion primitive is a significant challenge. In previous work, it was shown that designing a robot with two springs (one in series and one in parallel with the actuator) could dramatically improve performance. However, selecting these springs was an intricate process since their dynamics were tightly coupled, and accomplished through trial and error. This work presents a general optimization framework for interconnected systems that designs the time-based hopping motion, while also designing the shape of nonlinear springs on the robot to yield efficient hopping. Utilizing this method, hopping motions and spring designs were generated simultaneously and experimentally verified on a novel hopping robot.

Recent research in AI ethics has put forth explainability as an essential principle for AI algorithms. However, it is still unclear how this is to be implemented in practice for specific classes of algorithms—such as motion planners. In this paper we unpack the concept of explanation in the context of motion planning, introducing a new taxonomy of kinds and purposes of explanations in this context. We focus not only on explanations of failure (previously addressed in motion planning literature) but also on contrastive explanations—which explain why a trajectory A was returned by a planner, instead of a different trajectory B expected by the user. We develop two explainable motion planners, one based on optimization, the other on sampling, which are capable of answering failure and constrastive questions. We use simulation experiments and a user study to motivate a technical and social research agenda.

In this work, we report on how a sense of touch can be used to control an underactuated anthropomorphic robot hand, based on an integration that respects the hand’s mechanical functionality. Our focus is on integrating the sensorimotor control of the Pisa/IIT SoftHand, an anthropomorphic soft robot hand designed around the principle of adaptive synergies, with the BRL tactile fingertip (TacTip), a soft biomimetic optical tactile sensor. We consider: (i) closed-loop tactile control to establish a light contact on an unknown held object, based on the structural similarity of the tactile image; and (ii) controlling the estimated pose of a held object, using a convolutional neural network approach developed for other TacTip sensors. Accurate control was found for a range of hard and soft objects (to sub-millimetre accuracy and a few degrees). Overall, this gives a foundation to endow soft robotic hands with human-like touch, with implications for autonomous grasping, manipulation, human-robot interaction and prosthetics.

It is widely accepted that in the future, robots will cooperate with humans in everyday tasks. Robots interacting with humans will require social awareness when performing their tasks which will require navigation. While navigating, robots should aim to avoid distressing people in order to maximize their chance of social acceptance. For instance, avoiding getting too close to people or disrupting interactions. Most research approaches these problems by planning socially accepted paths, however, in everyday situations, there are many examples where a simple path planner cannot solve all of the predicted robots’ navigation problems. For instance, requesting permission to interrupt a conversation if an alternative path cannot be determined requires deliberative skills. This article presents the Social Navigation framework for Autonomous robots in Populated Environments (SNAPE), where different software agents are integrated within a robotics cognitive architecture. SNAPE addresses action planning aimed at social-awareness navigation in realistic situations: it plans socially accepted paths and conversations to negotiate its trajectory to reach targets. In this article, the framework is evaluated in different use-cases where the robot, during its navigation, has to interact with different people in order to reach its goal. The results show that participants report that the robot’s behavior was realistic and human-like.

It is challenging to develop an online path planning algorithm for Ackermann-steering vehicles to find collision-free and kinematically-feasible paths, that is efficient for dense environments, adaptable to various environments, and suitable for environments with narrow passages. In this paper, we propose a kinematically constrained RRT-based path planning algorithm integrating with a trajectory parameter space (TP-space) with three novel improvements to meet the above requirements. In specific, we introduce a new way to choose candidate nodes to expand the tree for an RRT-based algorithm, which can significantly increase the success rate of the expansion and improve the efficiency of the algorithm. We also introduce a procedure to incrementally adjust the step size for the expansion, which enables the algorithm to automatically adapt to various environments. At last, we integrate rapidly-exploring random vines (RRV) with a TP-space to handle kinematic constraints and improve the performance of the algorithm to expand the tree through a narrow passage. We also prove that the algorithm is probabilistic complete and asymptotically near-optimal. An ablation study shows that all three improvements can notably improve the performance of the RRT-based path planning algorithm. We also evaluate the algorithm in various environments. The experimental results show that our algorithm achieves competitive performance compared with the state-of-the-art. The source code is available at https://github.com/PengJieb/fastbkrrt.

Most existing surgical robots employ straight rigid instruments and are not compatible with imaging modalities, especially magnetic resonance imaging (MRI) that presents restrictive constraints on the robot and actuator materials. Employing continuum distal end effector and fulfilling multi-imager compatibility will potentially lead to wide adoption of surgical robots in intraoperative image-guided minimally invasive surgery (MIS). This paper introduces a 3-dimensional (3D) printed polyamide continuum robot with 2-degree of freedom, driven by modular shape memory alloy (SMA) spring actuators that enable real-time distal manipulation in 3D workspace. The multi-imager compatibility is conditionally satisfied by utilizing no ferromagnetic materials and MRI-conditional actuators in the robotic system. The use of modular SMA allows repeatable configuration setting, easy integration with active cooling strategies, and facilitates the creation of a sterile barrier between the end effector module and the actuation module. Detailed design of the robot, kinematics modeling and actuator modeling are discussed in the paper. We experimentally verified the robot kinematics and evaluated the dynamic performance of the continuum robot. It features an operating bandwidth of 0.12 Hz at -3 dB and a root-mean-square error of 0.98 mm under model predictive control when tracking sinusoidal signals, both with ±10 mm amplitude (distal bending angle of ±80◦). As a demonstration, the robot was used as a flexible endoscope manipulator and steered under inputs from a joystick to show its real-time performance.

It is becoming increasingly feasible for robots to share a workspace with humans. However, for them to do so safely while maintaining agile performance, they need the ability to smoothly handle the dynamics and uncertainty caused by human motions. Markov Decision Processes (MDPs) serve as a common framework to formulate robot planning problems. However, because of its single-agent formulation, such planner cannot account for human reaction when evaluating robot actions. The robot can thus suffer from unsafe motions and move in ways that are hard for nearby humans to understand. To resolve this, we instead model robot planning in human workspaces as a Stochastic Game, and contribute a robust planning algorithm, which enables the robot to account for its prediction errors in human responses to prevent collision, while not losing agility, opposed to traditional maximin optimization techniques, by applying maximin operation only at "critical states". We validate the approach under partial knowledge of pedestrian behaviors, and show that our approach encounters zero collision despite imperfect prediction, while improving path efficiency, compared to baselines.

Distributed manipulators - consisting of a set of actuators or robots working cooperatively to achieve a manipulation task - are robust and flexible tools for performing a range of planar manipulation skills. One novel example is the delta array, a distributed manipulator composed of a grid of delta robots, capable of performing dexterous manipulation tasks using strategies incorporating both dynamic and static contact. Hand-designing effective distributed control policies for such a manipulator can be complex and time consuming, given the high-dimensional action space and unfamiliar system dynamics. In this paper, we examine the principles guiding development and control of such a delta array for a planar translation task. We explore policy learning as a robust cooperative control approach, allowing for smooth manipulation of a range of objects, showing improved accuracy and efficiency over baseline human-designed policies.

Requiring multiple demonstrations of a task plan presents a burden to end-users of robots. However, robustly executing tasks plans from a single end-user demonstration is an ongoing challenge in robotics. We address the problem of one-shot task execution, in which a robot must generalize a single demonstration or prototypical example of a task plan to a new execution environment. Our approach integrates task plans with domain knowledge to infer task plan constituents for new execution environments. Our experimental evaluations show that our knowledge representation makes more relevant generalizations that result in significantly higher success rates over tested baselines. We validated the approach on a physical platform, which resulted in the successful generalization of initial task plans to 38 of 50 execution environments with errors resulting from autonomous robot operation included.

Global navigation satellite systems (GNSS) are one of the utterly popular sources for providing globally referenced positioning for autonomous systems. However, the performance of the GNSS positioning is significantly challenged in urban canyons, due to the signal reflection and blockage from buildings. Given the fact that the GNSS measurements are highly environmentally dependent and time-correlated, the conventional filtering-based method for GNSS positioning cannot simultaneously explore the time-correlation among historical measurements. As a result, the filtering-based estimator is sensitive to unexpected outlier measurements. In this paper, we present a factor graph-based formulation for GNSS positioning and real-time kinematic (RTK). The formulated factor graph framework effectively explores the time-correlation of pseudorange, carrier-phase, and doppler measurements, and leads to the non-minimal state estimation of the GNSS receiver. The feasibility of the proposed method is evaluated using datasets collected in challenging urban canyons of Hong Kong and significantly improved positioning accuracy is obtained, compared with the filtering-based estimator.

Most of the existing visual SLAM methods heavily rely on a static world assumption and easily fail in dynamic environments. Some recent works eliminate the influence of dynamic objects by introducing deep learning-based semantic information to SLAM systems. However such methods suffer from high computational cost and cannot handle unknown objects. In this paper, we propose a real-time semantic RGBD SLAM system for dynamic environments that is capable of detecting both known and unknown moving objects. To reduce the computational cost, we only perform semantic segmentation on keyframes to remove known dynamic objects, and maintain a static map for robust camera tracking. Furthermore, we propose an efficient geometry module to detect unknown moving objects by clustering the depth image into a few regions and identifying the dynamic regions via their reprojection errors. The proposed method is evaluated on public datasets and realworld conditions. To the best of our knowledge, it is one of the first semantic RGB-D SLAM systems that run in real-time on a low-power embedded platform and provide high localization accuracy in dynamic environments.

This research provides an overview of the challenges towards realizing interactive (near) real-time control of industrial robots within the context of creative industries. Within this field, there an increasing interest in the use of industrial robots for non-standard fabrication, building upon accessible, flow-based visual programming environments.Where robotics middleware is focused on communication, these software tools focus primarily on computational geometry within design software, with the goal of enabling mass-customization and individualization by linking parametric design with toolpath generation and robot simulation.Realizing interactive processes within such environments is challenging, as they form directed dataflows minimizing opportunities for recursive applications and are limited to non-real-time capable operating systems. A series of realized projects is presented that benefits from the advanced geometrical methods provided by the creative industries’ flow-based programming, while at the same time implementing strategies that enable cyclical dataflows and interaction. Novel software interfaces have allowed for fluent interaction with existing robot technologies, even from non-real-time capable environments.

In modern agriculture, measuring phenotypic traits helps breeders monitor plant growth, increase yield, and provide food, feed, and fiber. Traditional phenotyping requires intensive manual work, partially being intrusive. In this paper, we investigate the challenge of measuring phenotypic traits in an automated fashion through mobile robots operating in field environments. In particular, we want to measure plants from images acquired by mobile robots instead of using data from a static scanning environment. We propose to use a differentiable rendering approach to deform a generic 3D template of a plant to fit the observation recorded by a robot while ensuring a coherent deformation of the plant template. The experiments presented in this paper suggest that our approach allows for 3D reconstruction of different plant species at different growth stages using single images. From that model, we can compute important phenotypic traits, such as the leaf area index.

Active vision is a desirable perceptual feature for robots. Existing approaches usually make strong assumptions about the task and environment, thus are less robust and efficient. This study proposes an adaptive view planning approach to boost the efficiency and robustness of active object detection. We formulate the multi-object detection task as an active multiview object detection problem given the initial location of the objects. Next, we propose a novel adaptive action prediction (A2P) method built on a deep Q-learning network with a dueling architecture. The A2P method is able to perform view planning based on visual information of multiple objects; and adjust action ranges according to the task status. Evaluated on the AVD dataset, A2P leads to 21.9% increase in detection accuracy in unfamiliar environments, while improving efficiency by 22.7%. On the T-LESS dataset, multi-object detection boosts efficiency by more than 30% while achieving equivalent detection accuracy.

Soft robots have been applied widely to various constrained scenarios due to the advantages over traditional rigid manipulators such as softness, deformability and adaptability to constrained surroundings. To make full use of this merit, this paper proposes a method that integrates collision detection, localization and force estimation for a cable-driven soft manipulator without any prior geometrical knowledge of its surroundings. First of all, a collision detection algorithm is presented based upon Cosserat-rod statics by a threshold method through using the cable tension and the shape information, which are obtained by the load cells and the Vicon system, respectively. Secondly, a collision localization and force estimation method is proposed through optimizing the discrepancy between the actual and the theoretical shapes. Finally, experiments are carried out to validate these algorithms. The experimental results demonstrate that the site, the magnitude as well as the direction can be estimated.

Robots operating in open environments expect to have robust plans to achieve tasks successfully under environment uncertainties. However, both partial observability and dynamics of environment states have significantly decreased the robustness of task achievement, making robot task planning much more challenging. The partially observable states require the robot to obtain observations for optimally acting of the task goal. Also, state dynamics expects the robot to continuously observe surroundings for acting safely. Both challenges practically demand the purposeful and tight interactions between robot state-changing actuating actions and sensor-based observation actions. This paper proposes a novel model of Adjoint Sensing and Acting (ASA) that explicitly defines two parallel and sequential interaction schemes between actuating and observation actions, as well as an extended Behavior Tree for a concrete implementation of above schemes. We further propose an interleaving task planning approach for planning ASA-style plans, which integrates a deliberative POMDP planner for pursuing task goals, and a reactive Behavior Tree executive for fast responding to unexpected events. We experimentally demonstrate that ASA interaction schemes are practical and applicable to model and plan the open environment robot tasks. The plans from the interleaving task planning approach are both reactive in run-time response and efficient in task achievement.

Due to the limited field of view (FOV), the probe-based confocal laser endomicroscopy (pCLE) imaging system remains challenging to be widely used in clinic. Existing video mosaicking approaches are usually troubled by poor real-time capability and sensitivity to tissue deformations and intensity fluctuations. In this paper, a novel pCLE mosaicking algorithm that simultaneously implements rigid probe motion tracking and inter-frame tissue deformation correction is proposed. The sum of conditional variance (SCV) metric with low calculational complexity and intensity variation invariance is embedded into the reconstruction of pCLE mosaics. We also deduce the derivatives of the SCV metric with respect to transformation variables to enhance the robustness of numerical solutions. Moreover, a parallel acceleration mechanism that maximally exploits the resources of graphics processing unit (GPU) is designed to boost mosaicking efficiency. Experiments on lens tissue paper and swine tissue highlight the excellent mosaicking accuracy of the proposed method in this field. It is also verified that the mosaic-king rate of the proposed method exceeds 14 frames per second (fps), which can keep up with the acquisition rate of most pCLE imaging systems (8 fps~12 fps). Benefiting from finer mosaics and faster speed, the proposed algorithm has high potential to achieve large and accurate mosaicking for intraoperative endomicroscopy.

Point set registration plays a critical role in robotics and computer vision. Early methods considered registration as a purely geometric problem, presenting excellent extensibility for various tasks due to their explicit handling of correspondences; statistical methods were later introduced to handle noise. However, the two categories of algorithms have evolved independently without sharing much in common. In this paper, we leverage the concept of information geometry to theoretically unify the two classes together by interpreting them as the same operation but in different spaces associated with respective metrics. Moreover, based on the proposed unification, we also develop a novel bandwidth estimation strategy to solve the long-standing problem of statistical registration algorithms, and demonstrate its theoretical and practical advantages over deterministic annealing, the most commonly used technique. We also present a case study to show how geometric and statistical approaches can benefit from each other.