ICRA 2021 论文列表

Cell manipulation is a key technology in many biomedical and clinical applications, in which end-effector alignment is a critical procedure. Presently, end-effector alignment is performed manually and suffers from large misalignment error and inconsistency. Manual alignment often undesirably moves the end-effector (e.g., a glass micropipette) out of the limited field of view under microscopy and risks breaking the fragile end-effector. This paper presents automated end-effector alignment for robotic cell manipulation. A rotational degree of freedom was added to a micromanipulator with translational degrees of freedom. The kinematic model of end-effector’s rotation was established, and the unknown model parameters were calibrated and updated via quadratic optimization. A controller was designed based on the kinematics modeling and parameter optimization to compensate for rotation-induced translation and achieve end-effector alignment. Experimental results demonstrate that the robotic alignment technique achieved an accuracy of 0.6±0.3° and a time cost of 18.5 ± 10.2 s, both significantly less than manual alignment. The developed controller cost significantly less time for micropipette alignment than the PID controller. A glass micropipette was used as the end-effector for human sperm immobilization, a critical procedure in clinical cell surgery. The success rate of sperm immobilization was 97% by robotic micropipette alignment, higher than the success rate of 90% by manual alignment due to the higher accuracy of robotic alignment.

Reacting to unexpected conditions and recovering from errors is crucial for robots to perform their missions continuously in dynamic environments. This paper introduces a novel framework aiming to handle errors impairing the quality of robot services concerning on-line management of error recovery. The framework is a combination of an execution generation tool, a learning module, and a recovery pipeline for error detection, diagnosis, and recovery in robotic systems. The execution generation tool generates control flow instructions and parameter files, then extracts necessary skills from the skill library based on application descriptions or recovery solution recipes. In the learning module, a Bayesian Network (BN) decision model is interactively and continually trained, incorporating application information structured according to the Failure Mode and Effects Analysis (FMEA) method. The recovery pipeline implements a decision model inferring the cause of an error and choosing recovery solutions. We will provide an experimental evaluation of our framework on a pick-and-place application performed by a robot arm in the physical world.

Calibration of sensors is fundamental to robust performance for intelligent vehicles. In natural environments, disturbances can easily challenge calibration. One possibility is to use natural objects of known shape to recalibrate sensors. An approach based on recognition of traffic signs, such as stop signs, and use of them for recalibration of cameras is presented. The approach is based on detection, geometry estimation, calibration, and recursive updating. Results from natural environments are presented that clearly show convergence and improved performance.

Policies trained in simulation often fail when transferred to the real world due to the ‘reality gap’ where the simulator is unable to accurately capture the dynamics and visual properties of the real world. Current approaches to tackle this problem, such as domain randomization, require prior knowledge and engineering to determine how much to randomize system parameters in order to learn a policy that is robust to sim-to-real transfer while also not being too conservative. We propose a method for automatically tuning simulator system parameters to match the real world using only raw RGB images of the real world without the need to define rewards or estimate state. Our key insight is to reframe the auto-tuning of parameters as a search problem where we iteratively shift the simulation system parameters to approach the real world system parameters. We propose a Search Param Model (SPM) that, given a sequence of observations and actions and a set of system parameters, predicts whether the given parameters are higher or lower than the true parameters used to generate the observations. We evaluate our method on multiple robotic control tasks in both sim-to-sim and sim-to-real transfer, demonstrating significant improvement over naive domain randomization. Project videos at https://yuqingd.github.io/autotuned-sim2real/.

Today’s robots are expected to fulfill different requirements originated from executing complex tasks in uncertain environments, often in collaboration with humans. To deal with this type of multi-objective control problem, hierarchical least-square optimization techniques are often employed, defining multiple tasks as objective functions, listed in hierarchical manner. The solution to the Inverse Kinematics problem requires to plan and constantly update the Cartesian trajectories. However, we propose an extension to the classical Hierarchical Quadratic Programming formulation, that allows to optimally generate these trajectories at control level. This is achieved by augmenting the optimization variable, to include the Cartesian reference and allow for the formulation of an adaptive compliance controller, which retains an impedancelike behaviour under external disturbances, while switching to an admittance-like behavior when collaborating with a human. The effectiveness of this approach is tested using a 7-DoF Franka Emika Panda manipulator in three different collaborative scenarios.

Robotic grasping is one of the most fundamental robotic manipulation tasks and has been actively studied. However, how to quickly teach a robot to grasp a novel target object in clutter remains challenging. This paper attempts to tackle the challenge by leveraging object attributes that facilitate recognition, grasping, and quick adaptation. In this work, we introduce an end-to-end learning method of attribute-based robotic grasping with one-grasp adaptation capability. Our approach fuses the embeddings of a workspace image and a query text using a gated-attention mechanism and learns to predict instance grasping affordances. Besides, we utilize object persistence before and after grasping to learn a joint metric space of visual and textual attributes. Our model is self-supervised in a simulation that only uses basic objects of various colors and shapes but generalizes to novel objects and real-world scenes. We further demonstrate that our model is capable of adapting to novel objects with only one grasp data and improving instance grasping performance significantly. Experimental results in both simulation and the real world demonstrate that our approach achieves over 80% instance grasping success rate on unknown objects, which outperforms several baselines by large margins. Supplementary material is available at https://sites.google.com/umn.edu/attributes-grasping.

Autonomous vehicle navigation in shared pedestrian environments requires the ability to predict future crowd motion both accurately and with minimal delay. Understanding the uncertainty of the prediction is also crucial. Most existing approaches however can only estimate uncertainty through repeated sampling of generative models. Additionally, most current predictive models are trained on datasets that assume complete observability of the crowd using an aerial view. These are generally not representative of real-world usage from a vehicle perspective, and can lead to the underestimation of uncertainty bounds when the on-board sensors are occluded. Inspired by prior work in motion prediction using spatio-temporal graphs, we propose a novel Graph Convolutional Neural Network (GCNN)-based approach, Attentional-GCNN, which aggregates information of implicit interaction between pedestrians in a crowd by assigning attention weight in edges of the graph. Our model can either output a probabilistic distribution or faster deterministic prediction, demonstrating applicability to autonomous vehicle use cases where either speed or accuracy with uncertainty bounds are required. To further improve the training of predictive models, we propose an automatically labelled pedestrian dataset collected from an intelligent vehicle platform representative of real-world use. Through experiments on a number of datasets, we show our proposed method achieves an improvement over the state of the art by 10% on Average Displacement Error (ADE) and 12% on Final Displacement Error (FDE) with fast inference speeds.

Human activity/behaviour monitoring and recognition is a key for facilitating humans robot interaction, and allows robots for a better scheduling of future operations. It is challenging and often addressed at different levels, such as human activity classification, future activity prediction and monitoring of the on-going activities. The paper proposes a novel attention-based learn-able pooling mechanism for human activity classification from RGB videos. Recently, most of the best performing human activity recognition approaches are based on 3D skeleton positions. The 3D skeleton positions are not always available in videos captured using RGB cameras, which are widely used in robotics applications. RGB videos contain rich spatio-temporal information and processing them semantically is a difficult task. Moreover, accurately capturing spatial information and long-term temporal dependencies is the key to achieving high recognition accuracy. We use an existing Convolutional Neural Network for image recognition to extract video features which are then processed using our innovative application of attention mechanism to focus the network on features that are more important for discrimination. Afterwards, we use a novel learn-able pooling mechanism to extract activity-aware spatio-temporal cues for efficient activity recognition. The proposed pooling mechanism learns the structural information from hidden states of a bidirectional Long Short-Term Memory network via Fisher Vectors.

Attention control is a key cognitive ability for humans to select information relevant to the current task. This paper develops a computational model of attention and an algorithm for attention-based probabilistic planning in Markov decision processes. In attention-based planning, the robot decides to be in different attention modes. An attention mode corresponds to a subset of state variables monitored by the robot. By switching between different attention modes, the robot actively perceives task-relevant information to reduce the cost of information acquisition and processing, while achieving near-optimal task performance. Though planning with attention-based active perception inevitably introduces partial observations, a partially observable MDP formulation makes the problem computational expensive to solve. Instead, our proposed method employs a hierarchical planning framework in which the robot determines what to pay attention to and for how long the attention should be sustained before shifting to other information sources. During the attention sustaining phase, the robot carries out a sub-policy, computed from an abstraction of the original MDP given the current attention. We use an example where a robot is tasked to capture a set of intruders in a stochastic gridworld. The experimental results show that the proposed method enables information- and computation-efficient optimal planning in stochastic environments.

Graceful degradation is a potential advantage of Multi-Robot Systems over Single-Robot Systems. In aerial robotics applications, such as infrastructure inspection, this trait is desirable as it would improve mission reliability despite the use of failure-prone low-cost drones. The Reliability-Aware Multi-Agent Coverage Path Planning (RA-MCPP) problem finds path plans for each robot to maximise the probability of mission completion by a given deadline. This paper proposes a path planner for RA-MCPP formulated in continuous time, enabling more complex realistic environments to be considered. The proposed method (i) extends a reliability evaluation framework to evaluate the Probability of Completion metric on asynchronous strategies on non-unit lattice graph environments, and (ii) introduces a greedy-genetic meta-heuristic optimisation method as a scalable and accurate RA-MCPP solver. This method is shown to provide plans with higher reliability when compared with existing approaches in three real inspection scenarios.

Existing multi-camera SLAM systems assume synchronized shutters for all cameras, which is often not the case in practice. In this work, we propose a generalized multi-camera SLAM formulation which accounts for asynchronous sensor observations. Our framework integrates a continuous-time motion model to relate information across asynchronous multi-frames during tracking, local mapping, and loop closing. For evaluation, we collected AMV-Bench, a challenging new SLAM dataset covering 482 km of driving recorded using our asynchronous multi-camera robotic platform. AMV-Bench is over an order of magnitude larger than previous multi-view HD outdoor SLAM datasets, and covers diverse and challenging motions and environments. Our experiments emphasize the necessity of asynchronous sensor modeling, and show that the use of multiple cameras is critical towards robust and accurate SLAM in challenging outdoor scenes. The supplementary material is located at: https://www.cs.toronto.edu/~ajyang/amv-slam

Using sampling to estimate the connectivity of high-dimensional configuration spaces has been the theoretical underpinning for effective sampling-based motion planners. Typical strategies either build a roadmap, or a tree as the underlying search structure that connects sampled configurations, with a focus on guaranteeing completeness and optimality as the number of samples tends to infinity. Roadmap-based planners allow preprocessing the space, and can solve multiple kinematic motion planning problems, but need a steering function to connect pairwise-states. Such steering functions are difficult to define for kinodynamic systems, and limit the applicability of roadmaps to motion planning problems with dynamical systems. Recent advances in the analysis of single-query tree-based planners has shown that forward search trees based on random propagations are asymptotically optimal. The current work leverages these recent results and proposes a multi-query framework for kinodynamic planning. Bundles of kinodynamic edges can be sampled to cover the state space before the query arrives. Then, given a motion planning query, the connectivity of the state space reachable from the start can be recovered from a forward search tree reasoning about a local neighborhood of the edge bundle from each tree node. The work demonstrates theoretically that considering any constant radial neighborhood during this process is sufficient to guarantee asymptotic optimality. Experimental validation in five and twelve dimensional simulated systems also highlights the ability of the proposed edge bundles to express high-quality kinodynamic solutions. Our approach consistently finds higher quality solutions compared to SST, and RRT, often with faster initial solution times. The strategy of sampling kinodynamic edges is demonstrated to be a promising new paradigm.

Temporal task planning guarantees a robot will succeed in its task as long as certain explicit and implicit assumptions about the robot’s operating environment, sensors, and capabilities hold. A robot executing a plan can silently fail to fulfill the task if the assumptions are violated at runtime. Monitoring assumption violations at runtime can flag silent failures and also provide mitigation and remediation opportunities. However, this requires means for describing assumptions combining temporal and quantitative data, automatic construction of correct monitors and ensuring a correct interplay between the planning execution and monitors. In this paper we propose combining temporal planning with stream runtime verification, which offers a high-level language to describe monitors together with guarantees on execution time and memory usage. We demonstrate our approach both in real and simulated flights for some typical mission scenarios.

The Dorsal Grasper, an assistive wearable grasping device, incorporates supernumerary fingers and an artificial palm with the forearm and back of the hand, respectively. It enables power wrap grasping and adduction pinching with its V-shaped soft fingers. Designed with C6/C7 spinal cord injury in mind, it takes advantage of active wrist extension that remains in this population after injury. We propose that allowing the operator to actively participate in applying grasp forces on the object, using the back of the hand, enables intuitive, fast and reliable grasping relevant for the execution of activities of daily living. Functional grasping is tested in three normative subjects and a person with C6 SCI using the Grasp and Release Test. Results indicate that this device provides promising performance on a subset of objects that complements the existing compensatory strategies used by people with C6/C7 SCI. We find that the addition of the artificial palm is important for increasing maximum grip strength, by increasing contact friction and protecting the opisthenar.

To generate assembly sequences that robots can easily handle, this study tackled assembly sequence generation (ASG) by considering two tradeoff objectives: (1) insertion conditions and (2) degrees of the constraints affecting the assembled parts. We propose a multi-objective genetic algorithm to balance these two objectives. Furthermore, we extend our previously proposed 3D computer-aided design (CAD)-based method for extracting three types of two-part relationship matrices from 3D models that include deformable parts. The interference between deformable and other parts can be determined using scaled part shapes. Our proposed ASG can produce Pareto-optimal sequences for multi-component models with deformable parts such as rubber bands, rubber belts, and roller chains. We further discuss the limitation and applicability of the generated sequences to robotic assembly.

Wearable robots are designed to provide physical assistance and rehabilitation training. Light-weight designs are desirable for human usage and parallel robots are quite suitable due to low moving inertia. One of the challenges of using wearable parallel robots is to compute the end-effector position/orientation from joint angle measurements, as the forward kinematics problem is computationally difficult. It becomes even more challenging if the kinematic model changes from the nominal model due to deflection of the mechanical members, manufacturing tolerances, or misalignments of the joints between the machine and the user.Artificial neural networks have been used to provide solution of the forward kinematics problem for parallel robots. However, their efficacy has not yet been studied in wearable applications where the linkages of the robots are not fully rigid. In this paper, we conducted a case study to investigate the performance of using neural network models to compute forward kinematics of a wearable robot. Our results show that the performance of neural networks is superior to numerical approaches and produces reasonable solutions even at the boundary of the robot workspace.

In this paper, we aim to solve inverse kinematics of the integrated robotic arm-hand systems to achieve precision grasping, provided the desired grasp configuration (contact points + contact normals). The key insights of our approach are three-fold. First, we propose a human-inspired thumb-first strategy and consider one finger of the robotic hand as the "thumb" to narrow down the search space and increase the success rate of the algorithm. Second, we formulate the arm-thumb serial chain as a closed chain while other fingers are still as serial chains such that the entire arm-hand system is controlled as a hybrid parallel-serial system. The closed-chain formulation truncates and simplifies the task hierarchy of the entire arm-hand system. Third, we attach a virtual revolute joint to the thumb’s tip with its rotation axis aligning with the thumb’s contact normal to allow this virtual joint to act as the embodiment of the thumb’s functional redundancy. By selecting the thumb’s joints including the virtual revolute joint as the active joints of the arm-thumb closed chain, the arm-thumb system’s self-motion (i.e., the palm pose) and the thumb’s functional redundancy can be directly controlled without using the null space projection. This provides a new possibility to control the self-motion of robot manipulators. Simulation results will demonstrate the advantages and superb performance of the proposed approach for solving the problem of inverse kinematics of achieving precision grasps compared to other classical approaches based on the Damped Least-Squares method [1] in terms of the average success rate (96% v.s. 12%).

Sampling-based motion planning under task constraints is challenging because the null-measure constraint manifold in the configuration space makes rejection sampling extremely inefficient, if not impossible. This paper presents a learning-based sampling strategy for constrained motion planning problems. We investigate the use of two well-known deep generative models, the Conditional Variational Autoencoder (CVAE) and the Conditional Generative Adversarial Net (CGAN), to generate constraint-satisfying sample configurations. Instead of precomputed graphs, we use generative models conditioned on constraint parameters for approximating the constraint manifold. This approach allows for the efficient drawing of constraint-satisfying samples online without any need for modification of available sampling-based motion planning algorithms. We evaluate the efficiency of these two generative models in terms of their sampling accuracy and coverage of sampling distribution. Simulations and experiments are also conducted for different constraint tasks on two robotic platforms.

In this paper, we present a method for finding approximate Nash equilibria in a broad class of reachability games. These games are often used to formulate both collision avoidance and goal satisfaction. Our method is computationally efficient, running in real-time for scenarios involving multiple players and more than ten state dimensions. The proposed approach forms a family of increasingly exact approximations to the original game. Our results characterize the quality of these approximations and show operation in a receding horizon, minimally-invasive control context. Additionally, as a special case, our method reduces to local gradient-based optimization in the single-player (optimal control) setting, for which a wide variety of efficient algorithms exist.