ICRA 2021 论文列表

In the context of robotic high-precision soldering, we propose an image-based pin alignment control method based on active plastic deformation. The plastic deformation is a well-known failure mechanism in most situations, which includes a phenomenon that the objects do not return original state. Here, in contrast to this convention, we utilize the plastic deformation of the metal pin to do pin alignment for improving the quality of the solder joint. To address this, we embed the springback compensation into the image-based pin alignment controller. Lastly, the proposed strategy is successfully demonstrated and evaluated in a practical modified robotic manipulation system. The result shows that the alignment error is less than 20µm, which is far less than pin alignment without considering plastic deformation and elastic recovery. This work considers active plastic deformation and spontaneous elastic recovery of soft object, which would greatly promote the use of robotics in micromanufacturing and microfabrication in lad and industry, especially for soft objects.

This paper presents a framework that uses language instructions to define the constraints and objectives for robots gathering information about their environment. Designing autonomous robotic sampling missions requires deep knowledge of both autonomy systems and scientific domain expertise. Language commands provide an intuitive interface for operators to give complex instructions to robots. The key insight we leverage is using topological constraints to define routing directions from the language instruction such as ‘route to the left of the island.’ This work introduces three main contributions: a framework to map language instructions to constraints and rewards for robot planners, a topology constrained information gathering algorithm, and an automatic semantic feature detection algorithm for upwelling fronts. Our work improves on existing methods by not requiring training data with language instruction to planner constraint pairs, allowing new robotic domains such as marine robotics to use our method. This paper provides results demonstrating our framework producing correct constraints for 84.6% of instructions, from a systematically generated corpus of over 1.1 million instructions We also demonstrate the framework producing robot plans from language instructions for real-world scientific sampling missions with the Slocum underwater glider.

Robots are usually equipped with cameras to explore the indoor scene and it is expected that the robot can well describe the scene with natural language. Although some great success has been achieved in image and video captioning technology, especially on many public datasets, the caption generated from indoor scene video is still not informative and coherent enough. In this paper, we propose the problem of Indoor Scene Captioning from Streaming Video, which aims at generating a more accurate and informative caption from streaming video. To solve this problem, we firstly design an algorithm to organize the visual information of the indoor scene into a scene graph, and then implement a scene graph guided captioning method, which takes the scene graph and video frames as input to generate the caption from the video streaming. The proposed framework is evaluated both on the AI2THOR dataset and a real-world robotic platform, demonstrating the effectiveness of the framework.

Robotic assembly tasks involve complex and low-clearance insertion trajectories with varying contact forces at different stages. While the nominal motion trajectory can be easily obtained from human demonstrations through kinesthetic teaching, teleoperation, simulation, among other methods, the force profile is harder to obtain especially when a real robot is unavailable. It is difficult to obtain a realistic force profile in simulation even with physics engines. Such simulated force profiles tend to be unsuitable for the actual robotic assembly due to the reality gap and uncertainty in the assembly process. To address this problem, we present a combined learning-based framework to imitate human assembly skills through hybrid trajectory learning and force learning. The main contribution of this work is the development of a framework that combines hierarchical imitation learning, to learn the nominal motion trajectory, with a reinforcement learning-based force control scheme to learn an optimal force control policy. To further improve the imitation learning part, we develop a hierarchical architecture, following the idea of goal-conditioned imitation learning, to generate the trajectory learning policy on the skill level offline. Through experimental validations, we corroborate that the proposed learning-based framework is robust to uncertainty in the assembly task, can generate high-quality trajectories, and can find suitable force control policies, which adapt to the task’s force requirements more efficiently.

An autonomous robot that is able to physically guide humans through narrow and cluttered spaces could be a big boon to the visually-impaired. Most prior robotic guiding systems are based on wheeled platforms with large bases with actuated rigid guiding canes. The large bases and the actuated arms limit these prior approaches from operating in narrow and cluttered environments. We propose a method that introduces a quadrupedal robot with a leash to enable the robot-guidinghuman system to change its intrinsic dimension (by letting the leash go slack) in order to fit into narrow spaces. We propose a hybrid physical Human Robot Interaction model that involves leash tension to describe the dynamical relationship in the robot-guiding-human system. This hybrid model is utilized in a mixed-integer programming problem to develop a reactive planner that is able to utilize slack-taut switching to guide a blind-folded person to safely travel in a confined space. The proposed leash-guided robot framework is deployed on a Mini Cheetah quadrupedal robot and validated in experiments (Video 1)

We present a novel approach to robotic grasp planning using both a learned grasp proposal network and a learned 3D shape reconstruction network. Our system generates 6-DOF grasps from a single RGB-D image of the target object, which is provided as input to both networks. By using the geometric reconstruction to refine the candidate grasp produced by the grasp proposal network, our system is able to accurately grasp both known and unknown objects, even when the grasp location on the object is not visible in the input image.This paper presents the network architectures, training procedures, and grasp refinement method that comprise our system. Experiments demonstrate the efficacy of our system at grasping both known and unknown objects (91% success rate in a physical robot environment, 84% success rate in a simulated environment). We additionally perform ablation studies that show the benefits of combining a learned grasp proposal with geometric reconstruction for grasping, and also show that our system outperforms several baselines in a grasping task.

We present the design, implementation, and evaluation of RF-Grasp, a robotic system that can grasp fully-occluded objects in unknown and unstructured environments. Unlike prior systems that are constrained by the line-of-sight perception of vision and infrared sensors, RF-Grasp employs RF (Radio Frequency) perception to identify and locate target objects through occlusions, and perform efficient exploration and complex manipulation tasks in non-line-of-sight settings.RF-Grasp relies on an eye-in-hand camera and batteryless RFID tags attached to objects of interest. It introduces two main innovations: (1) an RF-visual servoing controller that uses the RFID’s location to selectively explore the environment and plan an efficient trajectory toward an occluded target, and (2) an RF-visual deep reinforcement learning network that can learn and execute efficient, complex policies for decluttering and grasping.We implemented and evaluated an end-to-end physical prototype of RF-Grasp. We demonstrate it improves success rate and efficiency by up to 40-50% over a state-of-the-art baseline. We also demonstrate RF-Grasp in novel tasks such mechanical search of fully-occluded objects behind obstacles, opening up new possibilities for robotic manipulation. Qualitative results (videos) available at rfgrasp.media.mit.edu

Electrospinning has exhibited excellent benefits to treat the trauma for tissue engineering due to its produced micro/nano fibrous structure. It can effectively adhere to the tissue surface for long-term continuous therapy. This paper develops a robotic electrospinning platform for endoluminal therapy. The platform consists of a continuum manipulator, the electrospinning device, and the actuation unit. The continuum manipulator has two bending sections to facilitate the steering of the tip needle for a controllable spinning direction. Non-circular joint profile is carefully designed to enable a constant length of the centreline of a continuum manipulator for stable fluid transmission inside it. Experiments are performed on a bronchus phantom, and the steering ability and bending limitation in each direction are also investigated. The endoluminal electrospinning is also fulfilled by a trajectory following and points targeting experiments. The effective adhesive area of the produced fibre is also illustrated. The proposed robotic electrospinning shows its feasibility to precisely spread more therapeutic drug to construct fibrous structure for potential endoluminal treatments.

Zebrafish (Danio Rerio) larvae have long been an important model organism for biomedicine and drug discovery. It is difficult to deliver the external materials into the circulatory system by conventional exposing administration, while vein microinjection is more efficient but more challenging. In this paper, a robotic cardinal vein microinjection system was presented for zebrafish larvae. The key points of injection pipette during penetration were analyzed, and their 3D positions were identified online by combining motion analysis and visual calibration. 3D path planning of the pipette tip was designed automatedly before injection. The injection system was evaluated by injecting green fluorescent microspheres into 20 zebrafish larvae. The experiment results show that the fluorescent microspheres spread rapidly in the blood circulation system after injection, achieving a success rate of 85.0% and a survival rate of 94.1%. The superior performance provided by the system will significantly facilitate tumor xenografts and canner studies on zebrafish.

A computed tomography (CT)-guided robotic needle requires registration to transfer coordinates between the robot and CT image for accurate insertion. In our previous work, we proposed a geometric marker that allows direct registration between a CT image and robot and demonstrated its proof of concept. In this paper, we present a registration algorithm for calculating the six-degrees-of-freedom error of one CT scan, and we demonstrated the registration with our needle insertion robot. We obtained its geometric shape from differences between multiple images, which we converted into the coordinate axes of the marker, to calculate the posture of the marker. Since the algorithm uses changes in the cross-sectional shapes of images, it can be adapted for markers of different sizes. In the evaluation, the registration error for insertion by the CT-guided robotic needle with the proposed algorithm was calculated to be 1.8 mm and 0.35°. The accuracy of the proposed algorithm is sufficient for lower abdominal insertion and shows its potential for clinical applications. When the markers are half the size of the original, the rotational error is the same as the original, but the positional error is about half. This suggested the possibility of miniaturizing the marker.

We propose an approach for robot-supervised learning that automates label generation for semantic segmentation with Convolutional Neural Networks (CNNs) for crop row detection in a field. Using a training robot equipped with RTK GNSS and RGB camera, we train a neural network that can later be used for pure vision-based navigation. We test our approach on an agri-robot in a strawberry field and successfully train crop row segmentation without any hand-drawn image labels. Our main finding is that the resulting segmentation output of the CNN shows better performance than the noisy labels it was trained on. Finally, we conduct open-loop field trials with our agri-robot and show that row-following based on the segmentation result is accurate enough for closed-loop guidance. We conclude that training with noisy segmentation labels is a promising approach for learning vision-based crop row following.

Safely generating impacts with robots is challenging due to subsequent discontinuous velocity and high impact forces. We aim at increasing the impact velocity – the robot’s relative speed prior to contact – such that impact-tasks like grabbing and boxing are made with the highest allowable speed performance when needed. Previous works addressed this problem for rigid bodies’ impacts. This letter proposes a control paradigm for generating intentional impacts with deformable contacts that incorporates hardware and task constraints. Based on data-driven learning of the shock-absorbing soft dynamics and a novel mapping of joint-space limits to contact-space, we devise a constrained model-predictive control to maximize the intentional impact within a feasible, robot-safe level. Our approach is assessed with real-robot experiments on the redundant Panda manipulator, demonstrating high pre-impact velocities (up to 0.9 m/s) of a rigid end-effector on soft objects and an end-effector soft suction-pump on rigid or deformable objects.

Robots need to be able to work in multiple different environments. Even when performing similar tasks, different behaviour should be deployed to best fit the current environment. In this paper, We propose a new approach to navigation, where it is treated as a multi-task learning problem. This enables the robot to learn to behave differently in visual navigation tasks for different environments while also learning shared expertise across environments. We evaluated our approach in both simulated environments as well as real-world data. Our method allows our system to converge with a 26% reduction in training time, while also increasing accuracy.

Challenging manipulation tasks can be solved effectively by combining individual robot skills, which must be parameterized for the concrete physical environment and task at hand. This is time-consuming and difficult for human programmers, particularly for force-controlled skills. To this end, we present Shadow Program Inversion (SPI), a novel approach to infer optimal skill parameters directly from data. SPI leverages unsupervised learning to train an auxiliary differentiable program representation ("shadow program") and realizes parameter inference via gradient-based model inversion. Our method enables the use of efficient first-order optimizers to infer optimal parameters for originally non-differentiable skills, including many skill variants currently used in production. SPI zero-shot generalizes across task objectives, meaning that shadow programs do not need to be retrained to infer parameters for different task variants. We evaluate our methods on three different robots and skill frameworks in industrial and household scenarios. Code and examples are available at https://innolab.artiminds.com/icra2021.

Navigating fluently around pedestrians is a necessary capability for mobile robots deployed in human environments, such as buildings and homes. While research on social navigation has focused mainly on the scalability with the number of pedestrians in open spaces, typical indoor environments present the additional challenge of constrained spaces such as corridors and doorways that limit maneuverability and influence patterns of pedestrian interaction. We present an approach based on reinforcement learning (RL) to learn policies capable of dynamic adaptation to the presence of moving pedestrians while navigating between desired locations in constrained environments. The policy network receives guidance from a motion planner that provides waypoints to follow a globally planned trajectory, whereas RL handles the local interactions. We explore a compositional principle for multi-layout training and find that policies trained in a small set of geometrically simple layouts successfully generalize to more complex unseen layouts that exhibit composition of the structural elements available during training. Going beyond walls-world like domains, we show transfer of the learned policy to unseen 3D reconstructions of two real environments. These results support the applicability of the compositional principle to navigation in real-world buildings and indicate promising usage of multi-agent simulation within reconstructed environments for tasks that involve interaction. https://ai.stanford.edu/∼cdarpino/socialnavconstrained/

A novel motion planning method is proposed to generate human-like motion for anthropomorphic robot arms. Its highlight is to consider the robot arm to be human-like not only in its configuration but also in its motion patterns. To achieve this, the intrinsic mechanisms of human arm motion generation are transferred to robot motion planning. First, human arm motion is modeled using human arm motion primitives. The mechanisms of human arm motion generation are dissected from a large number of motion samples, reflected in the types, sequencing and quantification rules/laws of the primitives. Next, the human arm motion patterns are studied based on primitive chains. Finally, a new motion planning method is built that autonomously performs motion pattern decisions, motion time allocation, and joint trajectory generation. The proposed method is validated by a motion planning app and a robot simulation.

Robot motion control aims to generate control inputs for a robotic system to track a planned trajectory. Feedback provided by sensors plays an essential role in motion control by improving system performance when external disturbances and/or initial errors exist. However, feedback signals, such as images are often of a large size, which imposes a heavy computational burden on the system. In this paper, a new robot motion control scheme is proposed based on compressive feedback to improve feedback rate. The controller is designed in non-vector space using compressive feedback. As an application, visual servoing is formulated under the proposed framework by considering a feedback image as a set, instead of a traditional feature vector. Experiments are conducted to validate the proposed scheme.

Robot learning is often simplified to planar manipulation due to its data consumption. Then, a common approach is to use a fully-convolutional neural network (FCNN) to estimate the reward of grasp primitives. In this work, we extend this approach by parametrizing the two remaining, lateral degrees of freedom (DoFs) of the primitives. We apply this principle to the task of 6 DoF bin picking: We introduce a model-based controller to calculate angles that avoid collisions, maximize the grasp quality while keeping the uncertainty small. As the controller is integrated into the training, our hybrid approach is able to learn about and exploit the model-based controller. After real-world training of 27 000 grasp attempts, the robot is able to grasp known objects with a success rate of over 92 % in dense clutter. Grasp inference takes less than 50 ms. In further real-world experiments, we evaluate grasp rates in a range of scenarios including its ability to generalize to unknown objects. We show that the system is able to avoid collisions, enabling grasps that would not be possible without primitive adaption.

Robots hold great potential for supporting exercise and physical therapy, but such systems are often cumbersome to set up and require expert supervision. We aim to solve these concerns by combining Captury Live, a real-time markerless motion-capture system, with a Rethink Robotics Baxter Research Robot to create the Robot Interaction Studio. We evaluated this platform for unsupervised human-robot interaction (HRI) through a 75-minute-long user study with seven adults who were given minimal instructions and no feedback about their actions. The robot used sounds, facial expressions, facial colors, head motions, and arm motions to sequentially present three categories of cues in randomized order while constantly rotating its face screen to look at the user. Analysis of the captured user motions shows that the cue type significantly affected the distance subjects traveled and the amount of time they spent within the robot’s reachable workspace, in alignment with the design of the cues. Heat map visualizations of the recorded user hand positions confirm that users tended to mimic the robot’s arm poses. Despite some initial frustration, taking part in this study did not significantly change user opinions of the robot. We reflect on the advantages of the proposed approach to unsupervised HRI as well as the limitations and possible future extensions of our system.