ICRA 2021 论文列表

We introduce a multi-functional robotic gripper equipped with a set of actions required for disassembly of electromechanical devices. The gripper consists of a robot arm with 5 degrees of freedom (DoF) for manipulation and a jaw gripper with a 1-DoF rotation joint and a 1-DoF closing joint. The system enables manipulation in 7 DoF and offers the ability to reposition objects in hand and to perform tasks that usually require bimanual systems. The sensor system of the gripper includes relative and absolute joint encoders, force and pressure sensors to provide feedback about interaction forces, a tool- mounted camera for screw detection and precise placement of the tool tip using image-based visual servoing. We present a data-driven method for estimating joint torques based on the output voltage and motor speed. Further, we provide methods for teaching disassembly actions based on human demonstration, their representation as movement primitives and execution based on sensory feedback. We provide quantitative results regarding positioning and torque estimation accuracy, disassembly success rate and qualitative results regarding the successful disassembly of hard disc drives.

This paper describes the development of a single reduced-order dynamic model that captures running on land, running while submerged, and for the first time swimming on the surface of water. By capturing the effect of fluid forces on both the body and the leg, the Spring-Loaded Inverted Pendulum (SLIP) model is extended to operate in amphibious and aquatic regimes. Three distinct stable motion patterns, or dog-paddle type gaits are identified when swimming at the air-water interface. The model shows that, for surfaces swimming, alteration of the leg stroke frequency and length produces gaits that are either smooth and efficient or are vertically oscillatory and exhibit rapid disturbance rejection. Furthermore, when the model is examined at the physical parameter values corresponding to dogs (specifically Labrador Retrievers), the animal-based control parameters demonstrate nearly optimal performance.

Human collaboration is more likely to lead to cognitive growth when all group-members are actively involved in the collaborative process. However, there are cases that intragroup relationships need support. In this paper, we present an autonomous robotic system designed to interact with a pair of children in a problem-solving setting, aiming to understand how the robot behaviour impacts the group-members’ social dynamics. We developed an autonomous system with the Haru robot which we evaluated with an experimental study with 5-8yo children (N =84) to test the impact of the robot’s cognitive reliability and social positioning on human-to-human social dynamics, task performance and help-seeking behaviour. All participants took part in a baseline session (without the robot), an intervention (with the robot in a turn-taking setting) and an evaluation session (with a robot in a voluntary interaction setting). Results indicate that children who interacted with the reliable robot had a better task performance but children who interacted with the unreliable robot exhibited more task-related social interactions. Based on the results, we propose an interaction design concept which combines the set of the evaluated robot behaviours for an adaptive targeted support of child-robot teaming.

This paper extends the results in [1] considering time delays in Standard Car Following models (CFMs). In [1], connected vehicles are characterized with CFMs models and controlled using linear string analysis to stabilize single-lane car following of human-driven vehicles (HDVs). In this paper, we revisit stability and safety conditions for traffic considering time delays due to lags in the input signals. We perform plant stability and string stability analysis to derive these conditions. Then we obtain the optimal number of HDVs that can be stabilized using one autonomous vehicle. Numerical simulations are provided to implement a case study on Intelligent Driver Model to discuss the influence of time delays that arise on HDVs and the autonomous vehicle on optimal number of HDVs that can be stabilized using one autonomous vehicle.

Registration is a fundamental task in medical robotics and is often a crucial step for many downstream tasks such as motion analysis, intra-operative tracking and image segmentation. Popular registration methods such as ANTs and NiftyReg optimize objective functions for each pair of images from scratch, which are time-consuming for 3D and sequential images with complex deformations. Recently, deep learning-based registration approaches such as VoxelMorph have been emerging and achieve competitive performance. In this work, we construct a test-time training for deep deformable image registration to improve the generalization ability of conventional learning-based registration model. We design multi-scale deep networks to consecutively model the residual deformations, which is effective for high variational deformations. Extensive experiments validate the effectiveness of multi-scale deep registration with test-time training based on Dice coefficient for image segmentation and mean square error (MSE), normalized local cross-correlation (NLCC) for tissue dense tracking tasks.

As robots work with human teams, they will be expected to fluently coordinate with them. While people are adept at coordination and real-time adaptation, robots still lack this skill. In this paper, we introduce TANDEM: Temporal Anticipation and Adaptation for Machines, a series of neurobiologically-inspired algorithms that enable robots to fluently coordinate with people. TANDEM leverages a humanlike understanding of external and internal temporal changes to facilitate coordination. We experimentally validated the approach via a human-robot collaborative drumming task across tempo-changing rhythmic conditions. We found that an adaptation process alone enables a robot to achieve human-level performance. Moreover, by combining anticipatory knowledge along with an adaptation process, robots can potentially perform such tasks better than people. We hope this work will enable researchers to create robots more sensitive to changes in team dynamics.

Additive Manufacturing, has evolved beyond prototyping to manufacturing end-products. The authors are involved in developing a large-scale extrusion-based 3D printer to print mining equipment - a Gravity Separation Spiral, and embedding sensors to monitor the operational conditions re-motely. This paper presents a temperature-compensated strain sensor that can be 3D printed inline within large-scale 3D printed equipment. The sensor is printed using conductive carbon filament and embedded in a Polylactic acid (PLA) base. A half-bridge setup is proposed to reduce the impact of temperature variations. Temperature-controlled tests have been conducted with the proposed half-bridge and compared with a non-temperature compensated quarter-bridge setup. Results show that the half-bridge configuration reduces the temperature impact on the strain measurement significantly (68%) compared to the quarter-bridge, in the range of 25-40 °C. Deflection testing conducted on the printed sensor shows a near-linear relationship between bending strain and voltage. Multiple bending cycles have shown that there is no significant hysteresis. ANSYS simulations are used to accurately estimate the internal temperature since embedding a temperature sensor would affect the structural integrity. Although carbon black material is naturally brittle, steps have been taken in the design to avoid undesirable cracking. Results from laser microscopy analysis of the printed traces showed no crack defects.

Sensor coverage is the critical multi-robot problem of maximizing the detection of events in an environment through the deployment of multiple robots. Large multi-robot systems are often composed of simple robots that are typically not equipped with a complete set of sensors, so teams with comprehensive sensing abilities are required to properly cover an area. Robots also exhibit multiple forms of relationships (e.g., communication connections or spatial distribution) that need to be considered when assigning robot teams for sensor coverage. To address this problem, in this paper we introduce a novel formulation of sensor coverage by multi-robot systems with heterogeneous relationships as a graph representation learning problem. We propose a principled approach based on the mathematical framework of regularized optimization to learn a unified representation of the multi-robot system from the graphs describing the heterogeneous relationships and to identify the learned representation’s underlying structure in order to assign the robots to teams. To evaluate the proposed approach, we conduct extensive experiments on simulated multi-robot systems and a physical multi-robot system as a case study, demonstrating that our approach is able to effectively assign teams for heterogeneous multi-robot sensor coverage.

Lecturers of Engineering courses around the world are struggling to increase the engagement of students through the introduction of appropriate hands-on activities and assignments. In Biomechatronics and Robotics courses these assignments typically focus on how certain devices are designed, modelled, fabricated, or controlled. The hardware for these assignments is usually purchased by some external vendor and the students only get the chance to analyze it or program it, so as to execute a useful task (e.g., programming mobile robots to perform path following tasks). Student engagement can be increased by instructing the students to prepare the hardware for their assignment. This also increases the sense of ownership of the project outcomes. In this paper, we present how a robotic gripper / hand design project and the introduction of a grasping and manipulation competition as a course assignment, can significantly increase the student engagement and their understanding of the taught concepts. The presented best practices have been trialed over the last four years in two different courses (one undergraduate and one postgraduate) of the Department of Mechanical Engineering at the University of Auckland in New Zealand. For the particular assignment the students were asked to fully develop a robotic gripper or hand from scratch using a single actuator (only the actuator and the power electronics were provided). The performance of the developed devices was assessed through the participation in a grasping and manipulation competition. All the details of the proposed assignment are presented, hoping that they could help other lecturers and teachers to prepare similar activities.

We present a novel task planner - TaskNet for an autonomous excavator based on a data-driven method, which plans feasible task-level sequence by learning from demonstration data. Given a high-level excavation objective, our TaskNet planner can decompose it into sub-tasks, each of which can be further decomposed into task primitives with specifications. We train our TaskNet using an excavation trace generator and evaluate its performance using a 3D physically-based terrain and excavator simulator. As compared to imitation learning-based methods, the experimental results show that TaskNet can effectively learn task decomposition strategies. The resulting sequences of task primitives can be used as inputs by any excavator motion planner for generating feasible joint-level trajectories. We further validate TaskNet on a state-of-the-art autonomous excavator hardware and software system. The 49-ton autonomous excavator can successfully perform material loading tasks.

This paper introduces a novel task-space decomposed motion planning framework for multi-robot simultaneous locomotion and manipulation. When several manipulators hold an object, closed-chain kinematic constraints are formed, and it will make the motion planning problems challenging by inducing lower-dimensional singularities. Unfortunately, the constrained manifold will be even more complicated when the manipulators are equipped with mobile bases. We address the problem by introducing a dual-resolution motion planning framework which utilizes a convex task region decomposition method, with each resolution tuned to efficient computation for their respective roles. Concretely, this dual-resolution approach enables a global planner to explore the low-dimensional decomposed task-space regions toward the goal, then a local planner computes a path in high-dimensional constrained configuration space. We demonstrate the proposed method in several simulations, where the robot team transports the object toward the goal in the obstacle-rich environments.

Natural control of limb motion is continuous and progressively adaptive to individual intent. While intuitive interfaces have the potential to rely on the neuromuscular input by the user for continuous adaptation, continuous volitional control of assistive devices that can generalize across various tasks has not been addressed. In this study, we propose a method to use spatiotemporal ultrasound features of the rectus femoris and vastus intermedius muscles of able-bodied individuals for task-invariant learning of continuous knee kinematics during steady-state and transient ambulation. The task-invariant learning paradigm was statistically evaluated against a task-specific paradigm for the steady-state (1) level-walk, (2) incline, (3) decline, (4) stair ascent, and (5) stair descent ambulation tasks. The transitions between steady-state stair ambulation and level-ground walking were also investigated. It was observed that the continuous knee kinematics can be learned using a task-invariant learning paradigm with statistically comparable accuracy to a task-specific paradigm. Statistical analysis further revealed that incorporating the temporal ultrasound features significantly improves the accuracy of continuous estimations (p < 0.05). The average root mean square errors (RMSEs) of knee angle and angular velocity estimation were 7.06° and 53.1°/sec, respectively, for the task-invariant learning compared to 6.00° and 51.8°/sec for the task-specific models. High accuracy of continuous task-invariant paradigms overcome the barrier of task-specific control schemes and motivate the implementation of direct volitional control of lower-limb assistive devices using ultrasound sensing, which may eventually enhance the intuitiveness and functionality of these devices towards a "free form" control approach.

Visual perception in autonomous driving is a crucial part of a vehicle to navigate safely and sustainably in different traffic conditions. However, in bad weather such as heavy rain and haze, the performance of visual perception is greatly affected by several degrading effects. Recently, deep learning-based perception methods have addressed multiple degrading effects to reflect real-world bad weather cases but have shown limited success due to 1) high computational costs for deployment on mobile devices and 2) poor relevance between image enhancement and visual perception in terms of the model ability. To solve these issues, we propose a task-driven image enhancement network connected to the high-level vision task, which takes in an image corrupted by bad weather as input. Specifically, we introduce a novel low memory network to reduce most of the layer connections of dense blocks for less memory and computational cost while maintaining high performance. We also introduce a new task-driven training strategy to robustly guide the high-level task model suitable for both high-quality restoration of images and highly accurate perception. Experiment results demonstrate that the proposed method improves the performance among lane and 2D object detection, and depth estimation largely under adverse weather in terms of both low memory and accuracy.

In reality, there is still much to be done for robots to be able to perform manipulation actions with full autonomy. Complicated manipulation tasks, such as cooking, may still require a person to perform some actions that are very risky for a robot to perform. On the other hand, some other actions may be very risky for a human with physical disabilities to perform. Therefore, it is necessary to balance the workload of a robot and a human based on their limitations while minimizing the effort needed from a human in a collaborative robot (cobot) set-up. This paper proposes a new version of our functional object-oriented network (FOON) that integrates weights in its functional units to reflect a robot’s chance of successfully executing an action of that functional unit. The paper also presents a task planning algorithm for the weighted FOON to allocate manipulation action load to the robot and human to achieve optimal performance while minimizing human effort. Through a number of experiments, this paper shows several successful cases in which using the proposed weighted FOON and the task planning algorithm allow a robot and a human to successfully complete complicated tasks together with higher success rates than a robot doing them alone.

Teleoperating robotic arms is a challenging task that requires years of training to master. It is mentally demanding, as the operator must internally compute transformations, or rely on muscle memory, to perform even the simplest tasks. Alternative methods that rely on embodiment –the immersive, first person experience of controlling the robot from its point of view are recently becoming more popular, thanks to the emergence of mixed reality devices. These methods create an intuitive experience by tracking the users motions, and retargetting them to the robot. However, even recent hardware fails at achieving total immersion, due to inherent discrepancies such as latency, imperfect tracking, and the differences between human and robot motor systems. Thus, performing even simple pick-and-place tasks with these systems, while more intuitive, is still cumbersome, and far from the level of human performance.In this paper we propose an immersive system that aims to bridge this gap. The system tracks the user’s motion and retargets them to the robot as usual, but it also detects the user’s intent, that is, the task they wish to perform. Based on this knowledge, the system can autocorrect the motion when it is about to fail, in a seamless manner, such that the task is successfully performed. We evaluate the efficacy of our autocorrection system in a user study. The results show a statistically significant performance improvement in terms of operation accuracy and time.

We propose a targetless method for calibrating the extrinsic parameters among multiple cameras and a LiDAR sensor using object pose estimation. Contrast to previous targetless methods requiring certain geometric features, the proposed method exploits any objects of unspecified shapes in the scene to estimate the calibration parameters in single-scan configuration. Semantic objects in the scene are initially segmented from each modal measurement. Using multiple images, a 3D point cloud is reconstructed up-to-scale. By registering the up-to-scale point cloud to the LiDAR point cloud, we achieve an initial calibration and find correspondences between point cloud segments and image object segments. For each point cloud segment, a 3D mesh model is reconstructed. Based on the correspondence information, the color appearance model for the mesh can be elaborately generated with corresponding object instance segment within the images. Starting from the initial calibration, the calibration is gradually refined by using an object pose estimation technique with the appearance models associated with the 3D mesh models. The experimental results confirmed that the proposed framework achieves multimodal calibrations successfully in a single shot. The proposed method can be effectively applied for extrinsic calibration for plenoptic imaging systems of dozens of cameras in single-scan configuration without specific targets.

Semantic segmentation has attracted increasing attention due to its important role in self-driving, and it is often realized by supervised learning with large number of well labeled maps. However, the labeled images are hard to be obtained in most circumstances, and the common way for unsupervised semantic segmentation is usually implemented by transferring the knowledge from source supervised domain to target unsupervised domain. Most researches focus on encouraging target predictions to be closer to the source ones through a weight-sharing network, and achieve certain performance. However, these methods often suffer from the domain shift problem that the networks are often trained towards the source domain and lead to performance degradation. In this paper, we propose a target-targeted domain adaptation approach by focusing the training on target domain. Our model consists of two components: the Image-to-image Translation (IIT) module to translate the source image to target domain and the Target-targeted Segmentation Adaptation (TSA) module to focus the semantic segmentation on target domain. The IIT module deals with image space alignment while the TSA module bridges the domain gap at the segmentation map level. In addition, we design a closed-loop learning to promote each other by employing feedback from TSA to IIT. Extensive experiments on GTA5 and SYNTHIA to Cityscapes demonstrate the effectiveness of our method in domain adaptation of unsupervised semantic segmentation.

Magnetic torsion springs are capable of producing unique and useful torque-displacement responses that are not possible with elastic springs. Millimeter-scale magnetically-actuated robots, which are gaining increasing interest in biomedical applications, would benefit from the use of magnetic torsion springs. However, existing magnetic torsion springs are difficult to fabricate at that scale and can only produce sinusoid-like responses. Here we show that the magnets embedded in the links of a robot for actuation purposes can also be leveraged to produce torsion spring-like behavior. This Simultaneous Magnetic Actuation and Restoring Torque (SMART) spring design can enable switching or pop-up behaviour in millimeter-scale magnetically-actuated mechanisms. A novel analytical model, validated both numerically and experimentally, is used to design constant-stiffness and nonlinear bistable SMART springs. These springs are integrated into a novel 3.5 mm diameter magnetic robot manipulator.

Object insertion is a classic contact-rich manipulation task. The task remains challenging, especially when considering general objects of unknown geometry, which significantly limits the ability to understand the contact configuration between the object and the environment. We study the problem of aligning the object and environment with a tactile-based feedback insertion policy. The insertion process is modeled as an episodic policy that iterates between insertion attempts followed by pose corrections. We explore different mechanisms to learn such a policy based on Reinforcement Learning. The key contribution of this paper is to demonstrate that it is possible to learn a tactile insertion policy that generalizes across different object geometries, and an ablation study of the key design choices for the learning agent: 1) the type of learning scheme: supervised vs. reinforcement learning; 2) the type of learning schedule: unguided vs. curriculum learning ; 3) the type of sensing modality: force/torque vs. tactile; and 4) the type of tactile representation: tactile RGB vs. tactile flow. We show that the optimal configuration of the learning agent (RL + curriculum + tactile flow) exposed to 4 training objects yields an closed-loop insertion policy that inserts 4 novel objects with over 85.0% success rate and within 3~4 consecutive attempts. Comparisons between F/T and tactile sensing, shows that while an F/T-based policy learns more efficiently, a tactile-based policy provides better generalization. See supplementary video and results at https://sites.google.com/view/tactileinsertion.