IROS 2022 论文列表

Grasping and manipulation are complex and demanding tasks, especially when executed in dynamic and unstructured environments. Typically, such tasks are executed by rigid articulated end-effectors, with a plethora of actuators that need sophisticated sensing and complex control laws to execute them efficiently. Soft robotics offers an alternative that allows for simplified execution of these demanding tasks, enabling the creation of robust, efficient, lightweight, and affordable solutions that are easy to control and operate. In this work, we introduce a new class of soft, kirigami-based robotic grippers, we study their post-contact behavior, and we investigate different cut patterns for their development. We follow an experimental approach in which several designs are proposed and employed in a series of grasping and force exertion tests to compare their capabilities and post-contact behavior. The results of such experiments indicate a clear relationship between degree of reconfiguration and grasping force, and provide key insights into the effect of the cut patterns in the performance of the designs. These findings are then used in the design process of an improved version of multi-layer, disposable kirigami grippers that are fabricated employing simple 3D printed layers and silicone rubber using the concept of Hybrid Deposition Manufacturing (HDM). A series of experimental results demonstrate that the proposed design and manufacturing methods can enable the creation of soft, kirigami-based grippers with superior grasping capabilities that can handle delicate, contaminated, and everyday life objects and can even be disposed off in an automated way (e.g., after handling hazardous materials, such as medical waste).

The automated cleaning of surfaces such as furniture, bathroom sinks, and even human bodies is challenging due to the three-dimensional nature of their geometries. Yet, enabling robots to effectively and safely perform these tasks would not only reduce user efforts spent on household cleaning chores, but would also alleviate the strenuous workload of caretakers as the elderly population continues to grow at an unprecedented rate. In this work, we unify the applications of wiping objects and bathing humans as a general contour-following problem. To this end, we utilize a depth camera-based soft tactile sensor to extract the contact geometries and force-correlated measures during interaction between the robot and the target object or body part, and design a general contour-following controller that not only maintains contact with the target throughout the cleaning process, but also regulates the amount of force applied. Our system enables successful cleaning of pipes, shelving, and even human limbs and torsos without the need for data-driven methods such as deep learning, upon which the majority of existing works have relied.

Facial paralysis is a challenging condition that alters a patient's ability to express emotion and communicate. Restoring facial movements thus has crucial implications for the patients' quality of life. This publication introduces an approach for artificial muscles implementation targeting facial reanimation, as well as the challenges and limitations of the proposed strategy. The aim is to develop a Dielectric Elastomer Actuator (DEA) prosthesis for patients suffering from facial paralysis. DEAs are chosen as they are soft, have large strain (up to 200%) and high dynamic behaviour (up to 20 kHz), making them a promising actuator for the application. Myo-electric signals are extracted using electromyography sensors from the Zygomaticus Major muscle, and they are processed in order to emphasize the activation phases. The resulting actuating signal is used to control a high voltage power supply to operate the DEA in an open loop. The resulting induced movement qualitatively matches the myoelectric signal, showing great potential of the proposed approach for facial paralysis reanimation.

Robots moving safely and in a socially compliant manner in dynamic human environments is an essential benchmark for long-term robot autonomy. However, it is not feasible to learn and benchmark social navigation behaviors entirely in the real world, as learning is data-intensive, and it is challenging to make safety guarantees during training. Therefore, simulation-based benchmarks that provide abstractions for social navigation are required. A framework for these benchmarks would need to support a wide variety of learning approaches, be extensible to the broad range of social navigation scenarios, and abstract away the perception problem to focus on social navigation explicitly. While there have been many proposed solutions, including high fidelity 3D simulators and grid world approximations, no existing solution satisfies all of the aforementioned properties for learning and evaluating social navigation behaviors. In this work, we propose SocialGym, a lightweight 2D simulation environment for robot social navigation designed with extensibility in mind, and a benchmark scenario built on SocialGym. Further, we present benchmark results that compare and contrast human-engineered and model-based learning approaches to a suite of off-the-shelf Learning from Demonstration (LfD) and Reinforcement Learning (RL) approaches applied to social robot navigation. These results demonstrate the data efficiency, task performance, social compliance, and environment transfer capabilities for each of the policies evaluated to provide a solid grounding for future social navigation research.

As robots across domains start collaborating with humans in shared environments, algorithms that enable them to reason over human intent are important to achieve safe inter-play. In our work, we study human intent through the problem of predicting trajectories in dynamic environments. We explore domains where navigation guidelines are relatively strictly defined but not clearly marked in their physical environments. We hypothesize that within these domains, agents tend to exhibit short-term motion patterns that reveal context information related to the agent's general direction, intermediate goals and rules of motion, e.g., social behavior. From this intuition, we propose Social-PatteRNN, an algorithm for recurrent, multi-modal trajectory prediction that exploits motion patterns to encode the aforesaid contexts. Our approach guides long-term trajectory prediction by learning to predict short-term motion patterns. It then extracts sub-goal information from the patterns and aggregates it as social context. We assess our approach across three domains: humans crowds, humans in sports and manned aircraft in terminal airspace, achieving state-of-the-art performance.

The recognition of actions performed by humans and the anticipation of their intentions are important enablers to yield sociable and successful collaboration in human-robot teams. Meanwhile, robots should have the capacity to deal with multiple objectives and constraints, arising from the collaborative task or the human. In this regard, we propose vision techniques to perform human action recognition and image classification, which are integrated into an Augmented Hierarchical Quadratic Programming (AHQP) scheme to hierarchically optimize the robot's reactive behavior and human ergonomics. The proposed framework allows one to intuitively command the robot in space while a task is being executed. The experiments confirm increased human ergonomics and usability, which are fundamental parameters for reducing musculoskeletal diseases and increasing trust in automation.

Recognizing objects in dense clutter accurately plays an important role to a wide variety of robotic manipulation tasks including grasping, packing, rearranging and many others. However, conventional visual recognition models usually miss objects because of the significant occlusion among instances and causes incorrect prediction due to the visual ambiguity with the high object crowdedness. In this paper, we propose an interactive exploration framework called Smart Explorer for recognizing all objects in dense clutters. Our Smart Explorer physically interacts with the clutter to maximize the recognition performance while minimize the number of motions, where the false positives and negatives can be alleviated effectively with the optimal accuracy-efficiency trade-offs. Specifically, we first collect the multi-view RGB-D images of the clutter and reconstruct the corresponding point cloud. By aggregating the instance segmentation of RGB images across views, we acquire the instance-wise point cloud partition of the clutter through which the existed classes and the number of objects for each class are predicted. The pushing actions for effective physical interaction are generated to sizably reduce the recognition uncertainty that consists of the instance segmentation entropy and multi-view object disagreement. Therefore, the optimal accuracy-efficiency trade-off of object recognition in dense clutter is achieved via iterative instance prediction and physical interaction. Extensive experiments demonstrate that our Smart Explorer acquires promising recognition accuracy with only a few actions, which also outperforms the random pushing by a large margin.

Robots using classical control have revolutionised assembly lines where the environment and manipulated objects are restricted and predictable. However, they have proven less effective when the manipulated objects are deformable due to their complex and unpredictable behaviour. The use of tactile sensors and continuous monitoring of tactile feedback is there-fore particularly important for pick-and-place tasks using these materials. This is in part due to the need to use multiple points of contact for the manipulation of deformable objects which can result in slippage with inadequate coordination between manipulators. In this paper, continuous monitoring of tactile feedback, using a liquid metal soft force sensor, for grasping deformable objects is presented. The trained data-driven model distinguishes between successful grasps, slippage and failure during a manipulation task for multiple deformable objects. Slippage could be anticipated before failure occurred using data acquired over a 30 ms period with a greater than 95% accuracy using a random forest classifier. The results were achieved using a single sensor that can be mounted on the fingertips of existing grippers and contributes to the development of an automated pick-and-place process for deformable objects.

This paper presents a novel visual servoing method that controls a robotic manipulator in the configuration space as opposed to the classical vision-based control methods solely focusing on the end effector pose. We first extract the robot's shape from depth images using a skeletonization algorithm and represent it using parametric curves. We then adopt an adaptive visual servoing scheme that estimates the Jacobian online relating the changes of the curve parameters and the joint velocities. The proposed scheme does not only enable controlling a manipulator in the configuration space, but also demonstrates a better transient response while converging to the goal configuration compared to the classical adaptive visual servoing methods. We present simulations and real robot experiments that demonstrate the capabilities of the proposed method and analyze its performance, robustness, and repeatability compared to the classical algorithms.

In this paper, we propose a new brachiation robot, a single-rod brachiation robot. Brachiation is a method of locomotion that makes clever use of gravity and has been tried to apply to robots. Conventional brachiation robots are multiple-pendulum-like robots that mimic a gibbon. Although the multiple-pendulum-like robot can easily change the length of one brachiation step by joints, it has complex structures and generates aperiodic motions such as chaos. In contrast, the single-rod brachiation robot has the advantages of simple structure and the ability to suppress complex multiple-pendulum trajectories. The single-rod brachiation robot has a disadvantage because it is difficult to adjust the distance to the next bar. However, we can solve it by aerial brachiation, which includes an aerial phase before grasping the next bar. Using the actual robot, we showed that the swinging amplitude could be increased by appropriately moving its center of gravity like a trapeze motion. In addition, using this, we achieved continuous brachiation across three bars and brachiation including the aerial phase with a flight distance of 140 mm.

Being able to grasp objects is a fundamental component of most robotic manipulation systems. In this paper, we present a new approach to simultaneously reconstruct a mesh and a dense grasp quality map of an object from a depth image. At the core of our approach is a novel camera-centric object representation called the “object shell” which is composed of an observed “entry image” and a predicted “exit image”. We present an image-to-image residual ConvNet architecture in which the object shell and a grasp-quality map are predicted as separate output channels. The main advantage of the shell representation and the corresponding neural network architecture, ShellGrasp-Net, is that the input-output pixel correspondences in the shell representation are explicitly represented in the architecture. We show that this coupling yields superior generalization capabilities for object reconstruction and accurate grasp quality estimation implicitly considering the object geometry. Our approach yields an efficient dense grasp quality map and an object geometry estimate in a single forward pass. Both of these outputs can be used in a wide range of robotic manipulation applications. With rigorous experimental validation, both in simulation and on a real setup, we show that our shell-based method can be used to generate precise grasps and the associated grasp quality with over 90% accuracy. Diverse grasps computed on shell reconstructions allow the robot to select and execute grasps in cluttered scenes with more than 93% success rate.

Gesture classification and motion speed regression are always two major issues in myoelectrical prosthetic hand research. However, there is little research considering these two issues in conjunction. Some shared EMG feature information in these two processing tasks is promising to improve the performance of prosthetic hand control. In this study, a joint-loss (JL) neural network architecture is proposed to implement gesture classification and motion speed regression problems in parallel by sharing the hidden neural units in the training process and optimizing the joint loss function. We evaluated the proposed control system through motion experiments performed on six participants. The experiment result shows that the classification and regression models can successfully reproduce smooth movement based on EMG measurement with high accuracy. Furthermore, the possibility of clinical application is demonstrated through the online movement of the real prosthetic hand.

Visual localization, which is a key technology to realize the automation of cell manipulation, has been widely studied. Since the depth of field of the microscope is narrow, the planar localization and depth estimation are usually coupled together. At present, most methods adopt the serial working mode of focusing first and then planar localization, but they usually do not have good real-time performance and stability. In this paper, a simultaneous depth estimation and localization network was developed for cell manipulation. The network takes a focused image and a defocus-offset image as inputs, and outputs the defocus in the depth direction and the offset in the plane at the same time after going through defocus-offset information extraction, defocus classification mapping and offset regression mapping. To train and test our network, we also create two datasets: An Adherent Cell dataset and an Injection Micropipette dataset. The experimental results demonstrated that the proposed method achieves the detection of all test samples with a frame rate of more than 40Hz, and the maximum errors of depth estimation and localization are \boldsymbol{2.44\mu m}\boldsymbol{2.44\mu m} and \boldsymbol{0.49\mu m}\boldsymbol{0.49\mu m}, respectively. The proposed method has good stability, which is mainly reflected in its strong generalization ability and anti-noise ability.

While motion planning of locomotion for legged robots has shown great success, motion planning for legged robots with dexterous multi-finger grasping is not mature yet. We present an efficient motion planning framework for simultaneously solving locomotion (e.g., centroidal dynamics), grasping (e.g., patch contact), and contact (e.g., gait) problems. To accelerate the planning process, we propose distributed optimization frameworks based on Alternating Direction Methods of Multipliers (ADMM) to solve the original large-scale Mixed-Integer NonLinear Programming (MINLP). The resulting frameworks use Mixed-Integer Quadratic Programming (MIQP) to solve contact and NonLinear Programming (NLP) to solve nonlinear dynamics, which are more computationally tractable and less sensitive to parameters. Also, we explicitly enforce patch contact constraints from limit surfaces with micro-spine grippers. We demonstrate our proposed framework in the hardware experiments, showing that the multi-limbed robot is able to realize various motions including free-climbing at a slope angle 45° with a much shorter planning time.

Joint estimation of grasped object pose and extrinsic contacts is central to robust and dexterous manipulation. In this paper, we propose a novel state-estimation algorithm that jointly estimates contact location and object pose in 3D using exclusively proprioception and tactile feedback. Our approach leverages two complementary particle filters: one to estimate contact location (CPFGrasp) and another to estimate object poses (SCOPE). We implement and evaluate our approach on real-world single-arm and dual-arm robotic systems. We demonstrate that by bringing two objects into contact, the robots can infer contact location and object poses simultaneously. Our proposed method can be applied to a number of downstream tasks that require accurate pose estimates, such as tool use and assembly. Code and data can be found at https://github.com/MMintLab/scope.

Increasingly volatile markets challenge companies and demand flexible production systems that can be quickly adapted to new conditions. Machine Learning has proven to show significant potential in supporting the human operator during the time-consuming and complex task of robot pro-gramming by identifying relevant parameters of the underlying robot control program. We present a solution to learn these parameters for contact-rich, force-controlled assembly tasks from a simulation using hardware-independent robot skills. We show that successful learning and real-world execution are possible even under process deviation and tolerances utilizing the designed learning system. We present learning skill param-eters as high-level robot control, evaluation and comparison of extensive simulations, and preliminary experiments on a physical robot test-bed. The developed solution approach is evaluated and discussed using the Peg-in-Hole process, a typical benchmark process in force-controlled assembly.

Accurate simulation of soft mechanisms under dynamic actuation is critical for the design of soft robots. We address this gap with our differentiable simulation tool by learning the material parameters of our soft robotic fish. On the example of a soft robotic fish, we demonstrate an experimentally-verified, fast optimization pipeline for learning the material parameters from quasi-static data via differentiable simulation and apply it to the prediction of dynamic performance. Our method identifies physically plausible Young's moduli for various soft silicone elastomers and stiff acetal copolymers used in creation of our three different robotic fish tail designs. We show that our method is compatible with varying internal geometry of the actuators, such as the number of hollow cavities. Our framework allows high fidelity prediction of dynamic behavior for composite bi-morph bending structures in real hardware to millimeter-accuracy and within 3% error normalized to actuator length. We provide a differentiable and robust estimate of the thrust force using a neural network thrust predictor; this estimate allows for accurate modeling of our experimental setup measuring bollard pull. This work presents a prototypical hardware and simulation problem solved using our differentiable framework; the framework can be applied to higher dimensional parameter inference, learning control policies, and computational design due to its differentiable character.