IROS 2020 论文列表

We address the monocular visual shape servoing problem. This pushes the challenging visual servoing problem one step further from rigid object manipulation towards deformable object manipulation. Explicitly, it implies deforming the object towards a desired shape in 3D space by robots using monocular 2D vision. We specifically concentrate on a scheme capable of controlling large isometric deformations. Two important open subproblems arise for implementing such a scheme. (P1) Since it is concerned with large deformations, perception requires tracking the deformable object's 3D shape from monocular 2D images which is a severely underconstrained problem. (P2) Since rigid robots have fewer degrees of freedom than a deformable object, the shape control becomes underactuated. We propose a template-based shape servoing scheme in which we solve these two problems. The template allows us to both infer the object's shape using an improved Shape-from-Template algorithm and steer the object's deformation by means of the robots' movements. We validate the scheme via simulations and real experiments.

Easy, yet robust long-term localization is still an open topic in research. Existing approaches require either dense maps, expensive sensors, specialized map features or proprietary detectors.We propose using semantic segmentation on a monocular camera to localize directly in a HD map as used for automated driving. This combines lightweight, yet powerful HD maps with the simplicity of monocular vision and the flexibility of neural networks.The major challenges arising from this combination are data association and robustness against misdetections. Association is solved efficiently by applying distance transform on binary per-class images. This provides not only a fast lookup table for a smooth gradient as needed for pose-graph optimization, but also dynamic association by default.A sliding-window pose graph optimization combines single image detections with vehicle odometry, smoothing results and helping overcome even misclassifications in consecutive frames.Evaluation against a highly accurate 6D visual localization shows that our approach can achieve accuracy levels as required for automated driving, being one of the most lightweight and flexible methods to do so.

This paper reports a new continuous 3D loss function for learning depth from monocular images. The dense depth prediction from a monocular image is supervised using sparse LIDAR points, which enables us to leverage available open source datasets with camera-LIDAR sensor suites during training. Currently, accurate and affordable range sensor is not readily available. Stereo cameras and LIDARs measure depth either inaccurately or sparsely/costly. In contrast to the current point-to-point loss evaluation approach, the proposed 3D loss treats point clouds as continuous objects; therefore, it compensates for the lack of dense ground truth depth due to LIDAR's sparsity measurements. We applied the proposed loss in three state-of-the-art monocular depth prediction approaches DORN, BTS, and Monodepth2. Experimental evaluation shows that the proposed loss improves the depth prediction accuracy and produces point-clouds with more consistent 3D geometric structures compared with all tested baselines, implying the benefit of the proposed loss on general depth prediction networks. A video demo of this work is available at https://youtu.be/5HL8BjSAY4Y.

Light-weight camera localization in existing maps is essential for vision-based navigation. Currently, visual and visual-inertial odometry (VO&VIO) techniques are well-developed for state estimation but with inevitable accumulated drifts and pose jumps upon loop closure. To overcome these problems, we propose an efficient monocular camera localization method in prior LiDAR maps using direct 2D-3D line correspondences. To handle the appearance differences and modality gaps between LiDAR point clouds and images, geometric 3D lines are extracted offline from LiDAR maps while robust 2D lines are extracted online from video sequences. With the pose prediction from VIO, we can efficiently obtain coarse 2D-3D line correspondences. Then the camera poses and 2D-3D correspondences are iteratively optimized by minimizing the projection error of correspondences and rejecting outliers. Experimental results on the EurocMav dataset and our collected dataset demonstrate that the proposed method can efficiently estimate camera poses without accumulated drifts or pose jumps in structured environments.

With the advent of service robots in public places (e.g., in airports and shopping malls), understanding socio-psychological interactions between humans and robots is of paramount importance. On the one hand, traditional robotic navigation systems consider humans and robots as moving obstacles and focus on the problem of real-time collision avoidance in Human-Robot Interaction (HRI) using mathematical models. On the other hand, the behavior of a robot has been determined with respect to a human. Parameters for human-human interaction have been assumed and applied to interactions involving robots. One major limitation is the lack of sufficient data for calibration and validation procedures. This paper models, calibrates and validates the socio-psychological interaction of the human in HRIs among crowds. The mathematical model is an extension of the Social Force Model for crowd modelling. The proposed model is calibrated and validated using open source datasets (including uninstructed human trajectories) from the Asia and Pacific Trade Center shopping mall in Osaka (Japan).In summary, the results of the calibration and validation on the multiple HRIs encountered in the datasets show that humans react to a service robot to a higher extend within a larger distance compared to the interaction range towards another human. This microscopic model, calibration and validation framework can be used to simulate HRI between service robots and humans, predict humans' behavior, conduct comparative studies, and gain insights into safe and comfortable human-robot relationships from the human's perspective.

In this paper, we study a wheeled robot with a prismatic extension joint. This allows the robot to build up momentum to perform jumps over obstacles and to swing up to the upright position after the loss of balance. We propose a template model for the class of such two-wheeled jumping robots. This model can be considered as the simplest wheeled-legged system. We provide an analytical derivation of the system dynamics which we use inside a model predictive controller (MPC). We study the behavior of the model and demonstrate highly dynamic motions such as swing-up and jumping. Furthermore, these motions are discovered through optimization from first principles. We evaluate the controller on a variety of tasks and uneven terrains in a simulator.

Current Navigation Among Movable Obstacles (NAMO) algorithms focus on finding a path for the robot that only optimizes the displacement cost of navigating and moving obstacles out of its way. However, in a human environment, this focus may lead the robot to leave the space in a socially inappropriate state that may hamper human activity (i.e. by blocking access to doors, corridors, rooms or objects of interest). In this paper, we tackle this problem of "Social Placement Choice" by building a social occupation costmap, built using only geometrical information. We present how existing NAMO algorithms can be extended by exploiting this new cost map. Then, we show the effectiveness of this approach with simulations, and provide additional evaluation criteria to assess the social acceptability of plans.

Cable-driven joints proved to be an effective solution in a wide variety of applications ranging from medical to industrial fields where light structures, interaction with unstructured and constrained environments and precise motion are required. These requirements are achieved by moving the actuators from joints to the robot chassis. Despite these positive properties a cable-driven robotic arm requires a complex cable routing within the entire structure to transmit motion to all joints. The main effect of this routing is a friction phenomenon which reduces the accuracy of the motion of the robotic device. In this paper a bond-graph approach is presented to model a family of cable-driven joints including a novel friction model that can be easily implemented into a control algorithm to compensate the friction forces induced by the rope sliding into bushings.

Robots and intelligent industrial systems that focus on sorting or inspection of products require end-effectors that can grasp and manipulate the objects surrounding them. The capability of such systems largely depends on their ability to efficiently identify the objects and estimate the forces exerted on them. This paper presents an underactuated, compliant, and lightweight hyper-adaptive robot gripper that can efficiently discriminate between different everyday life objects and estimate the contact forces exerted on them during a single grasp, using vision-based techniques. The hyper-adaptive mechanism consists of an array of movable steel rods that get reconfigured conforming to the geometry of the grasped object. The proposed object identification and force estimation techniques are model-free and do not rely on time consuming object exploration. A series of experiments have been carried out to discriminate between 12 different everyday life objects and estimate the forces exerted on a dynamometer. During each grasp, a series of images are captured that detect the reconfiguration of the hyper-adaptive grasping mechanism. These images are then used by an image processing algorithm to extract the required information about the gripper reconfiguration, classify the object grasped using a Random Forests (RF) classifier, and estimate the amount of force being exerted. The employed RF classifier gives a prediction accuracy of 100%, while the results of the force estimation techniques (Neural Networks, Random Forests, and 3rd order polynomial) range from 94.7% to 99.1%.

In recent years the need for manipulation tasks in the industrial as well as in the service robotics domain that require compliant interaction with the environment rose. Since then, an increased number of publications use a model-driven approach to describe these tasks. High-level tasks and sequences of skills are coordinated to achieve a desired motion for e.g., screwing, polishing, or snap mounting. Even though the awareness of the environment, especially in terms of contact situations, is essential for successful task execution, it is too often neglected or considered insufficiently.In this paper, we present a model-based approach, using domain-specific languages (DSL), that enables the explicit modeling of the environment in terms of contact situations. Decoupling the environment model from the skills, fosters exchangeability and allows the adaptation to different environmental situations. This way, an explicit but non-invasive link is established to the skills, enabling the environment model to provide a context to constrain the execution of the skills. Further, we present a synthesis from the modeled contact situations to a real-time component-based control architecture, which executes the skills subject to the active environmental context. A dual arm yoga mat rolling task is used to show the impact of the environment model on the skill execution.

Despite recent progress in robot learning, it still remains a challenge to program a robot to deal with open-ended object manipulation tasks. One approach that was recently used to autonomously generate a repertoire of diverse skills is a novelty based Quality-Diversity (QD) algorithm. However, as most evolutionary algorithms, QD suffers from sample- inefficiency and, thus, it is challenging to apply it in real-world scenarios. This paper tackles this problem by integrating a neural network that predicts the behavior of the perturbed parameters into a novelty based QD algorithm. In the proposed Model-based Quality-Diversity search (M-QD), the network is trained concurrently to the repertoire and is used to avoid executing unpromising actions in the novelty search process. Furthermore, it is used to adapt the skills of the final repertoire in order to generalize the skills to different scenarios. Our experiments show that enhancing a QD algorithm with such a forward model improves the sample-efficiency and performance of the evolutionary process and the skill adaptation.

Robust estimation of camera motion under the presence of outlier noisevision. Despite existing efforts that focus on detecting motion and scene degeneracies, the best existing approach that builds on Random Consensus Sampling (RANSAC) still has non-negligible failure rate. Since a single failure can lead to the failure of the entire visual simultaneous localization and mapping, it is important to further improve the robust estimation algorithm. We propose a new robust camera motion estimator (RCME) by incorporating two main changes: a model-sample consistency test at the model instantiation step and an inlier set quality test that verifies model-inlier consistency using differential entropy. We have implemented our RCME algorithm and tested it under many public datasets. The results have shown a consistent reduction in failure rate when comparing to the RANSAC-based Gold Standard approach and two recent variations of RANSAC methods.

The development of modern sensitive lightweight robots allows the use of robot arms in numerous new scenarios. Especially in applications where interaction between the robot and an object is desired, e.g. in assembly, conventional purely position-controlled robots fail. Former research has focused, among others, on control methods that center on robot-environment interaction. However, these methods often consider only separate scenarios, as for example a pure force control scenario. The present paper aims to address this drawback and proposes a control framework for robot-environment interaction that allows a wide range of possible interaction types. At the same time, the approach can be used for setpoint generation of position-controlled robot arms, where no interaction takes place. Thus, switching between different controller types for specific interaction kinds is not necessary. This versatility is achieved by a model predictive control-based framework which allows trajectory following control of joint or end-effector position as well as of forces for compliant or rigid robot-environment interactions. For this purpose, the robot motion is predicted by an approximated dynamic model and the force behavior by an interaction model. The characteristics of the approach are discussed on the basis of two scenarios on a lightweight robot.

In fields such as minimally invasive surgery, effective control strategies are needed to guarantee safety and accuracy of the surgical task. Mechanical designs and actuation schemes have inevitable limitations such as backlash and joint limits. Moreover, surgical robots need to operate in narrow pathways, which may give rise to additional environmental constraints. Therefore, the control strategies must be capable of satisfying the desired motion trajectories and the imposed constraints. Model Predictive Control (MPC) has proven effective for this purpose, allowing to solve an optimal problem by taking into consideration the evolution of the system states, cost function, and constraints over time. The high nonlinearities in tendon-driven systems, adopted in many surgical robots, are difficult to be modelled analytically. In this work, we use a model learning approach for the dynamics of tendon-driven robots. The dynamic model is then employed to impose constraints on the torques of the robot under consideration and solve an optimal constrained control problem for trajectory tracking by using MPC. To assess the capabilities of the proposed framework, both simulated and real world experiments have been conducted.

Soft links and actuators are nowadays emerging technologies aiming to overcome some problems in robotics such as weight, cost or human interaction. However, the nonlinear nature of their elements can make their characterization challenging and hinder the use of standard control engineering tools. In this paper, we explore different state-of-the-art identification methods for the soft neck, in order to find a reliable plant model. Even though the neck has three Degrees of freedom, in this work we only consider the planar deflection of the link as a starting point for future analysis. Given the nonlinear nature of the soft neck, we consider two identification strategies, i.e., set membership, which is a data driven, nonlinear and nonparametric identification strategy, and Recursive Least Squares at selected linearization points. A neural network identification is also given for comparison purposes. Results show that the explored methods offer a suitable alternative to identify the dynamics of the neck that allows their implementation for simulation and future control.

This work presents a practical method of obtaining a dynamic system model for small omnidirectional aquatic vehicles. The models produced can be used to improve vehicle localisation, aid in the design or tuning of control systems and facilitate the development of simulated environments. The use of a dynamic model for onboard real-time velocity prediction is of particular importance for aquatic vehicles because, unlike ground vehicles, fast and direct measurement of velocity using encoders is not possible. Previous work on model identification of aquatic vehicles has focused on large vessels that are typically underactuated and have low controllability in the sway direction. In this paper it is demonstrated that the procedure for identifying the model coefficients can be performed quickly, without specialist equipment and using only onboard sensors. This is of key importance because the dynamic model coefficients will change with the payload. Two different thrust allocation schemes are tested, one of which is a known method and another is proposed here. Validation tests are performed and the models are shown to be suitable for their intended applications. Significant reduction in model error is demonstrated using the novel thrust allocation method that is designed to avoid deadbands in the thruster responses.

Real-time object detection in videos using lightweight hardware is a crucial component of many robotic tasks. Detectors using different modalities and with varying computational complexities offer different trade-offs. One option is to have a very lightweight model that can predict from all modalities at once for each frame. However, in some situations (e.g., in static scenes) it might be better to have a more complex but more accurate model and to extrapolate from previous predictions for the frames coming in at processing time. We formulate this task as a sequential decision making problem and use reinforcement learning (RL) to generate a policy that decides from the RGB input which detector out of a portfolio of different object detectors to take for the next prediction. The objective of the RL agent is to maximize the accuracy of the predictions per image. We evaluate the approach on the Waymo Open Dataset and show that it exceeds the performance of each single detector.

In this paper, we study the localization problem under non-Gaussian noise. In particular, we consider systems that can be represented by a state transition and a measurement component. The state transition indicates how the system evolves given a control variable. The measurement component compares, for a given state, the received and predicted measurements. Here we consider a radio based range sensor which is the primary source of non-Gaussian noise in the system. We solve the problem using a MHE (Maximum Horizon Estimator) with a correntropy similarity metric. Given a time window, the MHE seeks the best set of states that explains the system for the received measurements. Moreover, the main advantage of a MHE is that it allows the re-estimation of past states. Additionally, the correntropy is a similarity metric that, given the amount of error in the estimation, behaves as L2, L1 or L0 norms and has been successfully used in many applications under non-Gaussian noise. We evaluate our proposed method using both simulated and real data. The results show that correntropy is able to work well in comparison with other methods in presence of impulsive noise.

We present a decision-theoretic model and robot system that interprets multimodal human communication to disambiguate item references by asking questions via a mixed reality (MR) interface. Existing approaches have either chosen to use physical behaviors, like pointing and eye gaze, or virtual behaviors, like mixed reality. However, there is a gap of research on how MR compares to physical actions for reducing robot uncertainty. We test the hypothesis that virtual deictic gestures are better for human-robot interaction (HRI) than physical behaviors. To test this hypothesis, we propose the Physio-Virtual Deixis Partially Observable Markov Decision Process (PVD-POMDP), which interprets multimodal observations (speech, eye gaze, and pointing gestures) from the human and decides when and how to ask questions (either via physical or virtual deictic gestures) in order to recover from failure states and cope with sensor noise. We conducted a between-subjects user study with 80 participants distributed across three conditions of robot communication: no feedback control, physical feedback, and MR feedback. We tested performance of each condition with objective measures (accuracy, time), as well as evaluated user experience with subjective measures (usability, trust, workload). We found the MR feedback condition was 10% more accurate than the physical condition and a speedup of 160%. We also found that the feedback conditions significantly outperformed the no feedback condition in all subjective metrics.