IROS 2018 论文列表

Visual tracking plays an important role in unmanned systems. In many cases, the system needs to keep track of targets it has never seen before, and the only training sample available is the specified object in the initial frame. In this paper, we propose a deep architecture called adversarial transfer networks (ATNet), which aims to make well use of offline video training data and solve the problem of lacking training samples in visual tracking. Different from most existing trackers which neglect significant differences between videos and gulp the training data all together, our method utilizes the special nature of tracking problem and concentrates on transferring domain-specific information across similar tracking tasks. We first propose an efficient way to select a training video that is most similar to online tracking task and regard it as source domain. With the labeled data in the selected source domain, we apply adversarial transfer learning to make the feature distribution of source-domain samples and target-domain samples as similar as possible. Therefore, the transferred source-domain samples can provide various possible appearance of tracked target for training and boost the tracking performance. Experimental results on three OTB tracking benchmarks show that our method outperforms the state-of-the-art trackers in both accuracy and robustness.

The paper proposes an on-line monitoring framework for continuous real-time safety/security in learning-based control systems (specifically application to a unmanned ground vehicle). We monitor validity of mappings from sensor inputs to actuator commands, controller-focused anomaly detection (CFAM), and from actuator commands to sensor inputs, system-focused anomaly detection (SFAM). CFAM is an image conditioned energy based generative adversarial network (EBGAN) in which the energy based discriminator distinguishes between proper and anomalous actuator commands. SFAM is based on an action condition video prediction framework to detect anomalies between predicted and observed temporal evolution of sensor data. We demonstrate the effectiveness of the approach on our autonomous ground vehicle for indoor environments and on Udacity dataset for outdoor environments.

We want to enable the robot to reorient its feet in order to face its direction of motion. Model Predictive Control schemes for biped walking usually assume fixed feet rotation since adapting them online leads to a nonlinear problem. Nonlinear solvers do not guarantee the satisfaction of nonlinear constraints at every iterate and this can be problematic for the real-time operation of robots. We propose to define safe linear constraints that are always inside the intersection of the nonlinear constraints. We make simulations of the robot walking on a crowd and compare the performance of the proposed method with respect to the original nonlinear problem solved as a Sequential Quadratic Program.

To perform a home service task through cooperation with a human in a real environment, a robot needs to deal with the environmental changes and accordingly plan appropriate behavior sequence. For this purpose, in this paper, we propose an adaptive task planner which is based on memory and reasoning. A robot perceives user behaviors and objects using an RGB-depth and thermal sensor. The robot stores a temporal sequence of behaviors for performing a task in its episodic memory that is realized by a sequence to sequence network. When the user command is given, the episodic memory is used to retrieve the behavior sequence to carry out the command. On the other hand, when the robot perceives user behaviors, the robot postpones its behavior till his/her behavior is stopped. Once stopped, the episodic memory retrieves the behavior sequence to conduct a task that the user has intended. A task scheduler schedules the behavior sequence from the memory and sends it to an internal simulator. The internal simulator confirms the behavior sequence to be executable and then if executable, it sends the next executable behavior to the execution module. If a behavior fails in the internal simulation test, fast forward planner generates an alternative behavior sequence to resolve the failed behavior problem. The effectiveness and applicability of the proposed planner is demonstrated by a wheel-based humanoid robot.

Nine degrees of freedom (DOF) inertial measurement units (IMUs) comprised of three-axis magnetometers, three-axis accelerometers, and three-axis angular rate sensors are commonly used in attitude and heading reference systems (AHRSs). Two classes of AHRSs exist: systems that estimate true-North heading and systems that estimate magnetic-North heading. True-North heading AHRSs require high-end angular rate sensors which are sensitive enough to dynamically estimate Earth-rate (typically fiber-optic and ring-laser gyros), while magnetic-North AHRSs employ gyros that are not sensitive enough to dynamically estimate Earth-rate (i.e. all MEMS gyros). Thus, magnetic-North AHRSs employ magnetometers for estimating heading. This paper will focus on this class of magnetic-North AHRSs. These systems fuse IMU measurements to generate estimates of the instrument's roll, pitch, and magnetic heading. However, their accuracy is limited by sensor measurement bias that is unknown a priori. Hence, accurate sensor bias estimation and compensation is essential for true attitude estimation. This paper reports a novel adaptive sensor bias observer for sensor measurement biases in 9-DOF IMUs. The algorithm requires smaller angular movements of the instrument than other reported sensor bias calibration methods, does not require a priori knowledge of local fields like the local magnetic field or the local gravity vector, and does not require knowledge of the attitude of the instrument. Stability proofs, preliminary simulations, and a full-scale vehicle experimental evaluation are reported.

The predictive functions that permit humans to infer their body state by sensorimotor integration are critical to perform safe interaction in complex environments. These functions are adaptive and robust to non-linear actuators and noisy sensory information. This paper introduces a computational perceptual model based on predictive processing that enables any multisensory robot to learn, infer and update its body configuration when using arbitrary sensors with Gaussian additive noise. The proposed method integrates different sources of information (tactile, visual and proprioceptive) to drive the robot belief to its current body configuration. The motivation is to provide robots with the embodied perception needed for self-calibration and safe physical human-robot interaction. We formulate body learning as obtaining the forward model that encodes the sensor values depending on the body variables, and we solve it by Gaussian process regression. We model body estimation as minimizing the discrepancy between the robot body configuration belief and the observed posterior. We minimize the variational free energy using the sensory prediction errors (sensed vs expected). In order to evaluate the model we test it on a real multi-sensory robotic arm. We show how different sensor modalities contributions, included as additive errors, improve the refinement of the body estimation and how the system adapts itself to provide the most plausible solution even when injecting strong sensory visuo-tactile perturbations. We further analyse the reliability of the model when different sensor modalities are disabled. This provides grounded evidence about the correctness of the perceptual model and shows how the robot estimates and adjusts its body configuration just by means of sensory information.

This paper investigates the straight path following problem for a class of underactuated bio-inspired snake robots on ground with unknown and varied friction coefficients. Existing works usually design control input requiring the exact values of these friction coefficients, which however rely on the specific operating terrain and may not always be known a priori. By virtue of backstepping technique, we present a novel adaptive controller that can compensate for unknown and varied friction coefficients in real-time. Moreover, it is proved via LaSalle-Yoshizawa theorem that the path following errors converge to zero asymptotically and all the parameter estimates are bounded. Simulations and experiments on an 8-link snake robot are carried out to illustrate the effectiveness of the proposed controller.

Robots and automated systems are increasingly being introduced to unknown and dynamic environments where they are required to handle disturbances, unmodeled dynamics, and parametric uncertainties. Robust and adaptive control strategies are required to achieve high performance in these dynamic environments. In this paper, we propose a novel adaptive model predictive controller that combines model predictive control (MPC) with an underlying L1 adaptive controller to improve trajectory tracking of a system subject to unknown and changing disturbances. The L1 adaptive controller forces the system to behave in a predefined way, as specified by a reference model. A higher-level model predictive controller then uses this reference model to calculate the optimal reference input based on a cost function, while taking into account input and state constraints. We focus on the experimental validation of the proposed approach and demonstrate its effectiveness in experiments on a quadrotor. We show that the proposed approach has a lower trajectory tracking error compared to non-predictive, adaptive approaches and a predictive, nonadaptive approach, even when external wind disturbances are applied.

Interaction of socially assistive robots with users is based on social cues coming from different interaction modalities, such as speech or gestures. However, using all modalities at all times may be inefficient as it can overload the user with redundant information and increase the task completion time. Additionally, users may favor certain modalities over the other as a result of their disability or personal preference. In this paper, we propose an Adaptive Modality Selection (AMS) algorithm that chooses modalities depending on the state of the user and the environment, as well as user preferences. The variables that describe the environment and the user state are defined as resources, and we posit that modalities are successful if certain resources possess specific values during their use. Besides the resources, the proposed algorithm takes into account user preferences which it learns while interacting with users. We tested our algorithm in simulations, and we implemented it on a robotic system that provides cognitive training, specifically Sequential memory exercises. Experimental results show that it is possible to use only a subset of available modalities without compromising the interaction. Moreover, we see a trend for users to perform better when interacting with a system with implemented AMS algorithm.

This paper presents an adaptive knee-joint based Functional Electrical Stimulation (FES)method to correct the foot drop of paretic patients during swing phase. The rationale behind the adaptive FES is to amplify dorsiflexor stimulation in the late swing when it is most needed in order to face the increased plantar flexor co-contraction as gastrocnemius muscles are stretched by knee re-extension. To accurately detect the swing phase (i.e., toes off (TO)and initial contact (IC)), a novel algorithm is proposed by using a foot-mounted inertial measurement unit (IMU). The proposed strategy is verified by experiments conducted with three healthy subjects and three paretic patients. The experimental results show that highly accurate detection of TO/I C can be achieved under different walking speeds and foot contact conditions (normal and abnormal gaits). The clinical experimental results with paretic patients also reveal that similar effects on ankle dorsiflexion can be observed during mid and late swing using the proposed adaptive FES with respect to the classical FES method, while the adaptive FES used lower stimulation intensity.

State-of-the-art monocular dense mapping methods usually divide the image sequence into several separate multi-view stereo problems thus have limited utilization of the information in multi-baseline observations and sequential depth estimations. In this paper, two core contributions are proposed to improve the mapping performance by exploiting the information. The first is an adaptive baseline matching cost computation that uses the sequential input images to provide each pixel with wide-baseline observations. The second is a frame-to-frame propagated depth filter which integrates the sequential depth estimation of the same physical point in a robust probabilistic manner. Two contributions are integrated into a monocular dense mapping system that generates the depth maps in real-time for both pinhole and fisheye cameras. Our system is fully parallelized and can run at more than 25 fps on a Nvidia Jetson TX2. We compare our work with state-of-the-art methods on the public dataset. Onboard UAV mapping and handhold experiments are also used to demonstrate the performance of our method. For the benefit of the community, we make the implementation open source.

Autonomous grasp selection for robot pick-and-place applications makes use of either empirical methods leveraging object databases, which generate grasps for specific objects at the initial cost of modeling effort, or analytical methods, which generalize to novel objects but fail on object subsets that require specific grasping strategies not captured by the algorithm. We introduce a grasp selection algorithm that ranks grasp candidates with a set of grasp metrics augmented with object features, creating an approach that adapts its strategies based on user-specified grasp preferences. We formulate grasp selection as a pairwise ranking problem, which significantly reduces data collection compared to traditional grasp ranking methods and generalizes to novel objects. Our approach outperforms a state-of-the-art grasp calculation baseline and a pointwise ranking formulation of the same problem.

This paper presents a control architecture for an underwater vehicle-manipulator system (UVMS) to enable simultaneous tracking of end-effector configuration and contact force during floating-base manipulation. The main feature of the architecture is its combination of a task-priority (TP) kinematic control algorithm with a custom force control strategy, based on impedance (admittance) control. The TP algorithm used in the work includes recent treatment of equality and inequality tasks as well as original concepts to handle operation in singular configurations of the system. In the force control part the impedance concept is extended to allow for direct control over the value of exerted force and torque. Additional feed-forward signal is used to ensure stable contact. The performance of the control architecture is demonstrated by experiments in a test tank, with GIRONA500 I-AUV performing pipe inspection.

This paper deals with a new dynamic formulation of parallel manipulators incorporating the actuator and friction dynamics to be utilized in control. A model-based controller, PD with computed feedforward, is implemented for a parallel robot taking into consideration the formulated dynamics. The motivation behind this contribution is to enhance the control performance by compensating the unfavourable nonlinearities abundant extensively in PKMs. Those nonlinearities may increase considerably when operating at high-speed motions. The proposed feedforward part relies on the reference trajectories instead of the measured ones improving the control performance and the computational efforts. To validate our contribution, real-time experiments are conducted on a four degree-of-freedom parallel robot named VELOCE in different operating conditions.

A reliable estimation of 3D parameters is a must for several applications like planning and control, in which is included Image-Based Visual Servoing. This control scheme depends directly on 3D parameters, e.g. depth of points, and/or depth and direction of 3D straight lines. Recently, a framework for Active Structure-from-Motion was proposed, addressing the former feature type. However, straight lines were not addressed. These are 1D objects, which allow for more robust detection, and tracking. In this work, the problem of Active Structure-from-Motion for 3D straight lines is addressed. An explicit representation of these features is presented, and a change of variables is proposed. The latter allows the dynamics of the line to respect the conditions for observability of the framework. A control law is used with the purpose of keeping the control effort reasonable, while achieving a desired convergence rate. The approach is validated first in simulation for a single line, and second using a real robot setup. The latter set of experiments are conducted first for a single line, and then for three lines.

3D radiation source localization is a common task across applications such as decommissioning, disaster response, and security, but traditional count-based sensors struggle to efficiently disambiguate between symmetries in sensor, source, and environment configurations. Recent works have demonstrated successful passive source localization using a bearing sensor called the Compton gamma camera that can image radiation. This paper first presents an approach to mapping the spatial distribution of radiation with a gamma camera to estimate source locations. An active source localization framework is then developed that greedily selects new waypoints that maximize the Fisher Information provided by the camera's range and bearing observations for source localization. Finally the common assumption of a static step size in between waypoints is relaxed to allow step sizes to adapt online to the observed information. The proposed radiation mapping approach is evaluated in 5×4 m2 and 14×6 m2 laboratory environments, where multiple point sources were localized to within an average of 0.26 m or 0.6% of the environment dimensions. The active source localization approach is evaluated in simulation and an adaptive step size yields a 27% decrease in the localization time and a 16% decrease in the distance traveled to localize a source in a 15×15×15 m3 environment.

We study the problem of learning a navigation policy for a robot to actively search for an object of interest in an indoor environment solely from its visual inputs. While scene-driven visual navigation has been widely studied, prior efforts on learning navigation policies for robots to find objects are limited. The problem is often more challenging than target scene finding as the target objects can be very small in the view and can be in an arbitrary pose. We approach the problem from an active perceiver perspective, and propose a novel framework that integrates a deep neural network based object recognition module and a deep reinforcement learning based action prediction mechanism. To validate our method, we conduct experiments on both a simulation dataset (AI2-THOR)and a real-world environment with a physical robot. We further propose a new decaying reward function to learn the control policy specific to the object searching task. Experimental results validate the efficacy of our method, which outperforms competing methods in both average trajectory length and success rate.

The objective of this work is to augment the basic abilities of a robot by learning to use new sensorimotor primitives to enable the solution of complex long-horizon problems. Solving long-horizon problems in complex domains requires flexible generative planning that can combine primitive abilities in novel combinations to solve problems as they arise in the world. In order to plan to combine primitive actions, we must have models of the preconditions and effects of those actions: under what circumstances will executing this primitive achieve some particular effect in the world? We use, and develop novel improvements on, state-of-the-art methods for active learning and sampling. We use Gaussian process methods for learning the conditions of operator effectiveness from small numbers of expensive training examples collected by experimentation on a robot. We develop adaptive sampling methods for generating diverse elements of continuous sets (such as robot configurations and object poses) during planning for solving a new task, so that planning is as efficient as possible. We demonstrate these methods in an integrated system, combining newly learned models with an efficient continuous-space robot task and motion planner to learn to solve long horizon problems more efficiently than was previously possible.

Robots can rapidly acquire new skills from demonstrations. However, during generalisation of skills or transitioning across fundamentally different skills, it is unclear whether the robot has the necessary knowledge to perform the task. Failing to detect missing information often leads to abrupt movements or to collisions with the environment. Active learning can quantify the uncertainty of performing the task and, in general, locate regions of missing information. We introduce a novel algorithm for active learning and demonstrate its utility for generating smooth trajectories. Our approach is based on deep generative models and metric learning in latent spaces. It relies on the Jacobian of the likelihood to detect non-smooth transitions in the latent space, i.e., transitions that lead to abrupt changes in the movement of the robot. When non-smooth transitions are detected, our algorithm asks for an additional demonstration from that specific region. The newly acquired knowledge modifies the data manifold and allows for learning a latent representation for generating smooth movements. We demonstrate the efficacy of our approach on generalising elementary skills, transitioning across different skills, and implicitly avoiding collisions with the environment. For our experiments, we use a simulated pendulum where we observe its motion from images and a 7-DoF anthropomorphic arm.