IROS 2017 Papers

In this paper, we provide a new generalized kinematic formulation of the Remote Center of Motion (RCM) constraint for serial robotic arms to be useful for implementation in torque-controlled robots. A minimally invasive surgery assisted by a serial robot could be used for inserting a surgical tool through a desired insertion position, i.e. trocar position. The restricted movement caused by the trocar, i.e. the RCM constraint, must be guaranteed by the robotic system in order to avoid injury to the patient. The generalized formulation, presented in this paper, associates the task-space with the RCM constraint coordinates, which are described by the minimum distance between the surgical tool and the trocar position. In order to simultaneously achieve the surgical tool-tip trajectory and the RCM constraint, two control methods are proposed and compared, one using a projection in the null-space, the other by combining both tasks using a task-space augmentation method. Moreover, when additional lower priority tasks have to be implemented in the null-space of the two higher priority tasks, the kinematic formulation simplifies the calculation of the null-space projectors. Furthermore, unlike the previous studies, the formulation represented can be applied not only in a kinematic control but also in a dynamic control approach. Simulations were conducted to validate effectiveness of the proposed formulation, using a planar 4-DOF system as well as the dynamic model of a Kuka LBR 7 iiwa R800 robot arm.

Modern robotics is gravitating toward increasingly collaborative human robot interaction. Tools such as acceleration policies can naturally support the realization of reactive, adaptive, and compliant robots. These tools require us to model the system dynamics accurately - a difficult task. The fundamental problem remains that simulation and reality diverge-we do not know how to accurately change a robot's state. Thus, recent research on improving inverse dynamics models has been focused on making use of machine learning techniques. Traditional learning techniques train on the actual realized accelerations, instead of the policy's desired accelerations, which is an indirect data source. Here we show how an additional training signal - measured at the desired accelerations - can be derived from a feedback control signal. This effectively creates a second data source for learning inverse dynamics models. Furthermore, we show how both the traditional and this new data source, can be used to train task-specific models of the inverse dynamics, when used independently or combined. We analyze the use of both data sources in simulation and demonstrate its effectiveness on a real-world robotic platform. We show that our system incrementally improves the learned inverse dynamics model, and when using both data sources combined converges more consistently and faster.

State-of-the-art calibration methods typically choose to use a checkerboard as the calibration target for its simplicity and robustness. They however require the complete checkerboard be captured to break symmetry. More recent multi-camera systems such as Google Jump, Jaunt, and camera arrays have limited overlapping field-of-view (FoV) and having all cameras viewing the complete checkerboard is extremely difficult in reality. Tailored patterns such as CALTag [1] introduce new image features within the checker blocks for breaking symmetry but they also break the grid topology. We present a new technique using such patterned calibration targets for a broad range of multi-camera systems. Our key observation is that applying directional gradient filters yields to heterogeneous responses on grid vs. non-grid features: the former are isolated and the latter are highly inter-connected. We therefore apply a simple but highly efficient technique to eliminate non-grid outliers based on connected component analysis and gradient histograms. Finally, we recover the complete grid by approximating each local checkerboard as a parallelogram and imposing the topology constraint. We conduct comprehensive experiments on a number of recent multi-camera systems and our technique significantly outperforms the state-of-the-art in accuracy and robustness.

Activities of daily living (ADLs) are important for quality of life. Robotic assistance offers the opportunity for people with disabilities to perform ADLs on their own. However, when a complex semi-autonomous system provides real-world assistance, occasional anomalies are likely to occur. Robots that can detect, classify and respond appropriately to common anomalies have the potential to provide more effective and safer assistance. We introduce a multimodal execution monitor to detect and classify anomalous executions when robots operate near humans. Our system builds on our past work on multimodal anomaly detection. Our new monitor classifies the type and cause of common anomalies using an artificial neural network. We implemented and evaluated our execution monitor in the context of robot-assisted feeding with a general-purpose mobile manipulator. In our evaluations, our monitor outperformed baseline methods from the literature. It succeeded in detecting 12 common anomalies from 8 able-bodied participants with 83% accuracy and classifying the types and causes of the detected anomalies with 90% and 81% accuracies, respectively. We then performed an in-home evaluation with Henry Evans, a person with severe quadriplegia. With our system, Henry successfully fed himself while the monitor detected, classified the types, and classified the causes of anomalies with 86%, 90%, and 54% accuracy, respectively.

Learning object models in the wild from natural human interactions is an essential ability for robots to perform general tasks. In this paper we present a robocentric multimodal dataset addressing this key challenge. Our dataset focuses on interactions where the user teaches new objects to the robot in various ways. It contains synchronized recordings of visual (3 cameras) and audio data which provide a challenging evaluation framework for different tasks. Additionally, we present an end-to-end system that learns object models using object patches extracted from the recorded natural interactions. Our proposed pipeline follows these steps: (a) recognizing the interaction type, (b) detecting the object that the interaction is focusing on, and (c) learning the models from the extracted data. Our main contribution lies in the steps towards identifying the target object patches of the images. We demonstrate the advantages of combining language and visual features for the interaction recognition and use multiple views to improve the object modelling. Our experimental results show that our dataset is challenging due to occlusions and domain change with respect to typical object learning frameworks. The performance of common out-of-the-box classifiers trained on our data is low. We demonstrate that our algorithm outperforms such baselines.

Fulfillment centers located in densely populated urban areas are an ever-growing need for leading online consumer websites. These urban fulfillment centers have limited land mass and must have innovative solutions to transport goods within the available vertical space. This work presents a Multi-Track Elevator (MTE) System, a competitive solution for rapid access and retrieval of goods in high-rise e-commerce fulfillment centers and warehouses. The MTE System consists of multiple vertical rails connected with angular traverse rails that allow multiple carriages to go up and down without collision. A novel turning point system switches track routes so that several carriages can move across the multiple rails for rapidly accessing many floors and collecting diverse goods. Unlike existing vertical-horizontal grid elevators and rail systems, the Roller-coaster type, self-powered carriages on the MTE system do not have to stop at switching points, but can continually move across the network of rails. This work walks through the architecture of the rail network system and techniques for switching multiple rails, followed by the design of vertical turntables for smooth, continuous rail switching. Finally, outlining the use of a simple route optimization algorithm, diverse elevator systems are compared with respect to total traveling time and distance. A proof-of-concept prototype has been built and is presented.

This paper discusses the time-phased deployment of wireless sensor networks, applied to surveillance areas growing in time. The focus herein is on the planning of the time-efficient delivery of static sensors to their designated nodes, given a network configuration. The novelty of the proposed strategy is in that it determines optimal delivery plans for spatio-temporally constrained static-sensor networks using multi-robot teams. The proposed sensor delivery planning strategy starts with an already determined (optimal) network plan specified by sensor placement locations (i.e., nodes) and deployment times. Thus, the goal at hand is to determine the optimal routes for the robots delivering the sensors to their intended locations just-in-time. The travel routes are, thus, determined to maximize spare time for the robots between the nodes. The problem is similar to the multiple travelling salesperson problem, but, with temporal constraints. Namely, sensors must be delivered to their designated nodes at designated (optimized) times in order to maintain the optimal deployment of the network configuration. Furthermore, the strategy is designed to be adaptive to new information that can become available during the search for the mobile target, allowing for replanning of the sensor network (i.e., new sensors locations and new deployment times). Numerous simulated experiments were conducted to validate the proposed strategy.

Satisfactory operation of civil infrastructure is of critical importance to an economy. In order to maintain performance, infrastructure needs to be properly maintained. Inspecting infrastructure is inherently labor-intensive work and costly. In this paper, we propose a solution to cost-effective infrastructure inspection by developing a multi-functional inspection robot. The robot is equipped with several state-of-the-art non-destructive evaluation (NDE) sensors to perform inspection. The robot is able to perform selected inspection methods in certain areas based on multiple sensor data fusion. With this, the overall inspection time is reduced, which in turn reduces maintenance cost. An inspection framework based on multiple NDE data sensor fusion is proposed. Detailed discussions include robot design, robot navigation and sensor data fusion.

Materials exhibiting variable stiffness properties have great potential for use in the growing field of soft robotics. Soft structural materials allow a robot to fit into enclosed spaces, resist shock and vibration, or even reconfigure its geometry and adapt to various environments. Rigid structural materials on the other hand allow environmental interactions through application of force and load-bearing capabilities. Materials that can be selectively switched between these two extremes could greatly expand the functionality of a robot that requires the properties of both. In this paper, we introduce a conductive epoxy composite that is self-softening through Joule heating via direct application of electrical current. The polymer can then become load-bearing and rigid again after being formed into a new shape. We demonstrate the capabilities of this material by attaching a pneumatic actuator and showing that the resulting variable stiffness device can be softened from its initial rigid state, reshape itself using the pneumatic actuator, then become rigid again and hold this new position without additional power being supplied to the actuator.

Tendon-driven mechanisms used in robotic surgery exhibit strong nonlinearities, particularly a static backlash-like hysteresis, in their motion transmission behavior. In this paper, an extension of a previously developed model is proposed that allows for estimation of angular displacements in multi-DOF tendon-driven devices where special attention is given to the coupling effect between DOFs. The proposed model consists of the conventional coupling matrix and a novel elongation matrix which compensates for the coupled hysteretic effect. The model is applied to the problem of position estimation in three DOFs (one pitch and two grasping DOFs) of a da Vinci® surgical instrument. As a further extension, a preliminary dynamic model is also suggested to deal with high-frequency inputs. According to the experimental results obtained, the proposed quasi-static model can describe the transmission behavior with goodness-of-fit of 76-92 per cent, and the estimates are improved by 35-72 per cent in terms of the RMSE for the proposed dynamic model as compared to the conventional rigid model.

Pedicle drilling is a crucial and high-risk process in spinal surgery. Due to the respiration and cardiac cycle, the position of spine would fluctuate during operations, which result in an increase of the difficulty in state recognition of pedicle drilling. To guarantee the safety and validity, a model-based compensation method is proposed in this paper. To build the empirical model of vertebral motion, vertebral displacement and tidal volume (Tv) signals are collected from volunteers. To rule out disturbances in original signal, FFT and wavelet transform (DWT) are used to process the experimental signals. In order to select the apt basis for different signals, the root mean square error decision-making method (RMSE-DMM) is introduced. When the filtered vertebral displacement signal is obtained, the particle swarm optimization (PSO) algorithm is used to figure out the empirical model. The robot assisted systems (RAS) can easily compensate vertebral fluctuation based on the empirical model. Due to the goal of pedicle drilling is to drill a hole from surface of first cortical layer to the interior of second cortical layer, a new key point recognition algorithm, based on force, proposed in this paper. To verify the effectiveness of compensation and the recognition algorithm, 3 sets of comparison experiments are carried out. And the results of experiment show the compensation method and new key point recognition algorithm perform effectively.

This paper introduces an optimization-based framework for robust multilegged walking motion planning. Previous approaches use fixed gait sequences, and rely on Zero Moment Point (ZMP) to guarantee dynamic stability. While this combination works well on flat ground, it does not generalize to uneven terrain requiring aggressive gait or gait transition. To overcome such difficulties, in this paper, we present an optimization framework, that can plan both the contact location and gait sequence simultaneously in a mixed-integer convex optimization program. Moreover, we rely on the Contact Wrench Cone (CWC) stability criterion, which generalizes the ZMP criterion to uneven terrain with friction cone constraints, and we plan the walking motion together with the angular momentum through a convex optimization program. Our approach is successfully tested on a LittleDog quadruped over simulated scenarios. We show that on the flat ground, our planner generates a periodic gait, same as Central Pattern Generator + ZMP planner; while on uneven terrain, our planner can successfully generate a motion containing different gaits, with a center-of-mass motion that respects the friction cone constraints, which are violated by ZMP planners. This improvement clearly demonstrates the advantage of our approach over traditional planning strategies.

We propose a novel integrator for implementing bilateral constraints in multibody simulation using variational integrator methods. We first construct a variational penalty method, which is used to enforce a constraint. The penalty term is simulated using an asynchronous variational integrator, allowing the penalty part of the system to be simulated using a smaller time step. We compute the Discrete Euler-Lagrange (DEL) equations for an equivalent penalty term with a larger time step and then use this rescaled system in the aforementioned variational penalty method, thereby enforcing the constraints. This enables us to incorporate some of the behavior of a very stiff system, which would only be stable on the small time scale, into the system on the large time scale. The effect is better adherence to the constraints, at a larger time step. We demonstrate the method with a simulation of a chain of rigid bodies. We then discuss the potential applications of the integrator and highlight how the work can be used to better interpret the tuned values of the coefficients used in penalty formulations.

Feedback controller for robotic bipedal walking models can be multi-layered, consisting of low-level continuous-time controller and high-level event-based controller. Stimulated by the success in our preceding study that demonstrates the validity of the l∞-induced norm as an adequate performance measure, we suggest a systematic methodology to design optimal event-based feedback controller for bipedal models walking on rough terrains. More precisely, we first assume that the system is already equipped with a low-level continuous-time feedback controller, capable of stable flat-ground-walking, and then formulate the design problem of the high-level event-based feedback control as the l1-optimal control problem for discrete-time linear systems defined on the linearized Poincarè map. In order to validate the proposed methodology, nonlinear dynamic simulations are conducted with a simple biped model walking on rough terrain. The terrain slope randomly varies at each footstep while the magnitude of slope variation is bounded by some maximum value. Simulation results indicate that the optimal system equipped with the proposed controller can successfully overcome a rough terrain on which the original system could not walk and fall.

In classical robot-camera calibration, a 6D transformation between the camera frame and the local frame of a robot is estimated by first observing a known calibration object from a number of different view points and then finding transformation parameters that minimize the reprojection error. The disadvantage with this is that often not all configurations can be reached by the end-effector, which leads to an inaccurate parameter estimation. Therefore, we propose a more versatile method based on the detection of oriented visual features, in our case AprilTags. From a collected number of such detections during a defined rotation of a joint, we fit a Bingham distribution by maximizing the observation likelihood of the detected orientations. After a tilt and a second rotation, a camera-to-joint transformation can be determined. In experiments with accurate ground truth available, we evaluate our approach in terms of precision and robustness, both for hand-eye/robot-camera and for camera-camera calibration, with classical solutions serving as a baseline.

Deployment of camera and laser based motion estimation systems for controlling platforms operating at high speeds, such as cars or trains, is posing increasingly challenging precision requirements on the temporal calibration of these sensors. In this work, we demonstrate a simple, low-cost system for calibrating any combination of cameras and time of flight LIDARs with respect to the CPU clock (and therefore, also to each other). The newly proposed device is based on widely available off-the-shelf components, such as the Raspberry Pi 3, which is synchronized using the Precision Time Protocol (PTP) with respect to the CPU of the sensor carrying system. The obtained accuracy can be shown to be below 0.1 ms per measurement for LIDARs and below minimal exposure time per image for cameras. It outperforms state-of-the-art approaches also not relying on hardware synchronization by more than a factor of 10 in precision. Moreover, the entire process can be carried out at a high rate allowing the study of how offsets evolve over time. In our analysis, we demonstrate how each building block of the system contributes to this accuracy and validate the obtained results using real-world data.

This paper introduces a leg design method aiming at speed maximization for quadruped robots with two-segment compliant legs, in trotting and bounding. The proposed method is an effort to address the leg design challenge in a holistic way, exploiting the coupling between gait parameters, leg design parameters and hardware constraints, while remaining control scheme independent. Optimal body trajectories and footfalls are derived using a simplified centroidal dynamics model, whereas joint trajectories and torques are computed by a more complex dynamic model, incorporating actuation parameters and constraints. The method is applied using real robot parameters to yield an optimal leg design, validated through a realistic trotting simulation experiment. In this experiment, the robot achieves accelerating motion from stance towards the maximum speed predicted by the method.

We present a novel method for synthesizing collision-free, dynamic locomotion behaviors for legged robots, including jumping, going down a very steep slope, or recovering from a push using the arms of the robot. The approach is automatic and generic: non-gaited motions, comprising arbitrary contact postures can be generated along any environment. At the core of our framework is a new steering method that generates trajectories connecting two states of the robot. These trajectories account for the state-dependent, centroidal dynamic constraints inherent to legged robots. The method, of low dimension, formulated as a Linear Program, is really efficient to compute, and can find an application in various problems related to legged locomotion. By incorporating this steering method into an existing sampling-based contact planner, we propose the first kinodynamic contact planner for legged robots.

Human Swarm Interaction (HSI) is a new field which relates to the effective control of robotic swarms by human operators. The iterature has shown that the control of swarms can become quite complicated. On the other hand, Brain Machine Interfaces (BMI) can offer intuitive control in a plethora of applications where other interfaces alone (e.g. joysticks) are inadequate or impractical, e.g. for people with motor disabilities. There are multiple types of BMI, but most of them rely on the analysis of ElectroEncephaloGraphic (EEG) signals. The authors have previously shown that swarm behaviors elicit specific brain activity on human subjects that observe them. Motivated by this result, in this work, we present preliminary results of a hybrid BMI that combines information from the brain and an external device. An algorithm for extracting information from the frequency domain of EEG signals that allows integration with the manual task of using a joystick is presented. The hybrid interface shows high accuracy and robustness when used as a brain-robot interface. Moreover, it allows for continuous control variables extracted from the EEG signals. Finally, its efficacy is proven across multiple subjects, while its performance is also demonstrated in the real-time control of a swarm of quadrotors.