IROS 2018 Papers

Lung cancer is the deadliest form of cancer, and early diagnosis is critical to favorable survival rates. Definitive diagnosis of lung cancer typically requires needle biopsy. Common lung nodule biopsy approaches either carry significant risk or are incapable of accessing large regions of the lung, such as in the periphery. Deploying a steerable needle from a bronchoscope and steering through the lung allows for safe biopsy while improving the accessibility of lung nodules in the lung periphery. In this work, we present a method for extracting a cost map automatically from pulmonary CT images, and utilizing the cost map to efficiently plan safe motions for a steerable needle through the lung. The cost map encodes obstacles that should be avoided, such as the lung pleura, bronchial tubes, and large blood vessels, and additionally formulates a cost for the rest of the lung which corresponds to an approximate likelihood that a blood vessel exists at each location in the anatomy. We then present a motion planning approach that utilizes the cost map to generate paths that minimize accumulated cost while safely reaching a goal location in the lung.

Depth cameras have accelerated research in many areas of computer vision. Most triangulation-based depth cameras, whether structured light systems like the Kinect or active (assisted) stereo systems, are based on the principle of stereo matching. Depth from stereo is an active research topic dating back 30 years. Despite recent advances, algorithms usually trade-off accuracy for speed. In particular, efficient methods rely on fronto-parallel assumptions to reduce the search space and keep computation low. We present SOS (Slanted O(1) Stereo), the first algorithm capable of leveraging slanted support windows without sacrificing speed or accuracy. We use an active stereo configuration, where an illuminator textures the scene. Under this setting, local methods - such as PatchMatch Stereo - obtain state of the art results by jointly estimating disparities and slant, but at a large computational cost. We observe that these methods typically exploit local smoothness to simplify their initialization strategies. Our key insight is that local smoothness can in fact be used to amortize the computation not only within initialization, but across the entire stereo pipeline. Building on these insights, we propose a novel hierarchical initialization that is able to efficiently perform search over disparity and slants. We then show how this structure can be leveraged to provide high quality depth maps. Extensive quantitative evaluations demonstrate that the proposed technique yields significantly more precise results than current state of the art, but at a fraction of the computational cost. Our prototype implementation runs at 4000 fps on modern GPU architectures.

This paper presents a rehabilitation wearable exoskeleton for wrist joint with two degrees of freedom (DOF), flexion-extension and adduction-abduction (radial and ulnar deviation), actuated with Shape Memory Alloy (SMA) based actuators. Thanks to this type of actuators, the proposed device presents a very light weight and noiseless operation, in comparison with similar devices. The preliminary results obtained over real tests with the wrist exoskeleton are presented. This prototype demonstrates that SMA actuator technology is a viable alternative when investigating possible improvement of rehabilitation robotic devices in terms of weight, size and cost.

We report on the development of a motion generation strategy that allows the robot to transit from one stable running motion to another in one stride by actively changing leg stiffness in real time. Stable motion of the robot is generated based on the stable fixed-point trajectories of the spring-loaded inverted pendulum (SLIP) model. While the transition of the ordinary SLIP model with fixed parameters gradually converges if stable, a robot that uses force control to simulate the passive spring of the SLIP can actively modulate leg-spring stiffness. This enables the robot to switch instantly to another fixed-point trajectory of the SLIP model without going through multi-stride transition. The proposed method is implemented on a leg-wheel transformable robot, TurboQuad, and is evaluated experimentally. The results confirm that the robot can successfully transit between different fixed-point trajectories.

We study the labeled multi-robot path planning problem in continuous 2D and 3D domains in the absence of obstacles where robots must not collide with each other. For an arbitrary number of robots in arbitrary initial and goal arrangements, we derive a polynomial time, complete algorithm that produces solutions with constant-factor optimality guarantees on both makespan and distance optimality, in expectation, under the assumption that the robot labels are uniformly randomly distributed. Our algorithm only requires a small constant-factor expansion of the initial and goal configuration footprints for solving the problem, i.e., the problem can be solved in a fairly small bounded region. Beside theoretical guarantees, we present a thorough computational evaluation of the proposed solution. In addition to the baseline implementation, adapting an effective (but non-polynomial time) routing subroutine, we also provide a highly efficient implementation that quickly computes near-optimal solutions. Hardware experiments on the microMVP platform composed of non-holonomic robots confirms the practical applicability of our algorithmic pipeline.

Industrial robots are increasingly used in various applications where the robot accuracy becomes very important, hence calibrations of the robot's kinematic parameters and the measurement system's extrinsic parameters are required. However, the existing calibration approaches are either too cumbersome or require another expensive external measurement system such as laser tracker or measurement spinarm. In this paper, we propose SCALAR, a calibration method to simultaneously improve the kinematic parameters of a 6-DoF robot and the extrinsic parameters of a 2D Laser Range Finder (LRF) that is attached to the robot. Three flat planes are placed around the robot, and for each plane the robot moves to several poses such that the LRF's ray intersect the respective plane. Geometric planar constraints are then used to optimize the calibration parameters using Levenberg-Marquardt nonlinear optimization algorithm. We demonstrate through simulations that SCALAR can reduce the average position and orientation errors of the robot system from 14.6 mm and 4.05° to 0.09 mm and 0.02°.

The regrasping function of a robotic hand and arm has been investigated by many studies. Dynamic regrasping is performed by accelerating objects and it has the advantage of being able to perform the regrasp function at high speed. However, the difficulty of increasing the success rate is a persistent problem. In this study, we aimed to realize this continuous high-speed operation by increasing the success rate of the regrasping function. The handling of the Rubik's cube was used as the specific task to be performed. The action that was required to handle the Rubik's cube consisted of two types of regrasping motion and one type of one-face turn motion. In this study, a Rubik's cube was placed in a plane and manipulated by combining these three types of motion. Continuous operation was realized with a robotic hand and high-speed vision by utilizing environmental constraints in order to minimize the error. As a result, we succeeded 3 times in turning and regrasping in 1 s. Additionally, we were able to succeed 30 times in turning and regrasping in 10 s, with a success rate of 70%.

The use of snake-like robots for surgery is a popular choice for intra-luminal procedures. In practice, the requirements for strength, flexibility and accuracy are difficult to be satisfied simultaneously. This paper presents a computational approach for optimizing the design of a snake-like robot using serial rolling-joints and tendons as the base architecture. The method optimizes the design in terms of joint angle range and tendon placement to prevent the tendons and joints from colliding during bending motion. The resulting optimized joints were manufactured using 3D printing. The robot was characterized in terms of workspace, dexterity, precision and manipulation forces. The results show a repeatability as low as 0.9mm and manipulation forces of up to 5.6N.

Comparing to IK (Inverse Kinematics) based motion control, joint-space motion control is more advantageous in terms of not being restricted by kinematics singularity problem. In this paper, we start with SIMBICON (Simple Biped Locomotion Control) based controller, a joint-space motion control method, extend it for enhancing walking's robustness and versatility. We propose a motion optimization method considering walking robustness, desired walking velocity and energy efficient minimization for walking motion generation. This method enables us to achieve human-like walking motion, which has stretched-knee posture and robust to large push disturbances. We also apply our proposed method to a life-sized biped robot and validate its effectiveness with push recovery and walking on unknown debris experiments.

Self-assembly is the aggregation of simple parts into complex patterns as frequently observed in nature. Following this inspiration, creating programmable systems of self-assembly that achieve similar complexity and robustness with robots is challenging. As a role model we pick the growth of natural plants that adapts to environmental conditions and is robust enough to withstand disturbances such as changes due to dynamic environments and cut parts. We program a robot swarm to self-assemble into tree-like shapes and to adapt efficiently to the environment. Our approach is inspired by the vascular morphogenesis of plants, the patterned formation of vascular tissue to transport fluids and nutrients internally. The aggregated robots establish an internal network of resource sharing, allowing them to make rational decisions collectively about where to add and where to remove robots. As a result, the growth is adaptive to an environmental feature (here, light) and robust to changes in a dynamic environment. The robot swarm is able to self-repair by regrowing lost parts. We successfully validate and benchmark our approach in a number of robot swarm experiments showing adaptivity, robustness, and self-repair.

The here presented flying system uses two pairs of wide-angle stereo cameras and maps a large area of interest in a short amount of time. We present a multicopter system equipped with two pairs of wide-angle stereo cameras and an inertial measurement unit (IMU) for robust visual-inertial navigation and time-efficient omni-directional 3D mapping. The four cameras cover a 240 degree stereo field of view (FOV) vertically, which makes the system also suitable for cramped and confined environments like caves. In our approach, we synthesize eight virtual pinhole cameras from four wide-angle cameras. Each of the resulting four synthesized pinhole stereo systems provides input to an independent visual odometry (VO). Subsequently, the four individual motion estimates are fused with data from an IMU, based on their consistency with the state estimation. We describe the configuration and image processing of the vision system as well as the sensor fusion and mapping pipeline on board the MAV. We demonstrate the robustness of our multi-VO approach for visual-inertial navigation and present results of a 3D-mapping experiment.

A fundamental problem of non-linear state estimation in robotics is the violation of assumptions about the sensors' error distribution. State of the art approaches reduce the impact of these violations with robust cost functions or predefined non-Gaussian error models. Both require extensive parameter tuning and fail if the sensors' error characteristic changes over time, due to environmental changes, ageing or sensor malfunctions. We demonstrate how the error distribution itself can be part of the state estimation process. Based on an efficient approximation of a Gaussian mixture, we optimize the sensor model simultaneously during the standard state estimation. Due to an implicit expectation-maximization approach, we achieve a fast convergence without prior knowledge of the true distribution parameters. We implement this self-tuning algorithm in a least-squares optimization framework and demonstrate its real time capability on a real world dataset for satellite localization of a driving vehicle. The resulting estimation quality is superior to previous robust algorithms.

Learning from demonstration (LfD) has enabled robots to rapidly gain new skills and capabilities by leveraging examples provided by novice human operators. While effective, this training mechanism presents the potential for sub-optimal demonstrations to negatively impact performance due to unintentional operator error. In this work we introduce Concept Constrained Learning from Demonstration (CC-LfD), a novel algorithm for robust skill learning and skill repair that incorporates annotations of conceptually-grounded constraints (in the form of planning predicates) during live demonstrations into the LfD process. Through our evaluation, we show that CC-LfD can be used to quickly repair skills with as little as a single annotated demonstration without the need to identify and remove low-quality demonstrations. We also provide evidence for potential applications to transfer learning, whereby constraints can be used to adapt demonstrations from a related task to achieve proficiency with few new demonstrations required.

Plant phenotyping is the measurement of observable plant traits. Current methods for phenotyping in the field are labour intensive and error prone. High throughput plant phenotyping in an automated and noninvasive manner is crucial to accelerating plant breeding methods. Occlusions and non-ideal sensing conditions is a major problem for high throughput plant phenotyping with most state-of-the-art 3D phenotyping algorithms relying heavily on heuristics or hand-tuned parameters. To address this problem, we present a novel model-based optimization approach for estimating plant physical traits from plant units called phytomers. The proposed approach involves sampling parameterized 3D plant models from an underlying probability distribution. It then optimizes, making the mass of this probability distribution approach true parameters of the model. Reformulating the phenotyping objective as a search in the space of plant models lets us reason about the plant structure in a holistic manner without having to rely on hand-tuned parameters. This makes our approach robust to noise and occlusions as frequently encountered in real world environments. We evaluate our approach for plant units taken across simulated, greenhouse and field environments. This work furthers field-based robotic phenotyping capabilities paving the way for plant biologists to study the coupled effect of genetics and environment on improving crop yields.

Invariant representations of demonstrated motion trajectories provide context-independent motion models that can be used in motion recognition and generalization applications such as robot programming by demonstration. In practice, the use of invariant representations is still limited because their numerical calculation from a demonstrated trajectory is complicated by sensitivity to measurement noise and singularities, yielding inaccurate invariant functions that do not correspond well with the original trajectory. This paper improves the calculation of invariant representations for point and rigid-body motions by reformulating their calculation as an optimization problem that minimizes the error between the trajectory reconstructed from the invariant representation and the measured trajectory. Robustness against noise and singularities is ensured through the addition of regularization terms on the invariants. Simulations and real motion experiments show that the accuracy of the calculated invariant representations greatly improves with respect to standard smoothing methods. These results encourage future developments of motion recognition and generalization applications based on invariant trajectory representations.

Odometry forms an important component of many manned and autonomous systems. In the rail industry in particular, having precise and robust odometry is crucial for the correct operation of the Automatic Train Protection systems that ensure the safety of high-speed trains in operation around the world. Two problems commonly encountered in such odometry systems are miscalibration of the wheel encoders and slippage of the wheels under acceleration and braking, resulting in incorrect velocity estimates. This paper introduces an odometry system that addresses these problems. It comprises of an Extended Kalman Filter that tracks the calibration of the wheel encoders as state variables, and a measurement pre-processing stage called Sensor Consensus Analysis (SCA) that scales the uncertainty of a measurement based on how consistent it is with the measurements from the other sensors. SCA uses the statistical z-test to determine when an individual measurement is inconsistent with the other measurements, and scales the uncertainty until the z-test passes. This system is demonstrated on data from German Intercity-Express highspeed trains and it is shown to successfully deal with errors due to miscalibration and wheel slip.

Despite the recent success of state-of-the-art deep learning algorithms in object recognition, when these are deployed as-is on a mobile service robot, we observed that they failed to recognize many objects in real human environments. In this paper, we introduce a learning algorithm in which robots address this flaw by asking humans for help, also known as a symbiotic autonomy approach. In particular, we bootstrap YOLOv2, a state-of-the-art deep neural network and train a new neural network, that we call HHELP, using only data collected from human help. Using an RGB camera and an onboard tablet, the robot proactively seeks human input to assist it in labeling surrounding objects. Pepper, located at CMU, and Monarch Mbot, located at ISR-Lisbon, were the service robots that we used to validate the proposed approach. We conducted a study in a realistic domestic environment over the course of 20 days with 6 research participants. To improve object detection, we used the two neural networks, YOLOv2 + HHELP, in parallel. Following this methodology, the robot was able to detect twice the number of objects compared to the initial YOLOv2 neural network, and achieved a higher mAP (mean Average Precision) score. Using the learning algorithm the robot also collected data about where an object was located and to whom it belonged to by asking humans. This enabled us to explore a future use case where robots can search for a specific person's object. We view the contribution of this work to be relevant for service robots in general, in addition to Pepper, and Mbot.

Flexible continuum robots have exhibited unique advantages in working in an unstructured environment. Many applications require robots to actively control the deformation of soft objects, such as soft tissues in surgery. Thus, this study presents a robust model-predictive deformation control of a soft object using a flexible continuum robot. A linear approximation model for mapping from actuation space of a continuum robot to deformation space of a soft object is established. Jacobian matrix is estimated online by using a robust Geman-McClure estimator. Then, the deformation of the soft object is regulated by using a prediction horizon-based controller with exponential weighting for model uncertainty. The proposed control approach is effective in manipulating a soft object with a flexible continuum robot that is in contact with obstacles.

This paper introduces a map-based localization method aiming to increase robustness in rainy conditions. This method utilizes two types of features: ground reflectivity features and vertical features extracted from 3D LIDAR scans and builds vehicle pose belief with two filters: a histogram filter and a particle filter. The posterior distributions from the two filters are integrated to estimate vehicle poses. This method exploits advantages of both features and filters, compensating respective weakness to deal with complex urban environments. Testing was performed in the fair and rainy weather. Road test results prove robustness and reliability of the proposed method.