ICRA 2018 Papers

Recent algorithms have demonstrated the feasibility of underwater feature-based SLAM using imaging sonar. But previous methods have either relied on manual feature extraction and correspondence or used prior knowledge of the scene, such as the planar scene assumption. Our proposed system provides a general-purpose method for feature-point extraction and correspondence in arbitrary scenes. Additionally, we develop a method of identifying point landmarks that are likely to be well-constrained and reliably reconstructed. Finally, we demonstrate that while under-constrained landmarks cannot be accurately reconstructed themselves, they can still be used to constrain and correct the sensor motion. These advances represent a large step towards general-purpose, feature-based SLAM with imaging sonar.

The-state-of-the-art of fruit detection with Faster R-CNN shows lack of detection advantage on small fruits. One of reasons is only single level features is used for localization of proposal candidates. In this paper, we propose to incorporate a multiple classifier fusion strategy into a Faster R-CNN network for small fruit detection. We utilize features from three different levels to learn three classifiers for objectness classification in the stage of proposal localization. Probabilities from classifiers are combined by a simple convolutional layer to generate final objectness classification for proposal candidates. In order to keep diversity of multiple classifiers, a novel loss term of classifier correlation is introduced into original loss function. Experimental results show that our model is feasible for detecting small fruits.

Automating precision subtasks such as debridement (removing dead or diseased tissue fragments) with Robotic Surgical Assistants (RSAs) such as the da Vinci Research Kit (dVRK) is challenging due to inherent nOnlinearities in cable-driven systems. We propose and evaluate a novel two-phase coarse-to-fine calibration method. In Phase I (coarse), we place a red calibration marker on the end effector and let it randomly move through a set of open-loop trajectories to obtain a large sample set of camera pixels and internal robot end-effector configurations. This coarse data is then used to train a Deep Neural Network (DNN) to learn the coarse transformation bias. In Phase II (fine), the bias from Phase I is applied to move the end -effector toward a small set of specific target points on a printed sheet. For each target, a human operator manually adjusts the end -effector position by direct contact (not through teleoperation) and the residual compensation bias is recorded. This fine data is then used to train a Random Forest (RF) to learn the fine transformation bias. Subsequent experiments suggest that without calibration, position errors average 4.55mm. Phase I can reduce average error to 2.14mm and the combination of Phase I and Phase II can reduces average error to 1.08mm. We apply these results to debridement of raisins and pumpkin seeds as fragment phantoms. Using an endoscopic stereo camera with standard edge detection, experiments with 120 trials achieved average success rates of 94.5 %, exceeding prior results with much larger fragments (89.4%) and achieving a speedup of 2.1x, decreasing time per fragment from 15.8 seconds to 7.3 seconds. Source code, data, and videos are available at https://sites.google.com/view/calib-icra/.

In this paper, we consider reorienting 3D objects on a table using a two-finger pinch gripper. Given the 3D mesh model of the object, our algorithm solves for the gripper motions that are required to transit between arbitrary object poses, grasping positions and gripper poses. The two motion primitives we used, pivoting and compliant rolling, enable us to decompose the planning problem and solve it more efficiently. Our algorithm can work with approximated (simplified) mesh models while being robust to approximation errors, thereby allowing us to efficiently handle object shapes with originally thousands of facets. We show the effectiveness of the proposed method by testing on objects with non-trivial geometry in both simulations and experiments. Results show that our algorithm can solve a larger range of reorienting problems with less number of making and breaking contacts when compared to traditional pick-and-place based methods, especially when the gripper workspace is highly constrained.

Robotic picking from cluttered bins is a demanding task, for which Amazon Robotics holds challenges. The 2017 Amazon Robotics Challenge (ARC) required stowing items into a storage system, picking specific items, and packing them into boxes. In this paper, we describe the entry of team NimbRo Picking. Our deep object perception pipeline can be quickly and efficiently adapted to new items using a custom turntable capture system and transfer learning. It produces high-quality item segments, on which grasp poses are found. A planning component coordinates manipulation actions between two robot arms, minimizing execution time. The system has been demonstrated successfully at ARC, where our team reached second places in both the picking task and the final stow-and-pick task. We also evaluate individual components.

We present a fast nonlinear approximation method for marginalizing out nodes on pose graphs for longterm simultaneous localization, mapping, and navigation. Our approximation preserves the pose graph structure to leverage the rich literature of pose graphs and optimization schemes. By re-parameterizing from absolute-to relative-pose spaces, our method does not suffer from the choice of linearization points as in previous works. We then join our approximation process with a scaled version of the recently-demoted pose-composition approach. Our approach eschews the expenses of many state-of-the-art convex optimization schemes through our efficient and simple O(N2) implementation for a given known topology of the approximate subgraph. We demonstrate its speed and near optimality in practice by comparing against state-of-the-art techniques on popular datasets.

3D models of the environment are used in numerous robotic applications and should reflect the current state of the world. In this paper, we address the problem of quickly finding structural changes between the current state of the world and a given 3D model using a small number of images. Our approach finds inconsistencies between pairs of images by re-projecting an image onto another one by passing through the given 3D model. This process leads to ambiguities, which we resolve by combining multiple images such that the 3D location of the change can be estimated. A focus of our approach is that it can be executed fast enough to allow the operation on a mobile system. We implemented our approach in C++ and released it as open source software. We tested it on existing datasets as well as on self-recorded image sequences and 3D models, which we publicly share. Our experiments show that our method quickly finds changes in the geometry of a scene.

Depth map estimation techniques from cameras often struggle to accurately estimate the depth of large textureless regions. In this work we present a vision-only method that accurately extracts planar priors from a viewed scene without making any assumptions of the underlying scene layout. Through a fast global labelling, these planar priors can be associated to the individual pixels leading to more complete depth-maps specifically over large, plain and planar regions that tend to dominate the urban environment. When these depth-maps are deployed to the creation of a vision only dense reconstruction over large scales, we demonstrate reconstructions that yield significantly better results in terms of coverage while still maintaining high accuracy.

Disparity estimation is a difficult problem in stereo vision because the correspondence technique fails in images with textureless and repetitive regions. Recent body of work using deep convolutional neural networks (CNN) overcomes this problem with semantics. Most CNN implementations use an autoencoder method; stereo images are encoded, merged and finally decoded to predict the disparity map. In this paper, we present a CNN implementation inspired by dense networks to reduce the number of parameters. Furthermore, our approach takes into account semantic reasoning in disparity estimation. Our proposed network, called DenseMapNet, is compact, fast and can be trained end-to-end. DenseMapNet requires 290k parameters only and runs at 30Hz or faster on color stereo images in full resolution. Experimental results show that DenseMapNet accuracy is comparable with other significantly bigger CNN-based methods.

This paper exemplifies the design process for legged machines capable of dynamic behaviors. In order to achieve high performance robots, it is crucial to guarantee harmonious integration between software and hardware. Hence, the development of such capable robotic platforms must address design requirements that meet the assumptions of typical model-based controllers but also respect the physical limitations of a real system. First, we show that proper hardware design choices can greatly aid the control algorithm by approximating the physical robot to the template assumptions. We include actuation and sensing design examples that allows a simple model to capture a major portion of the natural dynamic behavior of the physical machine. Results are applied to a real robot (Figure 1) and we show that the adopted methodology is able to address typical problems in legged robots such as high bandwidth force control and robustness to impact. Finally, a simple model-based balance controller that takes advantage of the fidelity of the template model to the real machine is implemented. These are examples of software-hardware codesign processes that vastly facilitate robotic control.

Limb amputation is a global problem. Prosthetic limbs can enhance the quality of life of amputees. To this end, anthropomorphic design and intuitive manipulation are two essential requirements. This paper presents a motor control framework for prosthetic control through Brain-Machine Interface (BMI) using Finite Automata Theory, and NeuCube Evolving Spiking Neural Network (SNN) architecture. Voluntary control of prosthetics requires decoding motor commands from the Central Nervous System of the amputee. Selection of the most suitable biomedical signal depends on many parameters such as level of amputation and muscle atrophy. Non-invasive BMI's have the possibility of supporting a wider range of amputees as it extracts the motor commands from the brain. In this paper, we present a proof of concept study on whether a cognitive computational model that is inspired by the motor control of the human body through muscle synergies combined with an anthropomorphic mechanical design, can result in accurate and robust prosthetic control through a noninvasive BMI. In future, learning of a complex Finite Automata that reflects complex upper limb motor behaviours will be investigated.

Person identification (PID) is a key issue in many IoT applications. It has long been studied and achieved by technologies such as RFID and face/fingerprint/iris recognition. These approaches, however, have their limitations due to environmental constraints (such as lighting and obstacles) or require close contact to specific devices. Therefore, their recognition rates highly depend on use scenarios. To enable reliable and remote PID, in this work, we present EOY (Eye On You)1, a data fusion approach that combines two kinds of sensors, a 3D depth camera and wearable sensors embedded with inertial measurement unit (IMU). Since these two kinds of data share common features, we are able to fuse them to conduct PID. Further, the result can be transferred to a mobile platform (such as robot) since we have less constraints on devices. To realize EOY, we develop fusion algorithms to address practical challenges, such as asynchronous timing and coordinate calibration. The experimental evaluation shows that EOY can achieve the recognition rate of 95% and is very robust even in crowded areas.

We present a machine learning methodology for actively controlling slip, in order to increase robot dexterity. Leveraging recent insights in deep learning, we propose a Deep Predictive Model that uses tactile sensor information to reason about slip and its future influence on the manipulated object. The obtained information is then used to precisely manipulate objects within a robot end-effector using external perturbations imposed by gravity or acceleration. We show in a set of experiments that this approach can be used to increase a robot's repertoire of motor skills.

In robotic actuation a well identified and modeled friction behavior of the actuator components helps to significantly improve friction compensation, output torque estimation, and dynamic simulations. The friction of two components, i.e. a brush-less DC motor and a harmonic drive gear (HD) is investigated in order to build an accurate dynamic model of the main actuator of the arms of the humanoid David namely the DLR Floating Spring Joint (FSJ). A dedicated testbed is built to precisely identify input and output torques, temperatures, positions, and elasticities of the investigated components at a controlled environment temperature. Extensive test series are performed in the full velocity operating range in a temperature interval from 24 to 50 °C. The nonlinear influences of velocity and temperature are identified to be dominant effects. It is proposed how to include these nonlinear velocity and temperature dependencies into a static and a dynamic friction model, e.g. LuGre. Dynamic models of the motor and HD are built with the proposed method and experimentally evaluated. The new models are compared to friction models with linear dependencies and show a significant improvement of correspondence with reality.

LED-based optical communication is emerging as a low-cost, high-data-rate alternative to the traditional acoustics mode of underwater communication. However, it is challenging to establish and maintain Line-Of-Sight (LOS) between the receiver and the transmitter, especially when such systems are used by mobile robots. Hence, there is a need for an active alignment system that enables the receiver to constantly align itself towards the direction of the transmitting device. In this paper, we propose and implement an active alignment control system capable of tracking a transmitting source moving in the three-dimensional (3D) space. An extended Kalman filter is used to estimate the components of the angle between the receiver orientation and the receiver-transmitter line. Using the estimate, a proportional-integral (PI) controller is implemented to adjust the receiver orientation. The algorithm uses one measurement of the light intensity from a single photo-diode, where successive measurements are obtained via a circular scanning technique. The amplitude of the scanning is adapted to the alignment performance, to achieve a sound trade-off between estimation accuracy, signal strength, and energy consumption. Simulation and experimental results are presented to illustrate the effectiveness of the proposed approach.

Under- and oversaturation can cause severe image degradation in many vision-based robotic applications. To control camera exposure in dynamic lighting conditions, we introduce a novel metric for image information measure. Measuring an image gradient is typical when evaluating its level of image detail. However, emphasizing more informative pixels substantially improves the measure within an image. By using this entropy weighted image gradient, we introduce an optimal exposure value for vision-based approaches. Using this newly invented metric, we also propose an effective exposure control scheme that covers a wide range of light conditions. When evaluating the function (e.g., image frame grab) is expensive, the next best estimation needs to be carefully considered. Through Bayesian optimization, the algorithm can estimate the optimal exposure value with minimal cost. We validated the proposed image information measure and exposure control scheme via a series of thorough experiments using various exposure conditions.

In this paper, we show how structures and strategies employed by thin-stemmed plants can be adapted to improve robot access to unstructured and congested environments. Specifically, we show how the use of vine-inspired movement strategies can enhance long thin continuum robot exploration and inspection operations. We introduce a new theoretical plant growth-inspired approach for modeling and motion generation of continuum robot backbones. The approach is demonstrated in numerous experiments including inspection within a high fidelity, full-scale mock-up of the International Space Station at NASA Johnson Space Center, using novel robot tendril hardware.

High speed navigation through unknown environments is a challenging problem in robotics. It requires fast computation and tight integration of all the subsystems on the robot such that the latency in the perception-action loop is as small as possible. Aerial robots add a limitation of payload capacity, which restricts the amount of computation that can be carried onboard. This requires efficient algorithms for each component in the navigation system. In this paper, we describe our quadrotor system which is able to smoothly navigate through mixed indoor and outdoor environments and is able to fly at speeds of more than 18 m/s. We provide an overview of our system and details about the specific component technologies that enable the high speed navigation capability of our platform. We demonstrate the robustness of our system through high speed autonomous flights and navigation through a variety of obstacle rich environments.

Grasp and manipulation is a complex task, deceivingly simple to accomplish for humans in everyday life, yet challenging to implement in a robotic hand. There is a trend in literature to use information obtained from studies on human grasp for the design and control of robotic manipulators. However, the effectiveness of such approach is dependent on the measurement tools that are available for use with human hands. While there are many sensing solutions that are designed for this purpose, obtaining a complete set of measurements of forces during grasp interaction is still challenging. In this work we aim to bridge this gap by introducing ExoSense, a passive hand exoskeleton. This device can provide position and orientation of the fingertips and, when integrated with the fingertip wearable force/torque sensing system ThimbleSense, a complete characterization of manipulation in terms of generalized forces and position of contacts on each fingertip in a completely wearable and unconstrained manner. After validating the device in terms of end-effector posture measurements and overall accuracy of grasp measurements, we report on a preliminary experiment aiming to show the potentialities of the system to study human internal grasp force variations and for neuroscientific investigation in general.