IROS 2018 Papers

In this paper, we propose a novel 3D laparoscopic imaging system based on stereo-photogrammetry which is assisted by projecting patterns on the tissue surface. The proposed laparoscopic imaging system has three optic channels, two of which are responsible for stereo vision feedback and the other one is used for coded structured patterns projection. The projected patterns provide the robustness to homogeneous tissue surface since they add more features that can be relied on in the stereo matching. Image fiber bundles (100k pixels) and Gradient-index (GRIN) lenses are utilized to facilitate the remote image acquisition and miniaturization of the laparoscopic probe. Moreover, we adopt a digital micromirror device (DMD) and high-speed cameras to achieve fast pattern switching (up to 4 kHz) and high frame rate image acquisition. The system configuration allows for implementation of the time multiplexing pattern codification strategy in the 3D laparoscopic imaging system to enhance the reliability and resolution of the 3D surface reconstruction. A prototype is established, and various experiments are conducted. Comparative experimental results prove the advantages of our system design. The static and dynamic 3D reconstruction results validate the performance of the proposed 3D laparoscopic imaging system quantitatively and qualitatively.

We focus on the task of amodal 3D object detection, which is to predict object locations, dimensions, poses and categories in the real world. We introduce a 3D Convolutional Neural Network that takes a volumetric representation of an indoor scene as input and predicts 3D object bounding boxes, object categories, and orientations. Unlike prior state-of-the-arts, our approach does not depend on region proposal techniques to hypothesize object locations. We treat detection and recognition as one regression problem in a single network. Our elegant model is extremely fast and all predictions are reasoned from the global context of a point cloud in a continuous pipeline. We evaluate our approach on two standard datasets: the NYUv2 RGBD dataset and the SUN RGBD dataset. Experiments show that our approach is faster than start-of-the-art 3D detectors by several orders of magnitude towards real-time amodal 3D object detection.

Flexure-based finger joints for prosthetic hands have been studied, but until now they lack stiffness and load bearing capacity. In this paper we present a design which combines large range of motion, stiffness and load bearing capacity, with an overload protection mechanism. Several planar and non-planar hinge topologies are studied to determine load capacity over the range of motion. Optimized topologies are compared, in 30 degrees deflected state, in terms of stresses by deflection and grasping forces. Additionally, support stiffnesses were computed for all hinges in the whole range of motion (45 degrees). The Hole Cross Hinge presented the best performance over the range of motion with a grasping force up to 15 N while deflected 30 degrees. A new concept, the Angle Three-Flexure Cross Hinge, provides outstanding performance for deflections from 17.5 up to 30 degrees with a 20 N maximum grasping force when fully deflected. Experimental verification of the support stiffness over the range of motion shows some additional compliances, but the stiffness trend of the printed hinge is in line with the model. The presented joints power grasping capability outperform current state flexure-base hands and are comparable to commercial non-flexure-based prosthetic hands. In the event of excessive loads, an overload protection mechanism is in place to protect the flexure- hinges.

Automatic subsurface mapping is essential in the construction services, as it is anticipated to become the main operational environment of the future robots to be realized in the respective domain. Towards this direction, the paper at hand, introduces for the first time herein, an integrated framework for subsurface mapping by exploiting a surface operating mobile robot with a Ground Penetrating Radar (GPR). The mobile robot tows the GPR antenna, which is mounted on a specifically designed trailer, and is utilized as the mean to cover the surface area, while at the same time the antenna scans the subsurface by emitting electromagnetic pulses. The gathered data are processed for the construction of a subsurface 3D map. Specifically, image processing techniques, that involve background segmentation, HOG [1] feature extraction, hypothesis verification and matching are applied on the 2D radargram (B-Scan) for the detection of the salient points that correspond to buried utilities. By employing the pulse propagation velocity into the subsurface and the soil utilities, the salient points are expressed in world coordinates and used for the composition of the 3D subsurface map. Our method has been evaluated on a real test site, accompanied by ground-truth annotation data of experts and revealed remarkable performance, exhibiting not only the feasibility of underground mapping but also the capacity to obtain exploitable results for underground robotic applications.

Perceiving accurate 3D object shape is important for robots to interact with the physical world. Current research along this direction has been primarily relying on visual observations. Vision, however useful, has inherent limitations due to occlusions and the 2D-3D ambiguities, especially for perception with a monocular camera. In contrast, touch gets precise local shape information, though its efficiency for reconstructing the entire shape could be low. In this paper, we propose a novel paradigm that efficiently perceives accurate 3D object shape by incorporating visual and tactile observations, as well as prior knowledge of common object shapes learned from large-scale shape repositories. We use vision first, applying neural networks with learned shape priors to predict an object's 3D shape from a single-view color image. We then use tactile sensing to refine the shape; the robot actively touches the object regions where the visual prediction has high uncertainty. Our method efficiently builds the 3D shape of common objects from a color image and a small number of tactile explorations (around 10). Our setup is easy to apply and has potentials to help robots better perform grasping or manipulation tasks on real-world objects.

Robots have the potential to assist people in bed, such as in healthcare settings, yet bedding materials like sheets and blankets can make observation of the human body difficult for robots. A pressure-sensing mat on a bed can provide pressure images that are relatively insensitive to bedding materials. However, prior work on estimating human pose from pressure images has been restricted to 2D pose estimates and flat beds. In this work, we present two convolutional neural networks to estimate the 3D joint positions of a person in a configurable bed from a single pressure image. The first network directly outputs 3D joint positions, while the second outputs a kinematic model that includes estimated joint angles and limb lengths. We evaluated our networks on data from 17 human participants with two bed configurations: supine and seated. Our networks achieved a mean joint position error of 77 mm when tested with data from people outside the training set, outperforming several baselines. We also present a simple mechanical model that provides insight into ambiguity associated with limbs raised off of the pressure mat, and demonstrate that Monte Carlo dropout can be used to estimate pose confidence in these situations. Finally, we provide a demonstration in which a mobile manipulator uses our network's estimated kinematic model to reach a location on a person's body in spite of the person being seated in a bed and covered by a blanket.

For the success of visually-guided robotic errand service, it is critical to ensure dependability under various ill-conditioned visual environments. To this end, we have developed Adaptive Bayesian Recognition Framework in which in-situ selection of multiple sets of optimal features or evidences as well as proactive collection of sufficient evidences are proposed to implement the principle of dependability. The framework has shown excellent performance with a limited number of objects in a scene. However, there arises a need to extend the framework for handling a larger number of objects without performance degradation, while avoiding difficulty in feature engineering. To this end, a novel deep learning architecture, referred to here as FER-CNN, is introduced and integrated into the Adaptive Bayesian Recognition Framework. FER-CNN has capability of not only extracting but also reconstructing a hierarchy of features with the layer-wise independent feedback connections that can be trained. Reconstructed features representing parts of 3D objects then allow them to be semantically linked to ontology for exploring object categories and properties. Experiments are conducted in a home environment with real 3D daily-life objects as well as with the standard ModelNet dataset. In particular, it is shown that FER-CNN allows the number of objects and their categories to be extended by 10 and 5 times, respectively, while registering the recognition rate for ModelNet10 and ModelNet40 by 97% and 89.5%, respectively.

In this paper, we proposed a pheromone-based aggregation method based on the state-of-the-art BEECLUST algorithm. We investigated the impact of pheromone-based communication on the efficiency of robotic swarms to locate and aggregate at areas with a given cue. In particular, we evaluated the impact of the pheromone evaporation and diffusion on the time required for the swarm to aggregate. In a series of simulated and real-world evaluation trials, we demonstrated that augmenting the BEECLUST method with artificial pheromone resulted in faster aggregation times.