IROS 2019 Papers

Precision grasps with multi-fingered hands are important for precise placement and in-hand manipulation tasks. Searching precision grasps on the object represented by point cloud, is challenging due to the complex object shape, high-dimensionality, collision and undesired properties of the sensing and positioning. This paper proposes an optimization model to search for precision grasps with multi-fingered hands. The model takes noisy point cloud of the object as input and optimizes the grasp quality by iteratively searching for the palm pose and finger joints positions. The collision between the hand and the object is approximated and penalized by a series of least-squares. The collision approximation is able to handle the point cloud representation of the objects with complex shapes. The proposed optimization model is able to locate collision-free optimal precision grasps efficiently. The average computation time is 0.50 sec/grasp. The searching is robust to the incompleteness and noise of the point cloud. The effectiveness of the algorithm is demonstrated by experiments.

As deep learning applications are becoming more and more pervasive in robotics, the question of evaluating the reliability of inferences becomes a central question in the robotics community. This domain, known as predictive uncertainty, has come under the scrutiny of research groups developing Bayesian approaches adapted to deep learning such as Monte Carlo Dropout. Unfortunately, for the time being, the real goal of predictive uncertainty has been swept under the rug. Indeed, these approaches are solely evaluated in terms of raw performance of the network prediction, while the quality of their estimated uncertainty is not assessed. Evaluating such uncertainty prediction quality is especially important in robotics, as actions shall depend on the confidence in perceived information. In this context, the main contribution of this article is to propose a novel metric that is adapted to the evaluation of relative uncertainty assessment and directly applicable to regression with deep neural networks. To experimentally validate this metric, we evaluate it on a toy dataset and then apply it to the task of monocular depth estimation.

Mobile multi-robot construction systems offer new ways to optimise the on-site construction process. In this paper we begin to investigate the functionality requirements for controlling a team of robots to build structures much greater than their individual workspace. To achieve these aims, we present a mobile extruder robot called YouWasp. We also begin to explore methods for collision aware printing and construction task decomposition and allocation. These are deployed via YouWasp and enable it to deposit material autonomously. In doing so, we are able to evaluate the potential for parallelization of tasks and printing autonomy in simulation as well as physical team of robots. Altogether, these results provide a foundation for future work that enable fleets of mobile construction systems to cooperate and help us shape our built environment in new ways.

Enabling machines to respond appropriately to natural language commands could greatly expand the number of people to whom they could be of service. Recently, advances in neural network-trained word embeddings have empowered non-embodied text-processing algorithms, and suggest they could be of similar utility for embodied machines. Here we introduce a method that does so by training robots to act similarly to semantically-similar word 2vec encoded commands. We show that this enables them to act appropriately, after training, to previously-unheard commands. Finally, we show that inducing such an alignment between motoric and linguistic similarities can be facilitated or hindered by the mechanical structure of the robot. This points to future, large scale methods that find and exploit relationships between action, language, and robot structure.

In this paper, we present a serial kinematic robot manipulator equipped with multimodal proximity sensing modules not only on the TCP but distributed on the robot's surface. The combination of close distance proximity information from capacitive and time-of-flight (ToF) measurements allows the robot to perform safe reflex-like and collision-free motions in a changing environment, e.g. where humans and robots share the same workspace. Our methods rely on proximity data and combine different strategies to calculate orthogonal avoidance motions. These motions are instantaneous optimal and are fed directly into the motion controller (proximity servoing). The strategies are prioritized, firstly to avoid collision and then secondly to maintain the task motion if kinematic redundancy is available. The motion is then optimized for avoidance, best manipulability, and smallest end-effector velocity deviation. We compare our methods with common force field based methods.

In this paper we propose a new framework for reconstructing underwater surfaces from wide aperture imaging sonar sequences. We demonstrate that when the leading object edge in each sonar image can be accurately triangulated in 3D, the remaining surface may be “filled in” using a generative sensor model. This process generates a full three-dimensional point cloud for each image in the sequence. We propose integrating these surface measurements into a cohesive global map using a truncated signed distance field (TSDF) to fuse the point clouds generated by each image. This allows for reconstructing surfaces with significantly fewer sonar images and viewpoints than previous methods. The proposed method is evaluated by reconstructing a mock-up piling structure and a real world underwater piling, in a test tank environment and in the field, respectively. Our surface reconstructions are quantitatively compared to ground-truth models and are shown to be more accurate than previous state-of-the-art algorithms.

In this paper, we present a novel framework for quadrupedal stair climbing, which considers force interaction with stairs. For stable and robust climbing, a key issue is to avoid falling down on the stairs. From this point of view, control to minimize rate of change of angular momentum about Center of Mass (CoM) which is produced by ground reaction force (GRF) at contact of robot`s foot is necessary. Using this approach, direct force-based Zero Moment Point (ZMP) for motion planning of the CoM and landing force control are implemented. The direct force-based ZMP method allows the robot to lift its foot with reduced possibility of tilting on the stairs, and the landing force control prevents instant increase of the moment by impact of the GRFs. In addition, terrain recognition to estimate parameters of the stairs and find proper footholds by vision system mounted on the robot is presented. Proposed framework is implemented to quadruped robot, AiDIN-VI, that has torque sensor at each joint, through experiments, capability of ascending several stairs including 3-step stairs which have 21cm height (31% of its maximum leg length) and 26.5° inclination is validated.

This article presents a trajectory planning framework for all-terrain vehicles with legs and wheels such as walking excavators. Our formulation takes into account the whole body of the robot while computing the plans for locomotion. Hence, we can produce motion plans over the rough terrain that would be hard to plan without considering all Degrees of Freedom (DoF) simultaneously. Our planner can also optimize over the contact schedule for all limbs, thereby finding the feasible motions even for the infeasible initial contact schedule. Furthermore, we introduce a novel formulation of the support area constraint. We generate plans for a Menzi Muck M545, a 31 DoF walking excavator with five limbs: four wheeled legs and an arm. We show motion plans for traversing a variety of terrains that require whole-body planning. To the best of our knowledge, this is the first work that addresses motion planning in rough terrain for vehicles with legs and wheels.

In this paper, we propose an approach for online whole-body control of position-controlled humanoid robot with 3D multi-contact to cope with contact wrench constraints and joint overload. In our method, robots are controlled under contact wrench constraints with three features: 1) internal wrench control to reduce joint load and prolong the time in which the high-load postures can be maintained 2) feasible utilization of self-collision to reduce joint load by turning off joint servo gains 3) handling degenerated degree of freedom by solving a quadratic optimization problem integrating wrench distribution and inverse kinematics in which internal wrench is controlled only in controllable directions.With our methods, HRP2-JSKNTS could pick up an object under a desk with squatting with the back of the upper leg on the back of the lower leg without sliding at the right arm. We also evaluated the effectiveness of our control to reduce joint load with another experiment.

In this work, we present a novel non-visual HAR system that achieves state-of-the-art performance on realistic SCE tasks via a single wearable sensor. We leverage surface electromyography and inertial data from a low-profile wearable sensor to attain performant robot perception while remaining unobtrusive and user-friendly. By capturing both convolutional and temporal features with a hybrid CNN-LSTM classifier, our system is able to robustly and effectively classify complex, full-body human activities with only this single sensor. We perform a rigorous analysis of our method on two datasets representative of SCE tasks, and compare performance with several prominent HAR algorithms. Results show our system substantially outperforms rival algorithms in identifying complex human tasks from minimal sensing hardware, achieving F1-scores up to 84% over 31 strenuous activity classes. To our knowledge, we are the first to robustly identify complex full-body tasks using a single, unobtrusive sensor feasible for real-world use in SCEs. Using our approach, robots will be able to more reliably understand human activity, enabling them to safely navigate sensitive, crowded spaces.

The use of high-capacity motors and harmonic drives increases the manufacturing cost and the energy consumption of the robot. It is well known that most motor torques are used to support the robot mass and payload. To deal with this problem, we propose a wall-mounted 3-DOF roll-pitch-pitch counterbalance mechanism (CBM). Unlike previous CBMs used for floor-mounted industrial robot arms, the proposed CBM can be applied to wall-mounted robots in which the first three joints are all coupled and affected by gravity. This mechanism is composed of pure mechanical components such as coil springs, differential gear structure, and belt-pulley system. A 6-DOF robot arm equipped with the proposed CBM was fabricated to verify the performance of gravity compensation. It is shown that the proposed 3-DOF CBM can effectively compensate for gravitational torques, thus drastically reducing the motor torques required to operate the robot.

The performance of traditional hydraulic robots is often limited by their hoses across moving joints or connecting hydraulic drive units, which would reduce their mobility and impede their ability to operate in complex environment. In response to this deficiency, this paper introduces the WLR-II (the second generation of wheel-legged robot), a novel hydraulic wheel-legged robot developed by using hose-less design approach which is focused on improving the reliability of the hydraulic system and perfecting the appearance of the robot. As its notable features, seven Hydraulic Hose-less Joints (HHJ) that include a pair of high and also low pressure oil pipes based on rotary seal, Cylinder-Valve-Skeleton (CVS) integration thighs and arms which are produced by subtractive manufacturing as well as oscillating cylinders driven by gear rack transmission are included. In addition to a description of its design, experimental characterizations of rough pavement adaptability and payload capability together with the achievement of the reliability of hydraulic system are also demonstrated. As a result, we confirmed effectiveness of the hose-less design by moving on the rugged ground, climbing slope, squatting with load, dragging and picking up a heavy load. To the authors' best knowledge, this is the first time that the design of a hose-less hydraulic wheel-legged robot has been presented.

We present volumetric tree*, a hybridization of sampling-based and optimization-based motion planning. Volumetric tree* constructs an adaptive sparse graph with volumetric vertices, hyper-spheres encoding free configurations, using a sampling-based motion planner for a homotopy exploration. The coarse-grained paths computed on the sparse graph are refined by optimization-based planning during the execution, while exploiting the probabilistic completeness of the sampling- based planning for the initial path generation. We also suggest a dropout technique probabilistically ensuring that the sampling- based planner is capable of identifying all possible homotopies of solution paths. We compare the proposed algorithm against the state-of-the-art planners in both synthetic and practical benchmarks with varying dimensions, and experimentally show the benefit of the proposed algorithm.

In this work, we propose a voice-controlled hand rehabilitation device driven by exotendons. A smartphone-based voice recognition system interprets user intention and is utilized for various grasping tasks. A bio-inspired tendon routing mechanism provides four-degrees-of-freedom (DoFs) across the thumb, index finger, and middle finger. A novel thumb sleeve design is presented for stable thumb movement. The exoskeleton is fabricated from polyurethane rubber and rigid 3D-printed parts to provide form-fitting properties while constraining tendon motion. Twisted string actuators and spring units provide active flexion and passive extension, respectively. The compact nature of the actuation unit allows for placement on the forearm, improving the portability of the system. The voice control system allows for easy user manipulation and accessibility and may improve rehabilitation efficiency. The performance of the tendon routing and thumb sleeve design were experimentally evaluated and voice control system was evaluated with various grasping tests.

End-to-end visual-based imitation learning has been widely applied in autonomous driving. When deploying the trained visual-based driving policy, a deterministic command is usually directly applied without considering the uncertainty of the input data. Such kind of policies may bring dramatical damage when applied in the real world. In this paper, we follow the recent real-to-sim pipeline by translating the testing world image back to the training domain when using the trained policy. In the translating process, a stochastic generator is used to generate various images stylized under the training domain randomly or directionally. Based on those translated images, the trained uncertainty-aware imitation learning policy would output both the predicted action and the data uncertainty motivated by the aleatoric loss function. Through the uncertainty-aware imitation learning policy, we can easily choose the safest one with the lowest uncertainty among the generated images. Experiments in the Carla navigation benchmark show that our strategy outperforms previous methods, especially in dynamic environments.

We propose an approach to the throw-and-go (TnG) problem for micro air vehicles (MAVs) using visual and inertial sensors. The key challenge is the fast on-board initialization of the visual odometry (VO) system, which usually requires user input to recover the visual scale. Our approach is based on the identification of the gravity vector from the acceleration data computed with images of the ground during in free fall. This enables scaling of the poses reconstructed with visual information. The proposed framework use inertial data to control the MAV attitude so the ground is visible after the throw. Using image to image homography a metric scale is estimated with which the MAV's height is propagated. Unlike existing literature, this approach requires no additional sensor nor user input or pre-throw assumptions and can recover from any initial attitude. We show results on both simulation and real data.

In this paper, we present a tightly-coupled monocular visual-inertial navigation system (VINS) using points and lines with degenerate motion analysis for 3D line triangulation. Based on line segment measurements from images, we propose two sliding window based 3D line triangulation algorithms and compare their performance. Analysis of the proposed algorithms reveals 3 degenerate camera motions that cause triangulation failures. Both geometrical interpretation and Monte-Carlo simulations are provided to verify these degenerate motions which prevent triangulation. In addition, commonly used line representations are compared through a monocular visual SLAM Monte-Carlo simulation. Finally, real-world experiments are conducted to validate the implementation of the proposed VINS system using the “closest point” line representation.

Combining camera, IMU and wheel encoder is a wise choice for car positioning because of the low cost and complementarity of the sensors. We propose a novel extended visual-inertial odometry algorithm tightly fusing data from the above three sensors. Firstly we propose an IMU-odometer pre-integration approach utilizing complete IMU measurements and wheel encoder readings, to make scale estimation more accurate in subsequent 4-degrees of freedom (DoF) optimization. Secondly we develop an original initialization module where encoder readings are fully utilized to refine gravity direction and provide an initial value for camera pose in real scale. Thirdly, we design a computationally efficient online extrinsic calibration method by fixing the linearization point for the rotational component of IMU-odometer extrinsic parameters, which is deployed depending on the convergence of accelerometer bias. Experimental results prove the robustness of our initialization module and the accuracy of the whole trajectory, as well as the improvement brought about by online extrinsic calibration. Our program can also run on an Nvidia Jetson TX2 module in real time.

Domain adaptation methods are used to address a problem, in which train scenario (source domain) and test scenario (target domain) are different. The existing methods mainly perform adaptation via reducing domain discrepancy from the view of a probability distribution. However, the idea of probability distribution matching always leads to a complex optimization process. Thereby these methods are difficult to apply in some scenario like online application or fast perception in dynamic environments. In this paper, we propose a new and simple domain adaptation method that utilizes confidence- samples to facilitate the classifier training on the target domain. Here, the confidence-samples are a subset of the target samples, and they have very credibly predicted labels. In order to detect the samples, a Category Similarity Collaborative Representation (CSCR) is first developed, by which the raw labels of all target samples are predicted using the smallest projection error according to the law of category. After this, the confidence score of the raw predicted labels is evaluated by the energy context information of CSCR. Finally, the target samples with a high confidence score are selected. Because of the linearity of CSCR, our method avoids complex optimization for matching the probability distribution. Empirical studies on a standard dataset demonstrate the advantages of our method.