IROS 2022 Papers

The paper reports the performance verification of vertical bend and T-branch travels of an articulated wheeled in-pipe inspection robot. The robot is composed of only a single active compliant middle joint, two passive compliant joints, three drive wheels, and two roll wheels. The passage of the bend pipe is achieved only by the joint torque control, while the T-branch travel is achieved by controlling both joint angle and torque. Instead of using a torque sensor, a polyurethane-based series elastic actuator (SEA) is installed in the middle joint. In this paper, the travel performances of our developed in-pipe robot were tested on bend pipes and 10 types of T-branch with different gravity directions. From the experiments, in all cases, the effectiveness of the bend and T-branch travels performance was confirmed.

We develop a generic mechanism for generating vehicle-type specific sequences of waypoints from a probabilistic foundation model of driving behavior. Many foundation behavior models are trained on data that does not include vehicle information, which limits their utility in downstream applications such as planning. Our novel methodology conditionally specializes such a behavior predictive model to a vehicle-type by utilizing byproducts of the reinforcement learning algorithms used to produce vehicle specific controllers. We show how to compose a vehicle specific value function estimate with a generic probabilistic behavior model to generate vehicle-type specific waypoint sequences that are more likely to be physically plausible then their vehicle-agnostic counterparts.

The lower limb morphology of biological locomotors is abundant in muscle-tendon units. Yet, not much is known about how these actuation units contribute to the output performance and energy economy of movements. In this work, we investigate the functionality of four of the important lower limb muscles - Vastus, Popliteus, Soleus, and Gastrocnemius - in a hopping task at different frequencies (1.5-3.5 Hz). These muscles are implemented as pneumatic artificial muscles (PAMs) on the EPA-Hopper-II robot, which is a human-sized 3-segmented leg co-actuated by electrical motors and PAMs. A bioinspired reflex-based Force Modulated Control (FMC) is also implemented on the robot to achieve hopping at different frequencies. The results show that the Vastus contributes the most to energy-efficient hopping at low to mid frequencies. The biarticular Gastrocnemius also helps increase efficiency at low frequencies. Further, it is found that the Gastrocnemius synchronizes the knee-ankle motion and mitigates lateral knee motion. The outcomes of this work add further evidence to hypotheses regarding human lower-limb actuation and proper recruitment of muscles for building more efficient robots.

From a medical standpoint, detecting the size and shape of hard inclusions hidden in soft three-dimensional objects is of great significance for early detection of cancer through palpation. Soft robots, especially soft grippers, substantially broaden robots' palpation capabilities from soft to hard materials without the assistance of a camera. We have recently introduced a CNN-Bayes approach which added a Naïve Bayes classifier to a convolutional neural network (CNN) architecture called SoftTactNet for variable stiffness object recognition on a three-finger FinRay soft gripper. SoftTactNet itself lacks uncertainty estimations though it can reach a certain level of recognition accuracy. In this paper, we further improve the framework by merging Bayes method directly into CNN architectures and build a new Bayes-SoftTactNet for object recognition. The new approach, using a prior distribution instead of point estimation, allows the network to present results with uncertainty estimates. We conduct new experiments using the same soft gripper with tactile sensor arrays to grasp different variable stiffness objects surrounded by non-different soft material and generate tactile images as dataset. The results show that our new algorithm is more efficient than the previous approach and still able to achieve higher recognition accuracy than general deterministic CNNs.

In recent years, haptic telemanipulation has been introduced to control robots remotely with an input device that generates force feedback. Compliant control strategies are needed to ensure safe interaction between humans and robots. Accurate and precise manipulation requires a stiff setup of the impedance parameters, while safety demands for low stiffness. This paper proposes an impedance-based control approach that combines stiff manipulation with a safety mechanism that adapts compliance when required. We introduce three system modes: operation, safety and recovery mode. If the external forces exceed a defined force threshold, the system switches to the compliant safety mode. A user input triggers the recovery process that increases the stiffness back to its nominal value. This paper suggests an energy tank, which limits the change of stiffness to ensure stability during recovering phase. We validate the functionality of this approach using a real telemanipulation setup and show that the suggested tank enables recovery even from large displacements.

VR Facial Animation is necessary in applications requiring clear view of the face, even though a VR headset is worn. In our case, we aim to animate the face of an operator who is controlling our robotic avatar system. We propose a real-time capable pipeline with very fast adaptation for specific operators. In a quick enrollment step, we capture a sequence of source images from the operator without the VR headset which contain all the important operator-specific appearance information. During inference, we then use the operator keypoint information extracted from a mouth camera and two eye cameras to estimate the target expression and head pose, to which we map the appearance of a source still image. In order to enhance the mouth expression accuracy, we dynamically select an auxiliary expression frame from the captured sequence. This selection is done by learning to transform the current mouth keypoints into the source camera space, where the alignment can be determined accurately. We, furthermore, demonstrate an eye tracking pipeline that can be trained in less than a minute, a time efficient way to train the whole pipeline given a dataset that includes only complete faces, show exemplary results generated by our method, and discuss performance at the ANA Avatar XPRIZE semifinals.

This study proposes a visual localization method using a vector map and voxel grid map with a stereo camera. The two maps provide different modality advantages and are integrated using a particle filter. In contrast to other vector map-based methods, our method does not use road markings because creating and maintaining vector maps that include high-accuracy road markings is laborious. Furthermore, it limits the regions where they are available. This method uses only lane center-lines from vector maps, which are easier to create than road markings. The method performs ray casting and computes the reprojection error to evaluate the vehicle position for voxel grid maps. Although this makes the method environmentally sensitive, the constraints by lanes make the estimation stable. Experiments confirmed that the method could perform localization stably and accurately without failure even over long distances. In addition, an ablation study showed the benefits of combining both maps.

One of the key challenges in high-speed off-road navigation on ground vehicles is that the kinodynamics of the vehicle-terrain interaction can differ dramatically depending on the terrain. Previous approaches to addressing this challenge have considered learning an inverse kinodynamics (IKD) model, conditioned on inertial information of the vehicle to sense the kinodynamic interactions. In this paper, we hypothesize that to enable accurate high-speed off-road navigation using a learned IKD model, in addition to inertial information from the past, one must also anticipate the kinodynamic interactions of the vehicle with the terrain in the future. To this end, we introduce Visual-Inertial Inverse Kinodynamics (VI-IKD), a novel learning based IKD model that is conditioned on visual information from a terrain patch ahead of the robot in addition to past inertial information, enabling it to anticipate kinodynamic interactions in the future. We validate the effectiveness of VI-IKD in accurate high-speed off-road navigation experimentally on a scale 1/5 UT-AlphaTruck off-road autonomous vehicle in both indoor and outdoor environments and show that compared to other state-of-the-art approaches, VI-IKD enables more accurate and robust off-road navigation on a variety of different terrains at speeds of up to 3.5m/s.

In this paper, we propose a novel Voting based Grasp Pose Network (VGPN) to detect 6-DoF grasps in cluttered scenes. The motivation of this paper is that local object geometry can provide useful clues about where the object can be grasped. Generated by the sampled seed points from raw point cloud, the votes allow seed points in different object regions to contribute to locations where the object can be grasped. Geometric features from various local regions are aggregated to generate grasps in a more confident and dense space, which enables grasp prediction utilizing more global context features. The search space of grasp pose detection is also greatly reduced. Experimental results on both simulation and real-world environments show that our proposed method outperforms state-of-the-art approaches in terms of both success rate and coverage of the ground truth grasps. The objects can be grasped with fewer attempts which is critical in real-world applications.

Terrain classification is a challenging task for robots operating in unstructured environments. Existing classification methods make simplifying assumptions, such as a reduced number of classes, clearly segmentable roads, or good lighting conditions, and focus primarily on one sensor type. These assumptions do not translate well to off-road vehicles, which operate in varying terrain conditions. To provide mobile robots with the capability to identify the terrain being traversed and avoid undesirable surface types, we propose a multimodal sensor suite capable of classifying different terrains. We capture high resolution macro images of surface texture, spectral reflectance curves, and localization data from a 9 degrees of freedom (DOF) inertial measurement unit (IMU) on 11 different terrains at different times of day. Using this dataset, we train individual neural networks on each of the modalities, and then combine their outputs in a fusion network. The fused network achieved an accuracy of 99.98% percent on the test set, exceeding the results of the best individual network component by 0.98%. We conclude that a combination of visual, spectral, and IMU data provides meaningful improvement over state of the art in terrain classification approaches. The data created for this research is available at https://github.com/RIVeR-Lab/vast_data.

Imitation learning has attracted attention as a method for realizing complex robot control without programmed robot behavior. Meta-imitation learning has been proposed to solve the high cost of data collection and low generalizability to new tasks that imitation learning suffers from. Meta-imitation can learn new tasks involving unknown objects from a small amount of data by learning multiple tasks during training. However, meta-imitation learning, especially using images, is still vulnerable to changes in the background, which occupies a large portion of the input image. This study introduces a human gaze into meta-imitation learning-based robot control. We created a model with model-agnostic meta-learning to predict the gaze position from the image by measuring the gaze with an eye tracker in the head-mounted display. Using images around the predicted gaze position as an input makes the model robust to changes in visual information. We experimentally verified the performance of the proposed method through picking tasks using a simulated robot. The results indicate that our proposed method has a greater ability than the conventional method to learn a new task from only 9 demonstrations even if the object's color or the background pattern changes between the training and test.

In this work, we propose a data-driven approach to optimize the parameters of a simulation such that control policies can be directly transferred from simulation to a real-world quadrotor. Our neural network-based policies take only onboard sensor data as input and run entirely on the embed-ded hardware. In real-world experiments, we compare low-level Pulse-Width Modulated control with higher-level control structures such as Attitude Rate and Attitude, which utilize Proportional-Integral-Derivative controllers to output motor commands. Our experiments show that low-level controllers trained with Reinforcement Learning require a more accurate simulation than higher-level control policies at the expense of being less robust towards parameter uncertainties.

Various forms of human knowledge can be explicitly used to enhance deep robot learning from demonstrations. Annotation of subtasks from task segmentation is one type of human symbolism and knowledge. Annotated subtasks can be referred to as action labels, which are more primitive symbols that can be building blocks for more complex human reasoning, like language instructions. However, action labels are not widely used to boost learning processes because of problems that include (1) real-time annotation for online manipulation, (2) temporal inconsistency by annotators, (3) difference in data characteristics of motor commands and action labels, and (4) annotation cost. To address these problems, we propose the Gated Action Motor Predictive Learning (GAMPL) framework to leverage action labels for improved performance. GAMPL has two modules to obtain soft action labels compatible with motor commands and to generate motion. In this study, GAMPL is evaluated for towel-folding manipulation tasks in a real environment with a six degrees-of-freedom (6 DoF) robot and shows improved generalizability with action labels.

To extend the coverage of people able to receive high-quality rehabilitation, remote rehabilitation is required in addition to traditional face-to-face rehabilitation. Although remote rehabilitation using video conferencing systems has been realized to date, communication through physical sensations such as detailed patient motoring information and manual instructions from the therapist has not yet been realized. Therefore, the ultimate goal of this study was to develop multimodal wearable sensor system to support remote rehabilitation with a somatosensory system. To this end, we conducted a basic study of sensing technology in multimodal wear. Multiple strain sensors were attached to the shoulder to digitize the detailed behavior of the shoulder girdle. It was confirmed that the movement of the scapula can be acquired by this strain sensor. Furthermore, it was confirmed that the combination of strain sensors and an inertia measurement unit can be applied for the motion estimation of the entire upper limb.

Depth perception is pivotal in many fields, such as robotics and autonomous driving, to name a few. Consequently, depth sensors such as LiDARs rapidly spread in many applications. The 3D point clouds generated by these sensors must often be coupled with an RGB camera to understand the framed scene semantically. Usually, the former is projected over the camera image plane, leading to a sparse depth map. Unfortunately, this process, coupled with the intrinsic issues affecting all the depth sensors, yields noise and gross outliers in the final output. Purposely, in this paper, we propose an effective unsupervised framework aimed at explicitly addressing this issue by learning to estimate the confidence of the LiDAR sparse depth map and thus allowing for filtering out the outliers. Experimental results on the KITTI dataset highlight that our framework excels for this purpose. Moreover, we demonstrate how this achievement can improve a wide range of tasks.

We present an unsupervised simultaneous learning framework for the task of monocular camera re-localization and depth estimation from unlabeled video sequences. Monocular camera re-localization refers to the task of estimating the absolute camera pose from an instance image in a known environment, which has been intensively studied for alternative localization in GPS-denied environments. In recent works, cam-era re-localization methods are trained via supervised learning from pairs of camera images and camera poses. In contrast to previous works, we propose a completely unsupervised learning framework for camera re-localization and depth estimation, requiring only monocular video sequences for training. In our framework, we train two networks that estimate the scene coordinates using directions and the depth map from each image which are then combined to estimate the camera pose. The networks can be trained through the minimization of loss functions based on our loop closed view synthesis. In experiments with the 7-scenes dataset, the proposed method outperformed the re-localization of the state-of-the-art visual SLAM, ORB-SLAM3. Our method also outperforms state-of-the-art monocular depth estimation in a trained environment.

Unsupervised domain adaptation for point cloud semantic segmentation has attracted great attention due to its effectiveness in learning with unlabeled data. Most of existing methods use global-level feature alignment to transfer the knowledge from the source domain to the target domain, which may cause the semantic ambiguity of the feature space. In this paper, we propose a graph-based framework to explore the local-level feature alignment between the two domains, which can reserve semantic discrimination during adaptation. Specifically, in order to extract local-level features, we first dynamically construct local feature graphs on both domains and build a memory bank with the graphs from the source domain. In particular, we use optimal transport to generate the graph matching pairs. Then, based on the assignment matrix, we can align the feature distributions between the two domains with the graph-based local feature loss. Furthermore, we consider the correlation between the features of different categories and formulate a category-guided contrastive loss to guide the segmentation model to learn discriminative features on the target domain. Extensive experiments on different synthetic-to-real and real-to-real domain adaptation scenarios demonstrate that our method can achieve state-of-the-art performance. Our code is available at https://github.com/BianYikai/PointUDA.

Flying animals possess highly complex physical characteristics and are capable of performing agile maneuvers using their wings. The flapping wings generate complex wake structures that influence the aerodynamic forces, which can be difficult to model. While it is possible to model these forces using fluidstructure interaction, it is very computationally expensive and difficult to formulate. In this paper, we follow a simpler approach by deriving the aerodynamic forces using a relatively small number of states and presenting them in a simple state-space form. The formulation utilizes Prandtl's lifting line theory and Wagner's function to determine the unsteady aerodynamic forces acting on the wing in a simulation, which then are compared to experimental data of the bat-inspired robot called the Aerobat. The simulated trailingedge vortex shedding can be evaluated from this model, which then can be analyzed for a wake-based gait design approach to improve the aerodynamic performance of the robot.

The article focuses on acquiring a 3D radiation map of a building via a two-phase survey performed with an unmanned aircraft system (UAS). First, a model of the stud-ied building is created by means of photogrammetry. Then, radiation data are collected using a 2-inch NaI(Tl) detector in a regular grid at a distance of 2 m from all accessible surfaces of the building (i.e., the walls and the roof). The data are then georeferenced, filtered, projected to the building model, and interpolated to yield the detailed radiation map. A method to estimate the parameters of the radiation sources located inside is introduced and successfully tested, providing a localization accuracy in the order of meters. This task is aimed to deliver the proof of concept for employing such a mapping technique within nuclear safeguards. The acquisition of the radiation data was performed via a manual flight to ensure an appropriate safety level; in this context, it should be noted that the autonomous flight mode still requires major improvements in terms of safety.