IROS 2021 Papers

Two main challenges that need to be addressed in physical human-robot interaction (pHRI) are efficient recognition of human intention and interaction safety. In this paper, a general human intention framework was summarized, firstly, according to the robot's roles: a passive follower and a compliant leader. Secondly, we proposed variable admittance control models governed by human intentions. Power envelope approaches were then proposed to impose constraints on the variable admittance parameters inferred from human intention for maintaining passivity conservatively. Our passivity preserving approaches were validated via simulation and shown to avoid mismatching of time-varying admittance parameters that restrain drastic changes of admittance controller dynamics, which usually result in instability. Finally, the relationship between the robot's passivity and stability when it interacts with the human was analyzed.

Robot hand is essential for a fully functional robot and designing a good robot hand is a sophisticated job that challenges the designer’s knowledge and experience. This paper presents a computational framework for automatic optimal robot hand design based on reinforcement learning (RL), which considers desired grasping tasks, grasp control strategies, and performance quality measures altogether. The RL-based framework intends to grow finger joints with different types and link lengths at different positions from null. Then, the reward function for such a growing action is defined in terms of quality indexes of the generated robot hand to perform desired grasping tasks under expected control strategies. To demonstrate the effectiveness of this framework, in this paper we set the desired task to simply grasping objects of three primitive shapes (i.e., box, cylinder, and sphere) with predefined hand positions and strategies to close fingers to achieve grasps for each object. The force closure condition, quantitative stability indexes, and energy consumption of grasps as well as some penalty terms are used to assemble the reward function. Through simulation and practical prototype experiments, we show that capable robot hands can be automatically generated by the proposed framework. Potential factors that affect the output of the framework and deserve further exploration are also discussed.

In this paper, a compliant five-bar leg mechanism is proposed, designed and manufactured for heavy-load legged robots, by using two magneto-rheological actuators (MRAs) that are capable of offering a maximal torque of 78Nm. To address the rate-dependent hysteresis of the MRA, a hybrid rate-dependent hysteresis model is derived based on the idea of mappings between different hysteresis loops. With integrating the classical Preisach model and the NARX neural network, the hybrid model is able to model hysteresis nonlinearity of the magneto-rheological clutch (MRC). It is then used to estimate and control the output torque of the MRA at the absent of external force/torque sensors. High fidelity force control and variable compliance of the leg mechanism are realized and validated in various experiments with using the MRAs.

Advancing maturity in mobile and legged robotics technologies is changing the landscapes where robots are being deployed and found. This innovation calls for a transformation in simultaneous localization and mapping (SLAM) systems to support this new generation of service and consumer robots. No longer can traditionally robust 2D lidar systems dominate while robots are being deployed in multi-story indoor, outdoor unstructured, and urban domains with increasingly inexpensive stereo and RGB-D cameras. Visual SLAM (VSLAM) systems have been a topic of study for decades and a small number of openly available implementations have stood out: ORB-SLAM3, OpenVSLAM and RTABMap.This paper presents a comparison of these 3 modern, feature rich, and uniquely robust VSLAM techniques that have yet to be benchmarked against each other, using several different datasets spanning multiple domains negotiated by service robots. ORB-SLAM3 and OpenVSLAM each were not compared against at least one of these datasets previously in literature and we provide insight through this lens. This analysis is motivated to find general purpose, feature complete, and multi-domain VSLAM options to support a broad class of robot applications for integration into the new and improved ROS 2 Nav2 System as suitable alternatives to traditional 2D lidar solutions.

We present a collaborative visual simultaneous localization and mapping (SLAM) framework for service robots. With an edge server maintaining a map database and performing global optimization, each robot can register to an existing map, update the map, or build new maps, all with a unified interface and low computation and memory cost. We design an elegant communication pipeline to enable real-time information sharing between robots. With a novel landmark organization and retrieval method on the server, each robot can acquire landmarks predicted to be in its view, to augment its local map. The framework is general enough to support both RGB-D and monocular cameras, as well as robots with multiple cameras, taking the rigid constraints between cameras into consideration. The proposed framework has been fully implemented and verified with public datasets and live experiments.

This paper proposes the design of a robotic gripper motivated by the bin-picking problem, where a variety of objects need to be picked from cluttered bins. The presented gripper design focuses on an enveloping cage-like approach, which surrounds the object with three hooked fingers, and then presses into the object with a movable palm. The fingers are flexible and imbue grasps with some elasticity, helping to conform to objects and, crucially, adding friction to cases where an object cannot be caged. This approach proved effective on a set of basic shapes, such as cuboids and cylinders, in which every object could be grasped. In particular, flat bottom parts could be grasped in a very stable manner, as demonstrated by testing grasps with multiple 5N and 10N disturbances. A set of supermarket items were also tested, highlighting promising features such as effective grasping of fruits and vegetables, as well as some limitations in the current embodiment, which is not always able to slip the fingers underneath objects.

The nature animals have evolved highly efficient and robust organs that support their complex daily navigation tasks. To mimic animal’s navigation capability, we present a novel bio-inspired navigation system that draws inspirations from nature animals in this paper. The system consists of a three-axis magnetometer, a monocular camera, a micro inertial measurement unit (MIMU) and a polarization camera. While dead reckoning, orientation, and landmark recognition are considered as three most important capability for various species, we also designed corresponding algorithms based on the bio-inspired sensing system to perform autonomous navigation. In detail, the dead reckoning component is accomplished by integrating the monocular camera and the MIMU into a visual inertial odometry (VIO) and the orientation capability is achieved by combining the absolute orientation from the magnetometer with the relative orientation from the VIO. A loop closure detection is then used as the landmark recognition component to reduce the navigation drifts. All the three components are fused with a graph optimization method to generate the robust navigation result. To valid the proposed navigation sensing system and the algorithms, we have conducted series of experiments on ground and aerial unmanned vehicles, and have added orientation noise to testify the accuracy and robustness of the system.

In this work we propose 3D-FFS, a novel approach to make sensor fusion based 3D object detection networks significantly faster using a class of computationally inexpensive heuristics. Existing sensor fusion based networks generate 3D region proposals by leveraging inferences from 2D object detectors. However, as images have no depth information, these networks rely on extracting semantic features of points from the entire scene to locate the object. By leveraging aggregated intrinsic properties (e.g. point density) of point cloud data, 3D-FFS can substantially constrain the 3D search space and thereby significantly reduce training time, inference time and memory consumption without sacrificing accuracy. To demonstrate the efficacy of 3D-FFS, we have integrated it with Frustum ConvNet (F-ConvNet), a prominent sensor fusion based 3D object detection model. We assess the performance of 3D-FFS on the KITTI dataset. Compared to F-ConvNet, we achieve improvements in training and inference times by up to 62.80% and 58.96%, respectively, while reducing the memory usage by up to 58.53%. Additionally, we achieve 0.36%, 0.59% and 2.19% improvements in accuracy for the Car, Pedestrian and Cyclist classes, respectively. 3D-FFS shows a lot of promise in domains with limited computing power, such as autonomous vehicles, drones and robotics where LiDAR-Camera based sensor fusion perception systems are widely used.

The paper proposes a reliable and robust planning solution to the long range robotic navigation problem in extremely cluttered environments. A two-layer planning architecture is proposed that leverages both the environment map and the direct depth sensor information to ensure maximal information gain out of the onboard sensors. A frontier-based pose sampling technique is used with a fast marching cost-to-go calculation to select a goal pose and plan a path to maximize robot exploration rate. An artificial potential function approach, relying on direct depth measurements, enables the robot to follow the path while simultaneously avoiding small scene obstacles that are not captured in the map due to mapping and localization uncertainties. We demonstrate the feasibility and robustness of the proposed approach through field deployments in a structurally complex warehouse using a micro-aerial vehicle (MAV) with all the sensing and computations performed onboard.

Future urban transportation concepts include a mixture of ground and air vehicles with varying degrees of autonomy in a congested environment. In such dynamic environments, occupancy maps alone are not sufficient for safe path planning. Safe and efficient transportation requires reasoning about the 3D flow of traffic and properly modeling uncertainty. Several different approaches can be taken for developing 3D velocity maps. This paper explores a Bayesian approach that captures our uncertainty in the map given training data. The approach involves projecting spatial coordinates into a high-dimensional feature space and then applying Bayesian linear regression to make predictions and quantify uncertainty in our estimates. On a collection of air and ground datasets, we demonstrate that this approach is effective and more scalable than several alternative approaches.

Aerial vehicles are revolutionizing applications that require capturing the 3D structure of dynamic targets in the wild, such as sports, medicine and entertainment. The core challenges in developing a motion-capture system that operates in outdoors environments are: (1) 3D inference requires multiple simultaneous viewpoints of the target, (2) occlusion caused by obstacles is frequent when tracking moving targets, and (3) the camera and vehicle state estimation is noisy. We present a real-time aerial system for multi-camera control that can reconstruct human motions in natural environments without the use of special-purpose markers. We develop a multi-robot coordination scheme that maintains the optimal flight formation for target reconstruction quality amongst obstacles. We provide studies evaluating system performance in simulation, and validate real-world performance using two drones while a target performs activities such as jogging and playing soccer. Supplementary video: https://youtu.be/jxt91vx0cns

Estimating ocean flow fields in 3D is a critical step in enabling the reliable operation of underwater gliders and other small, low-powered autonomous marine vehicles. Existing methods produce depth-averaged 2D layers arranged at discrete vertical intervals, but this type of estimation can lead to severe navigation errors. Based on the observation that real-world ocean currents exhibit relatively low vertical velocity components, we propose an accurate 3D estimator that extends our previous work in estimating 2D flow fields as a linear combination of basis flows. The proposed algorithm uses data from ensemble forecasting to build a set of 3D basis flows, and then iteratively updates basis coefficients using point measurements of underwater currents. We report results from experiments using actual ensemble forecasts and synthetic measurements to compare the performance of our method to the direct 3D extension of the previous work. These results show that our method produces estimates with dramatically lower error metrics, with and without measurement noise.

This paper presents the design and evaluation of the low-cost capacitive proximity-force-fusion sensor "safe skin", which can measure simultaneously the proximity of humans as well as the contact force. It was designed such that the force and proximity sensing functions can work concurrently without interfering with each other. Moreover, active shielding, on-chip digitization and ground isolation are implemented for the sensor to minimize the influence from stray capacitance and electromagnetic interference (EMI) from the environment, which ensures that the sensor has a high system robustness for industrial applications. The prototype version has the capability of detecting a grounded human hand sized object from a distance of 400 mm. Moreover, forces in the range of 5 to 40 N can be measured, with 43.7% hysteresis and 6.7% nonlinearity. Due to its sensing characteristics, when used on a robot, the sensor could be used to ensure the safety of nearby humans in the future. The sensor could also potentially be used as an interface for human-robot interaction (HRI).

Inducing a user’s behavior through social interaction is a goal that a social robot aims to achieve. It has been argued that empathy has a strong effect on behavior inducement. In human-human interaction, it has been verified that the influence of a nonverbal cue on empathy outweighs that of a verbal cue when those are used in a combined way. The objectives of this study are to explore if such outweighing effect of nonverbal cues is maintained in human-robot interaction (HRI) and to investigate the mechanism of communication cues’ effects by analyzing the mediation structure with the following mediator variables: perceived emotion, perceived intentionality, perceived malfunction, and empathy inducement. To this end, we conducted 2 (verbal type: positive verbal cue vs. negative verbal cue) × 2 (nonverbal type: positive nonverbal cue vs. negative nonverbal cue) within-participant experiment (N = 48). The experiment created a situation in which the social robot was harshly criticized during a conversation. The analysis of experiment results showed the outweighing effect of a nonverbal cue was maintained. When a nonverbal cue conveyed a negative, situation-accordant emotion, it had a decisive effect on perceived emotion, empathy, and behavior inducement. In contrast, a verbal cue induced participants’ empathy and behavior when it conveyed a positive, situation-discordant emotion. This inconsistency between verbal cue and nonverbal cue made the combination of positive verbal cue and negative nonverbal cue have the strongest effect. It implies that participants had different expectations for each of the two communication cues, just as they did in social interactions with human beings. It implies that participants might have applied normative expectations for social interaction with human beings to the social robot.