IROS 2021 Papers

Pose estimation is a key challenge in robot manipulation and grasping task. Current object pose estimation approaches based on 3D models and depth sensor information have difficulties to handle transparent objects because of the limitation to capture the accurate depth information. To address these issues, we present a 6DoF pose estimation approach, called GhostPose, which utilizes a novel 3D bounding box prediction network and multi-view geometry with cameras on manipulator robot. Our 3D bounding box prediction network is simple and light-weight by adding a small branch to a one-stage object detector. The network detects 2D projections of 3D bounding box vertices. Then, 3D points are reconstructed from the 2D results of the multiple viewpoints with camera motion information, i.e. extrinsic parameters, calculated from the robot joint angles. We also present generalized pose definition to address pose ambiguity of symmetric objects and keep consistency of geometric properties around feature points across both of the symmetric and asymmetric objects. Comparing with the previous pose estimation approaches, GhostPose is more generalized to environments and object types, because it does not require 3D models, object specific key points, predefined stereo settings and depth map. In experiments, it outperforms a state-of-the-art approach and shows generalized properties by applying to a real manipulator robot grasping system.

Lifelong SLAM considers long-term operation of a robot where already mapped locations are revisited many times in changing environments. As a result, traditional graph-based SLAM approaches eventually become extremely slow due to the continuous growth of the graph and the loss of sparsity. Both problems can be addressed by a graph pruning algorithm. It carefully removes vertices and edges to keep the graph size reasonable while preserving the information needed to provide good SLAM results. We propose a novel method that considers geometric criteria for choosing the vertices to be pruned. It is efficient, easy to implement, and leads to a graph with evenly spread vertices that remain part of the robot trajectory. Furthermore, we present a novel approach of marginalization that is more robust to wrong loop closures than existing methods. The proposed algorithm is evaluated on two publicly available real-world long-term datasets and compared to the unpruned case as well as ground truth. We show that even on a long dataset (25h), our approach manages to keep the graph sparse and the speed high while still providing good accuracy (40 times speed up, 6cm map error compared to unpruned case).

Whole-arm manipulation (WAM) is often used to manipulate large and bulky objects. Contact point-based methods for generating the robot configurations for WAM mostly search for suitable contact points and configurations simultaneously. However, in order to learn good contact points, or allow an operator to select them, inverse kinematics (IK) solvers are needed which take such points along with a surface normal into account. Therefore, we propose a geometry-based IK method for WAM generate robot configurations in contact with two specific points. The method is compared against two other methods, one numerical method and a commercial IK solver. We show our method to be the fastest.

We propose an autonomous grasping pipeline that relies on geometric information extracted from segmented point cloud data. This is in contrast to many recent approaches leveraging deep learning and thus relying on a rather large amount of training samples. We argue that the proposed geometric approach facilitates task-level planning as the shape, size, and symmetry of objects can be directly taken into account during the planning process that utilizes the new MoveIt! Task Constructor (MTC) framework to define and plan action sequences composed of several inter-related sub-tasks. The efficiency of the proposed grasping pipeline is illustrated in pick-and-place scenarios, including a long-distance pick-and-place requiring a hand-over between two hands.

Traditional geometric mechanics models used in locomotion analysis rely heavily on systems having symmetry in SE(2) (i.e., the dynamics and constraints are invariant with respect to a system’s position and orientation) to simplify motion planning. As a result, the symmetry assumption prevents locomotion analysis on non-flat surfaces because the system dynamics may vary as a function of position and orientation. In this paper, we develop geometric motion planning strategies for a mobile system moving on a position space whose manifold structure is a cylinder: constant non-zero curvature in one dimension and zero curvature in another. To handle this non-flat position space, we adapt conventional geometric mechanics tools - in particular the system connection and the constraint curvature function - to depend on the system orientation. In addition, we introduce a novel constraint projection method to a variational gait optimizer and demonstrate how to design gaits that allow the example system to move on the cylinder with optimal efficiency.

Natural and human-like arm motions are promising features to facilitate social understanding of humanoid robots. To this end, we integrate biophysical characteristics of human arm-motions into sampling-based motion planning. We show the generality of our method by evaluating it with multiple manipulators. Our first contribution is to introduce a set of cost functions to optimize for human-like arm postures during collision-free motion planning. In a subsequent step, an optimization phase is used to improve the human-likeness of the initial path. Additionally, we present an interpolation approach for generating obstacle-aware and multi-modal velocity profiles. We thus generate collision-free and human-like motions in narrow passages while allowing for natural acceleration in free space.

Intelligent robots are redefining a multitude of critical domains but are still far from being fully capable of assisting human peers in day-to-day tasks. An important requirement of collaboration is for each teammate to maintain and respect an understanding of the others’ expectations of itself. Lack of which may lead to serious issues such as loose coordination between teammates, reduced situation awareness, and ultimately teaming failures. Hence, it is important for robots to behave explicably by meeting the human’s expectations. One of the challenges here is that the expectations of the human are often hidden and can change dynamically as the human interacts with the robot. However, existing approaches to generating explicable plans often assume that the human’s expectations are known and static. In this paper, we propose the idea of active explicable planning to relax this assumption. We apply a Bayesian approach to model and predict dynamic human belief and expectations to make explicable planning more anticipatory. We hypothesize that active explicable plans can be more efficient and explicable at the same time, when compared to explicable plans generated by the existing methods. In our experimental evaluation, we verify that our approach generates more efficient explicable plans while successfully capturing the dynamic belief change of the human teammate.

Imitation Learning (IL) is an effective framework to learn visuomotor skills from offline demonstration data. However, IL methods often fail to generalize to new scene configurations not covered by training data. On the other hand, humans can manipulate objects in varying conditions. Key to such capability is hand-eye coordination, a cognitive ability that enables humans to adaptively direct their movements at task-relevant objects and be invariant to the objects’ absolute spatial location. In this work, we present a learnable action space, Hand-eye Action Networks (HAN) that learns coordinated hand-eye movements from human teleoperated demonstrations. Through a set of challenging multi-stage manipulation tasks, we show that a visuomotor policy equipped with HAN is able to inherit the key spatial invariance property of handeye coordination and achieve generalization to new scene configurations. Additional materials available at https://sites.google.com/stanford.edu/han

Optimization of gear ratios for dual-motor actuators is presented for the development of a walker-type assist robot. The robot is reconfigurable to provide an elderly user with multiple physical support functions; one is to assist sitto-stand transitions and the other is to serve as a walker to aid the user in walking. To avoid falling while walking, the robot must react quickly and reconfigure its footprint for supporting the user. This requires high speed actuators. In contrast, for assisting sit-to-stand transitions, high torque actuators are required. To meet the bimodal, conflicting load conditions, i.e. high-torque, low-speed v.s. high-speed, low-torque, this paper presents a dual-motor actuation solution, where two motors with diverse gear ratios are combined. The system is characterized with two key parameters: an internal gear ratio between a high-speed motor and a torque-booster, and an external gear ratio for connecting the dual motor actuator to the load. The internal and external gear ratios are optimized for performing both sit-to-stand assist and rapid foot reconfiguration.

Autonomous vehicles may not behave as expected when subject to environmental disturbances. For instance, control commands suitable for driving on dry, paved roads may lead to unsafe conditions and undesired deviations when on slippery dirt or icy roads. Furthermore, it becomes increasingly important to offer human-understandable explanations of autonomous robots’ actions – especially when they operate in a space shared with humans such as public roads. In this work, we present an interpretable, risk-aware online adaptation approach that leverages a combination of Gaussian process regression and decision tree theory. Our approach predicts the system state in unknown environments using pre-trained models, which are continuously refined during runtime using a fast lookup table-based procedure. Every time the originally intended behavior is predicted to cause an unsafe state, our method computes a set of safe behaviors under the current model uncertainty and risk. The most suitable behavior is subsequently selected depending on mission requirements and a human-understandable explanation about the system’s decision-making is delivered. The proposed framework is applied to unmanned ground vehicles and validated in simulations, in which we demonstrate a successful safety-critical cargo delivery through a priori unknown, rough and slippery terrain.

Research in powered prostheses and orthoses has relied on COP measurements to inform a device’s controller about the body’s progression through the gait cycle, and to provide sensory substitution for prosthesis users, thereby helping them maintain balance during locomotion. Obtaining accurate COP measurements in out-of-the-lab contexts currently requires pressure sensitive insoles with dense arrays of sensing elements, which are expensive and bulky, limiting the accessibility and scalability of this technology. In this paper, we present a new method to reconstruct COP trajectories in over-ground walking tasks, using an affordable sensor array with eight sensing elements embedded in shoe insoles. The method leverages Gaussian Process Regression (GPR) to perform predictions from raw sensor data using Bayesian inference. A preliminary validation was carried out with a convenience sample of healthy individuals and patients with neuromuscular disorders. Combined mediolateral (ML) and anteroposterior (AP) errors where 2% and 3% for healthy individuals and patients, respectively. The analysis evidenced larger stride-to-stride variability in the ML COP excursion for the patient group, suggesting higher levels of motor noise associated with selective muscle weakness. These promising results indicate the potential of the proposed method to accurately estimate COP trajectories for future applications in wearable robotics and out-of-the-lab clinical gait assessments.

Fast and robust gate perception is of great importance in autonomous drone racing. We propose a convolutional neural network-based gate detector (GateNet1) that concurrently detects gate’s center, distance, and orientation with respect to the drone using only images from a single fish-eye RGB camera. GateNet achieves a high inference rate (up to 60 Hz) on an onboard processor (Jetson TX2). Moreover, GateNet is robust to gate pose changes and background disturbances. The proposed perception pipeline leverages a fish-eye lens with a wide field-of-view and thus can detect multiple gates in close range, allowing a longer planning horizon even in tight environments. For benchmarking, we propose a comprehensive dataset (AU-DR) that focuses on gate perception. Throughout the experiments, GateNet shows its superiority when compared to similar methods while being efficient for onboard computers in autonomous drone racing. The effectiveness of the proposed framework is tested on a fully-autonomous drone that flies on previously-unknown track with tight turns and varying gate positions and orientations in each lap.

Novel methods for the inference of radiation intensity functions defined over known surfaces are proposed, intended for use in surveying applications with mobile spectrometers. Previous approaches, based on the maximum likelihood expectation maximization (ML-EM) framework with Poisson likelihoods, are extended to better handle spatially continuous intensity statistics using ideas from Gaussian filtering. The resulting algorithm is evaluated against a classical ML-EM method, and a recently proposed sparse additive point source localization (APSL) algorithm in a Monte-Carlo simulation study. The new generalized ASPL (GASPL) is shown to compare favorably in terms of estimation accuracy when the true intensity is not well described by a set of point sources. Finally, the GASPL is used in an experiment where a detector is mounted to an unmanned aerial vehicle to estimate the intensity and location of radioactive sources placed in a meadow.

It is important for autonomous vehicles to have the ability to infer the goals of other vehicles (goal recognition), in order to safely interact with other vehicles and predict their future trajectories. This is a difficult problem, especially in urban environments with interactions between many vehicles. Goal recognition methods must be fast to run in real time and make accurate inferences. As autonomous driving is safety- critical, it is important to have methods which are human interpretable and for which safety can be formally verified. Existing goal recognition methods for autonomous vehicles fail to satisfy all four objectives of being fast, accurate, interpretable and verifiable. We propose Goal Recognition with Interpre table Trees (GRIT), a goal recognition system which achieves these objectives. GRIT makes use of decision trees trained on vehicle trajectory data. We evaluate GRIT on two datasets, showing that GRIT achieved fast inference speed and comparable accuracy to two deep learning baselines, a planning-based goal recognition method, and an ablation of GRIT. We show that the learned trees are human interpretable and demonstrate how properties of GRIT can be formally verified using a satisfiability modulo theories (SMT) solver.

This paper presents a tightly coupled pipeline, which efficiently fuses measurements of LiDAR, camera, IMU, encoder, and GNSS to estimate the robot state and build a map even in challenging situations. The depth of visual features is extracted by projecting the LiDAR point cloud and ground plane into image. We select the tracked high-quality visual features and LiDAR features and tightly coupled the pre-integrated values of the IMU and the encoder to optimize the state increment of a robot. We use the estimated relative pose to re-evaluate the matching distance between features in the local window and remove dynamic objects and outliers. In the mapping node, we use refined features and tightly coupled the GNSS measurements, increment factors, and local ground constraints to further refine the robot’s global state by aligning LiDAR features with the global map. Furthermore, the method can detect sensor degradation and automatically reconfigure the optimization process. Based on a six-wheeled ground robot, we perform extensive experiments in both indoor and outdoor environments and demonstrated that the proposed GR-Fusion outperforms state-of-the-art SLAM methods in terms of accuracy and robustness.

Legged robots require fast and accurate representation of their surrounding terrain to achieve behaviors such as running, push recovery, continuous walking, backflips, while also utilizing on-board computational resources efficiently. The desired tasks can be achieved efficiently by representing the environment using planar regions. However, existing methods for planar region extraction are either too slow or require significant compute time on the Central Processing Unit (CPU). In this work we exploit key properties of depth images and Graphical Processing Unit (GPU) to estimate planar regions around the robot at very high frame rates of 150-200 Hz. The proposed algorithm uses a set of fully customizable and interchangeable set of kernel layers on the GPU to process the depth map in parallel and generate a locally connected graph structure, which is later separated into planar components using a basic depth-first search. We test the proposed algorithm on the Atlas robot while performing different walking behaviors on oriented cinder blocks, as well as in simulation with simulated sensor and robot. The algorithm is open-sourced for research on legged robots and other fields.

Autonomous grasping is an important factor for robots physically interacting with the environment and executing versatile tasks. However, a universally applicable, cost-effective, and rapidly deployable autonomous grasping approach is still limited by those target objects with fuzzy-depth information. Examples are transparent, specular, flat, and small objects whose depth is difficult to be accurately sensed. In this work, we present a solution to those fuzzy-depth objects. The framework of our approach includes two major components: one is a soft robotic hand and the other one is a Fuzzy-depth Soft Grasping (FSG) algorithm. The soft hand is replaceable for most existing soft hands/grippers with body compliance. FSG algorithm exploits both RGB and depth images to predict grasps while not trying to reconstruct the whole scene. Two grasping primitives are designed to further increase robustness. The proposed method outperforms reference baselines in unseen fuzzy-depth objects grasping experiments (84% success rate).

In this paper, a fuzzy PID controller based on yaw angle prediction is applied to design an attitude controller for a spherical rolling robot. The robot consists of a 2-DOF pendulum located inside a spherical shell with freedom to rotate about the transversal and longitudinal axis. The proposed controller allows the robot to autonomously change its parameters to adapt to different environments based on current state. The past researches on the motion of spherical robots mostly focused on simulation or ideal experimental environment. But in this paper, a physical system is built and experiments are carried out to demonstrate the effectiveness, robustness and adaptability of the controller.

Frictional contact has been extensively studied as the core underlying behavior of legged locomotion and manipulation, and its nearly-discontinuous nature makes planning and control difficult even when an accurate model of the robot is available. Here, we present empirical evidence that learning an accurate model in the first place can be confounded by contact, as modern deep learning approaches are not designed to capture this non-smoothness. We isolate the effects of contact’s non-smoothness by varying the mechanical stiffness of a compliant contact simulator. Even for a simple system, we find that stiffness alone dramatically degrades training processes, generalization, and data-efficiency. Our results raise serious questions about simulated testing environments which do not accurately reflect the stiffness of rigid robotic hardware. Significant additional investigation will be necessary to fully understand and mitigate these effects, and we suggest several avenues for future study.