IROS 2021 Papers

We present an approach for radar-inertial odometry which uses a continuous-time framework to fuse measurements from multiple automotive radars and an inertial measurement unit (IMU). Adverse weather conditions do not have a significant impact on the operating performance of radar sensors unlike that of camera and LiDAR sensors. Radar’s robustness in such conditions and the increasing prevalence of radars on passenger vehicles motivate us to look at the use of radar for ego-motion estimation. A continuous-time trajectory representation is applied not only as a framework to enable heterogeneous and asynchronous multi-sensor fusion, but also, to facilitate efficient optimization by being able to compute poses and their derivatives in closed-form and at any given time along the trajectory. We compare our continuous-time estimates to those from a discrete-time radar-inertial odometry approach and show that our continuous-time method outperforms the discrete-time method. To the best of our knowledge, this is the first time a continuous-time framework has been applied to radar-inertial odometry.

In this paper, we extend a famous motion planning approach, GPMP2, to multi-robot cases, yielding a novel centralized trajectory generation method for the multi-robot formation. A sparse Gaussian Process model is employed to represent the continuous-time trajectories of all robots as a limited number of states, which improves computational efficiency due to the sparsity. We add constraints to guarantee collision avoidance between individuals as well as formation maintenance, then all constraints and kinematics are formulated on a factor graph. By introducing a global planner, our proposed method can generate trajectories efficiently for a team of robots which have to get through a width-varying area by adaptive formation change. Finally, we provide the implementation of an incremental replanning algorithm to demonstrate the online operation potential of our proposed framework. The experiments in simulation and real world illustrate the feasibility, efficiency and scalability of our approach.

Maintaining lateral and longitudinal trajectory tracking accuracy is challenging for autonomous ground vehicles (AGVs). This paper considers kinematics and dynamics of longitudinal and lateral motion to form a novel composite structure considering the cross-impacts of acceleration and steering commands on tracking errors in the lateral and longitudinal directions, respectively. The multi-tiered structure uses backstepping with smooth robust control to iteratively map kinematics-based velocity and yaw rate commands to slip-yaw dynamics-based acceleration and steering commands. In kinematics, longitudinal tracking error is stabilized by sliding mode control (SMC) while variable structure control (VSC) stabilizes lateral tracking error and balances tracking accuracy and steering gracefulness. Backstepping extends these commands through vehicle dynamics to provide robust steering and acceleration commands. Cross impacts between lateral and longitudinal motion is addressed by vehicle modeling and controller designs. A robust observer is applied for sideslip estimation to reject uncertainties. Peaking from the high gain observer and robust control is addressed. Stability analysis is provided and field experiments on an open road demonstrate and validate effectiveness of the controllers.

Autonomous vehicles on city roads and especially in pedestrian environments require agility to navigate narrow passages and turn in tight spaces, leading to the need for a real-time, robust and adaptable controller. In this paper, we present orientation and context aware controllers for autonomous vehicles that can closely track the reference path wit alh respect to the current state of the vehicle, environmental properties, and the desired target orientation at the desired target location. Our proposed controllers are derived from the widely used pure pursuit controller. We validate our proposed controllers with respect to the baseline pure pursuit controller in simulation and on a full-size autonomous vehicle in a pedestrian environment. Our experimental results suggest significant improvements in adaptability and tracking performance compared to the pure pursuit controller.

Synthetic data generation is an appealing approach to generate novel traffic scenarios in autonomous driving. However, deep learning perception algorithms trained solely on synthetic data encounter serious performance drops when they are tested on real data. Such performance drops are commonly attributed to the domain gap between real and synthetic data. Domain adaptation methods that have been applied to mitigate the aforementioned domain gap achieve visually appealing results, but usually introduce semantic inconsistencies into the translated samples. In this work, we propose a novel, unsupervised, end-to-end domain adaptation network architecture that enables semantically consistent sim2real image transfer. Our method performs content disentanglement by employing shared content encoder and fixed style code.

We present a novel, low cost framework for reconstructing surface contact movements during in-hand manipulations. Unlike many existing methods focused on hand pose tracking, ours models the behavior of contact patches, and by doing so is the first to obtain detailed contact tracking estimates for multi-contact manipulations. Our framework is highly accessible, requiring only low cost, readily available paint materials, a single RGBD camera, and a simple, deterministic interpolation algorithm. Despite its simplicity, we demonstrate the framework’s effectiveness over the course of several manipulations on three common household items. Finally, we demonstrate the use of a generated contact time series in manipulation learning for a simulated robot hand.

In this paper, we present a framework that unites obstacle avoidance and deliberate physical interaction for robotic manipulators. As humans and robots begin to coexist in work and household environments, pure collision avoidance is insufficient, as human–robot contact is inevitable and, in some situations, desired. Our work enables manipulators to anticipate, detect, and act on contact. To achieve this, we allow limited deviation from the robot’s original trajectory through velocity reduction and motion restrictions. Then, if contact occurs, a robot can detect it and maneuver based on a novel dynamic contact thresholding algorithm. The core contribution of this work is dynamic contact thresholding, which allows a manipulator with onboard proximity sensors to track nearby objects and reduce contact forces in anticipation of a collision. Our framework elicits natural behavior during physical human–robot interaction. We evaluate our system on a variety of scenarios using the Franka Emika Panda robot arm; collectively, our results demonstrate that our contribution is not only able to avoid and react on contact, but also anticipate it.

Motion planning under uncertainty is of significant importance for safety-critical systems such as autonomous vehicles. Such systems have to satisfy necessary constraints (e.g., collision avoidance) with potential uncertainties coming from either disturbed system dynamics or noisy sensor measurements. However, existing motion planning methods cannot efficiently find the robust optimal solutions under general nonlinear and non-convex settings. In this paper, we formulate such problem as chance-constrained Gaussian belief space planning and propose the constrained iterative Linear Quadratic Gaussian (CILQG) algorithm as a real-time solution. In this algorithm, we iteratively calculate a Gaussian approximation of the belief and transform the chance-constraints. We evaluate the effectiveness of our method in simulations of autonomous driving planning tasks with static and dynamic obstacles. Results show that CILQG can handle uncertainties more appropriately and has faster computation time than baseline methods.

Soft fluidic actuators are increasingly being used for wearable haptic devices due to their high energy density and low encumbrance. These actuators are typically controlled using constant fluidic pressure control (CFPC), where the actuator pressure is switched between a high pressure source and atmospheric pressure using a fluidic valve. However, this type of control has several limitations for soft actuators including limited dynamic range, slow actuator response, low pressure control resolution and unnatural haptic interaction. In this paper, we present a novel control strategy for soft fluidic actuators, called constant fluidic mass control (CFMC), where the mass of fluid introduced into the actuator is kept constant during actuation, rather than the pressure as in CFPC. Our experimental results show that compared to CFPC, CFMC results in a larger dynamic range of actuator output forces, faster actuator response time to reach a desired target pressure, and higher resolution of pressure control, which makes it particularly useful for wearable haptics. In addition, CFMC enables analog pressure control and we present a neural-network-based supervised learning algorithm for accurate pressure control of soft actuators. Results show that our algorithm can predict actuator pressure with an accuracy of 99% and can be generalized to different soft TPU-fabric fluidic actuators.

We construct a Virtual Kinematic Chain (VKC) that readily consolidates the kinematics of the mobile base, the arm, and the object to be manipulated in mobile manipulations. Accordingly, a mobile manipulation task is represented by altering the state of the constructed VKC, which can be converted to a motion planning problem, formulated and solved by trajectory optimization. This new VKC perspective of mobile manipulation allows a service robot to (i) produce well-coordinated motions, suitable for complex household environments, and (ii) perform intricate multi-step tasks while interacting with multiple objects without an explicit definition of intermediate goals. In simulated experiments, we validate these advantages by comparing the VKC-based approach with baselines that solely optimize individual components. The results manifest that VKC-based joint modeling and planning promote task success rates and produce more efficient trajectories.

In the present work we address the problem of achieving a consistent estimator for SLAM. We propose a novel method capable of computing approximately consistent global uncertainties without scaling in complexity with the total size of the explored area. The method allows arbitrary selection of local areas for optimization, introducing a methodology for building a virtual prior in bounded time. The constructed prior topologies serve as a way of contextualizing the local optimizations, resulting in consistent uncertainty estimations. This overcomes several shortcomings of previous approaches that rely on conditioning (fixing variables) and/or sliding window marginalization. Evaluations are presented in different simulated scenarios comparing results against a global batch optimization and other canonical approaches.

Building object-level maps can facilitate robot-environment interactions (e.g. planning and manipulation), but objects could often have multiple probable poses when viewed from a single vantage point, due to symmetry, occlusion or perceptual failures. A robust object-level simultaneous localization and mapping (object SLAM) algorithm needs to be aware of this pose ambiguity. We propose to maintain and subsequently disambiguate the multiple pose interpretations to gradually recover a globally consistent world representation. The max-mixtures model is applied to implicitly and efficiently track all pose hypotheses, but the resulting formulation is non-convex, and therefore subject to local optima. To mitigate this problem, temporally consistent hypotheses are extracted, guiding the optimization into the global optimum. This consensus-informed inference method is applied online via landmark variable re-initialization within an incremental SLAM framework, iSAM2, for robust real-time performance. We demonstrate that this approach improves SLAM performance on both simulated and real object SLAM problems with pose ambiguity.

Deep Reinforcement Learning has emerged as an efficient dynamic obstacle avoidance method in highly dynamic environments. It has the potential to replace overly conservative or inefficient navigation approaches. However, integrating Deep Reinforcement Learning into existing navigation systems is still an open frontier due to the myopic nature of Deep-Reinforcement-Learning-based navigation, which hinders its widespread integration into current navigation systems. In this paper, we propose the concept of an intermediate planner to interconnect novel Deep-Reinforcement-Learning-based obstacle avoidance with conventional global planning methods using waypoint generation. Therefore, we integrate different waypoint generators into existing navigation systems and compare the joint system against traditional ones. We found an increased performance in terms of safety, efficiency and path smoothness, especially in highly dynamic environments.

Evaluating a grasp generated by a set of hand-object contact locations is a key component of many grasp planning algorithms. In this paper, we present a novel second-order cone program (SOCP) based optimization formulation for evaluating a grasps’ ability to apply wrenches to generate a linear motion along a given direction and/or an angular motion about the given direction. Our quality measure can be computed efficiently since the SOCP is a convex optimization problem, which can be solved optimally with interior point methods. A key feature of our approach is that we can consider the effect of contact wrenches from any contact of the object with the environment. This is different from the extant literature where only the effect of finger-object contacts is considered. Exploiting the environmental contact is useful in many manipulation scenarios either to enhance the dexterity of simple hands or improve the payload capability of the manipulator. In contrast to most existing approaches, our approach also takes into account the practical constraint that the maximum contact force that can be applied at a finger-object contact can be different for each contact. We can also include the effect of external forces like gravity, as well as the joint torque constraints of the fingers/manipulators. Furthermore, for a given motion path as a constant screw motion or a sequence of constant screw motions, we can discretize the path and compute a global grasp metric to accomplish the whole task with a chosen set of finger-object contact locations.

This paper proposes a novel and practical approach to enhance the computational efficiency of the hierarchical quadratic programming (HQP)-based whole-body control. The HQP method is known to offer control solutions satisfying strict priority with various constraints for multiple-tasks execution. However, it inherently comes at the price of high computation time to solve QP optimization problems in each hierarchical level which limits practicability in a real-time control system with fast sampling time. To mitigate this issue, we propose that the operational space formulation is incorporated into the HQP method, where the decision variables are intuitively defined at the task level and possess smaller dimensions. Indeed, it serves faster whole-body control solution for multiple tasks under equality and inequality constraints yet strictly fulfilling the task priority. The performance of the pro-posed method is experimentally verified on the actual floating-based humanoid, named TOCABI with 33 degrees-of-freedom. In addition, computation time is analyzed by comparison with conventional HQP and other advanced implementation forms.

We propose a hierarchical control framework for generating versatile motions by a humanoid robot. The central feature of our framework is computational affordability: a large amount of computation time is allowable in the upper-level hierarchy. Consequently, whole-body trajectory optimization for a long time horizon becomes feasible. To ensure such affordability, a fast feedback loop is established in the lower-level hierarchy to increase the robustness against the large latency in the upper level. We experimentally examined the advantages of the achieved computational affordability. Our framework allowed a large computational time of 100 ms in each control cycle. This enables online trajectory optimization to predict 50 time steps ahead while taking full-body dynamics into account. Due to such a long prediction range, 20 motions were successfully generated in real time with our computation-ally affordable framework.

We present an optimization-based structural-parametric synthesis method for reconfigurable closed-chain underactuated linkages for robotic systems that physically interact with the environment with an emphasis on adaptive grasping. The key idea is to implement morphological computation concepts to keep both necessary trajectory-specific holonomic constraints and mechanism adaptivity using variable length links (VLL), while we evolve from a fully actuated to an underactuated system satisfying imposed design requirements. It allows to minimize the number of actuators, weight, and cost but keep high payload and endurance that are not reachable by tendon-driven designs. Despite the method is general enough, for clarity, we demonstrate its use on a number of finger mechanisms for adaptive grippers.

Autonomous industrial assembly relies on the precise measurement of spatial constraints as designed by computer-aided design (CAD) software such as SolidWorks. This paper proposes a framework for an intelligent industrial robot to understand the spatial constraints for model assembly. An extended generative adversary network (GAN) with a 3D long short-term memory (LSTM) network was designed to composite 3D point clouds from a single RGB-D scan. The spatial constraints of the segmented point clouds are identified by a neural-logic network that incorporates general knowledge of spatial constraints in terms of first-order logic. The model was designed to comprehend a complete set of spatial constraints that are consistent with industrial CAD software, including left, right, above, below, front, behind, parallel, perpendicular, concentric, and coincident relations. The accuracy of 3D model composition and spatial constraint identification was evaluated by the RGB-D scans and 3D models in the ABC dataset. The proposed model achieved 57.23% intersection over union (IoU) in 3D model composition, and over 99% in comprehending all spatial constraints.

Artificial potential fields (APFs) and their variants have been a staple for collision avoidance of mobile robots and manipulators for almost 40 years. Its model-independent nature, ease of implementation, and real-time performance have played a large role in its continued success over the years. Control barrier functions (CBFs), on the other hand, are a more recent development, commonly used to guarantee safety for nonlinear systems in real-time in the form of a filter on a nominal controller. In this paper, we address the connections between APFs and CBFs. At a theoretic level, we show that given a broad class of APFs, one can construct a CBF that guarantees safety. Additionally, we prove that CBFs obtained from these APFs have additional beneficial properties and can be applied to nonlinear systems. Practically, we compare the performance of APFs and CBFs in the context of obstacle avoidance on simple illustrative examples and for a quadrotor with unknown dynamics, both in simulation and on hardware using onboard sensing.