ICRA 2022 Papers

Mobile manipulators for indoor human environments can serve as versatile devices that perform a variety of tasks, yet adoption of this technology has been limited. Reducing size, weight, and cost could facilitate adoption, but risks restricting capabilities. We present a novel design that reduces size, weight, and cost, while supporting a variety of tasks. The core design consists of a two-wheeled differential-drive mobile base, a lift, and a telescoping arm configured to achieve Cartesian motion at the end of the arm. Design extensions include a 1 degree-of-freedom (DOF) wrist to stow a tool, a 2-DOF dexterous wrist to pitch and roll a tool, and a compliant gripper. We justify our design with anthropometry and mathematical models of static stability. We also provide empirical support from teleoperating and autonomously controlling a commercial robot based on our design (the Stretch RE1 from Hello Robot Inc.) to perform tasks in real homes.

Wrist-driven orthotics have been designed to assist people with C6-7 spinal cord injury, however, the kinematic constraint imposed by such a control strategy can impede mobility and lead to abnormal body motion. This study characterizes body compensation using the novel Tenodesis Grasp Emulator, an adaptor orthotic that allows for the investigation of tenodesis grasping in subjects with unimpaired hand function. Subjects perform a series of grasp-and-release tasks in order to compare normal (test control) and constrained wrist-driven modes, showing significant compensation as a result of the constraint. A motor-augmented mode is also compared against traditional wrist-driven operation, to explore the potential role of hybrid human-robot control. We find that both the passive wrist-driven and motor-augmented modes fulfill different roles throughout various tasks tested. Thus, we conclude that a flexible control scheme that can alter intervention based on the task at hand holds the potential to reduce compensation in future work.

In this paper, we aim to apply deep saliency prediction to automatic drone exploration, which should consider not only one single image, but multiple images from different view angles or localizations in order to determine the exploration direction. However, little attention has been paid to such saliency prediction problem over multiple-discontinuous-image and none of existing methods take temporal information into consideration, which may mean that the current predicted saliency map is not consistent with the previous predicted results. For this purpose, we propose a method named Temporally-Aggregating Multiple-Discontinuous-Image Saliency Prediction Network (TA-MSNet). It utilizes a transformer-based attention module to correlate relative saliency information among multiple discontinuous images and, furthermore, applies the ConvLSTM module to capture the temporal information. Experiments show that the proposed TA-MSNet can estimate better and more consistent results than previous works for time series data.

Dynamic movement primitives (DMPs) are a flexible trajectory learning scheme widely used in motion generation of robotic systems. However, existing DMP-based methods mainly focus on simple go-to-goal tasks. Motivated to handle tasks beyond point-to-point motion planning, this work presents temporal logic guided optimization of motion primitives, namely \mathbf{PI}^{\mathbf{BB}-\mathbf{TL}}\mathbf{PI}^{\mathbf{BB}-\mathbf{TL}} algorithm, for complex manipulation tasks with user preferences. In particular, weighted truncated linear temporal logic (wTLTL) is incorporated in the \mathbf{PI}^{\mathbf{BB}-\mathbf{TL}}\mathbf{PI}^{\mathbf{BB}-\mathbf{TL}} algorithm, which not only enables the encoding of complex tasks that involve a sequence of logically organized action plans with user preferences, but also provides a convenient and efficient means to design the cost function. The black-box optimization is then adapted to identify optimal shape parameters of DMPs to enable motion planning of robotic systems. The effectiveness of the \mathbf{PI}^{\mathbf{BB}-\mathbf{TL}}\mathbf{PI}^{\mathbf{BB}-\mathbf{TL}} algorithm is demonstrated via simulation and experiment.

Despite the recent advances in continuum robots for minimally invasive surgery or interventions, their applications to endovascular neurosurgery have remained technically challenging due to the difficulty of miniaturization. Aimed at enabling robotic applications to neurovascular interventions for endovascular treatments of stroke or brain aneurysms, we present a telerobotically controlled magnetic soft continuum robot capable of active steering and navigation under externally applied magnetic fields. For magnetic steering, a seven-degree-of-freedom (7-DOF) serial manipulator is employed to place an actuating magnet that can be manipulated via real-time teleoperation of the robot arm. A motorized linear drive is used to advance or retract the magnetic soft continuum robot, the distal tip of which is steered by the actuating magnet to enable endovascular navigation in the complex cerebral vasculature. We demonstrate the system's ability to guide selective navigation in different branches of cerebral arteries using anatomical models under visual feedback. We also compare the navigational performance of our system with that of a manually controlled passive guidewire and a conventional magnet-tipped guidewire. We found that the telerobotically controlled magnetic soft continuum robot allows for safer and quicker access to hard-to-reach areas in clinically challenging anatomies by enabling smooth navigation in narrow and winding pathways.

Soft robot serial chain manipulators with the capability for growth, stiffness control, and discrete joints have the potential to approach the dexterity of traditional robot arms, while improving safety, lowering cost, and providing an increased workspace, with potential application in home environments. This paper presents an approach for design optimization of such robots to reach specified targets while minimizing the number of discrete joints and thus construction and actuation costs. We define a maximum number of allowable joints, as well as hardware constraints imposed by the materials and actuation available for soft growing robots, and we formulate and solve an optimization problem to output a planar robot design, i.e., the total number of potential joints and their locations along the robot body, which reaches all the desired targets, avoids known obstacles, and maximizes the workspace. We demonstrate a process to rapidly construct the resulting soft growing robot design. Finally, we use our algorithm to evaluate the ability of this design to reach new targets and demonstrate the algorithm's utility as a design tool to explore robot capabilities given various constraints and objectives.

We present a whole-body control architecture for the generation of stable task-oriented motions in Wheeled Inverted Pendulum (WIP) robots. Controlling WIP systems is challenging because the successful execution of tasks is subordinate to the ability to maintain balance. Our feedback control approach relies both on partial feedback linearization and Model Predictive Control (MPC). The partial feedback linearization reshapes the system into a convenient form, while the MPC computes inputs to execute the desired task by solving a constrained optimization problem. Input constraints account for actuation limits and a stability constraint is in charge of stabilizing the unstable body pitch angle dynamics. The proposed approach is validated by simulations on an ALTER-EGO robot performing navigation and loco-manipulation tasks.

This paper presents a solution to the problem of executing robotic tasks over time horizons that exceed the robot's total battery capacity. In the presented robotics application, the robot's mission is to satisfy two tasks: environmental exploration and environmental monitoring, both of which need to be executed over long time periods. These tasks need therefore to be persistified. Ensuring the longevity of the system requires to consider a maximum energy consumption at all times. Including a dependency of battery voltage dynamics on the control input introduces a quadratic term in the battery's dynamic equation, making previous persistification approaches no longer directly applicable, as they used control-affine dynamics. In this paper an alternative task persistification approach is formulated. The control strategy used is based on Control Barrier Functions (CBFs). Using which, the generated controller renders the system state variables, position and battery voltage, to remain within the boundaries of their safe sets. This generated safe controller minimally modifies the nominal controller, which commands the robot to satisfy its environmental mission. Once the CBFs are selected, the minimization problem that results is one with a quadratic cost and two nested scalar constraints: one of which is quadratic and the other is linear. This new class of problems is noted as Quadratic Cost Scalar Linear and Quadratically Constrained (QCSLQC) problems. An analytical solution to the general QCSLQC problem is presented.

In operations of multi-agent teams ranging from homogeneous robot swarms to heterogeneous human-autonomy teams, unexpected events might occur. While efficiency of operation for multi-agent task allocation problems is the primary objective, it is essential that the decision-making framework is intelligent enough to manage unexpected task load with limited resources. Otherwise, operation effectiveness would drastically plummet with overloaded agents facing unforeseen risks. In this work, we present a decision-making framework for multiagent teams to learn task allocation with the consideration of load management through decentralized reinforcement learning, where idling is encouraged and unnecessary resource usage is avoided. We illustrate the effect of load management on team performance and explore agent behaviors in example scenarios. Furthermore, a measure of agent importance in collaboration is developed to infer team resilience when facing handling potential overload situations.

We present TartanDrive, a large scale dataset for learning dynamics models for off-road driving. We collected a dataset of roughly 200,000 off-road driving interactions on a modified Yamaha Viking ATV with seven unique sensing modalities in diverse terrains. To the authors' knowledge, this is the largest real-world multi-modal off-road driving dataset, both in terms of number of interactions and sensing modalities. We also benchmark several state-of-the-art methods for model-based reinforcement learning from high-dimensional observations on this dataset. We find that extending these models to multi-modality leads to significant performance on off-road dynamics prediction, especially in more challenging terrains. We also identify some shortcomings with current neural network architectures for the off-road driving task. Our dataset is available at https://github.com/castacks/tartan_drive.

Recent studies demonstrated the vulnerability of control policies learned through deep reinforcement learning against adversarial attacks, raising concerns about the application of such models to risk-sensitive tasks such as autonomous driving. Threat models for these demonstrations are limited to (1) targeted attacks through real-time manipulation of the agent's observation, and (2) untargeted attacks through manipulation of the physical environment. The former assumes full access to the agent's states/observations at all times, while the latter has no control over attack outcomes. This paper investigates the feasibility of targeted attacks through visually learned patterns placed on physical objects in the environment, a threat model that combines the practicality and effectiveness of the existing ones. Through analysis, we demonstrate that a pre-trained policy can be hijacked within a time window, e.g., performing an unintended self-parking, when an adversarial object is present. To enable the attack, we adopt an assumption that the dynamics of both the environment and the agent can be learned by the attacker. Lastly, we empirically show the effectiveness of the proposed attack on different driving scenarios, perform a location robustness test, and study the tradeoff between the attack strength and its effectiveness Code is available at https://github.com/ASU-APG/ Targeted-Physical-Adversarial-Attacks-on-AD

This work presents a method for tactile classification of materials for virtual reality (VR) based robot teleoperation. In our system, a human-operator uses a remotely controlled robot-manipulator with an optical fibre-based tactile and proximity sensor to scan surfaces of objects in a remote environment. Tactile and proximity data and the robot's end-effector state feedback are used for the classification of objects' materials which are then visualized in the VR reconstruction of the remote environment for each object. Machine learning techniques such as random forest, convolutional neural and multi-modal convolutional neural networks were used for material classification. The proposed system and methods were tested with five different materials and classification accuracy of 90 % and more was achieved. The results of material classification were successfully exploited for visualising the remote scene in the VR interface to provide more information to the human-operator.

Large-area and high-precision tactile sensing information can not only improve the stability of robot grasping but also compensate for the lack of visual information in specific environments such as turbid underwater, dimness, and smoke. In this paper, we devise a universal jamming gripper with high-quality tactile sensing capability. The gripper adopts the particle jamming mechanism for grasping, and simultaneously uses a built-in camera to detect the deformation of its surface to obtain tactile information. To make the inside of the gripper transparent, glass beads and liquid with the same refractive index are applied as the internal filling. Besides, special treatments are taken to improve the tactile perception resolution of the gripper. The design perfectly merges visual-based tactile sensing into the traditional universal jamming gripper without changing its original gripping performance, making it possible for simultaneous grasping and sensing. To verify the tactile perception and grasping ability of the gripper in specific environments, we design two underwater experiments for grasping and pipe leak detection based on tactile information. Both have achieved a success rate not less than 95%, which demonstrates the effectiveness of the proposed gripper for manipulation in low visibility environments.

Fast and accurate tracking of an object's motion is one of the key functionalities of a robotic system for achieving reliable interaction with the environment. This paper focuses on the instance-level six-dimensional (6D) pose tracking problem with a symmetric and textureless object under occlusion. We propose a Temporally Primed 6D pose tracking framework with Auto-Encoders (TP-AE) to tackle the pose tracking problem. The framework consists of a prediction step and a temporally primed pose estimation step. The prediction step aims to quickly and efficiently generate a guess on the object's real-time pose based on historical information about the target object's motion. Once the prior prediction is obtained, the temporally primed pose estimation step embeds the prior pose into the RGB-D input, and leverages auto-encoders to reconstruct the target object with higher quality under occlusion, thus improving the framework's performance. Extensive experiments show that the proposed 6D pose tracking method can accurately estimate the 6D pose of a symmetric and textureless object under occlusion, and significantly outperforms the state-of-the-art on T-LESS dataset while running in real-time at 26 FPS.

This paper presents a unified controller for dual-arm robot dynamic multi-object nonprehensile transportation. The controller is composed of time-optimal path parameteri-zation (TOPP) and model predictive control (MPC) and aimed at efficiently and dynamically transporting objects using the dual-arm robot under physical constraints while avoiding the slippage of the objects. A force tracking controller without using the force sensor is also proposed to achieve accurate contact force control between the arms and objects. Experiments on the real robot show the effectiveness of the proposed TOPP-MPC-based controller.

We present a novel method for reliable robot navigation in uneven outdoor terrains. Our approach employs a fully-trained Deep Reinforcement Learning (DRL) network that uses elevation maps of the environment, robot pose, and goal as inputs to compute an attention mask of the environment. The attention mask is used to identify reduced stability regions in the elevation map and is computed using channel and spatial attention modules and a novel reward function. We continuously compute and update a navigation cost-map that encodes the elevation information or the level-of-flatness of the terrain using the attention mask. We then generate locally least-cost waypoints on the cost-map and compute the final dynamically feasible trajectory using another DRL-based method. Our approach guarantees safe, locally least-cost paths and dynamically feasible robot velocities in uneven terrains. We observe an increase of 35.18% in terms of success rate and, a decrease of 26.14% in the cumulative elevation gradient of the robot's trajectory compared to prior navigation methods in high-elevation regions. We evaluate our method on a Husky robot in real-world uneven terrains (∼ 4m4m of elevation gain) and demonstrate its benefits.

This paper presents a systematical approach to develop a novel reduced complexity quadruped (RCQ) robot designed for serpentine robotic tail research purposes. The critical design requirements are determined based on careful dynamic analysis and synthesis results. Guided by formulated design requirements and principles, a robot prototype was designed and built. The robot has an overall weight of 5 Kg and the body size of a domestic cat. The existing electronic system allows a control frequency of up to 1 kHz and accepts both torque and position commands. These features guarantee that the platform could be used to explore the dynamic usages of robotic tails on legged locomotion. The preliminary tests show that the hardware can lift itself off the ground up to 112 mm (46.7% of its body height) and stay in the air for at least 0.3 seconds.

We consider the problem of completing a set of nn tasks with a human-robot team using minimum effort. In many domains, teaching a robot to be fully autonomous can be counterproductive if there are finitely many tasks to be done. Rather, the optimal strategy is to weigh the cost of teaching a robot and its benefit- how many new tasks it allows the robot to solve autonomously. We formulate this as a planning problem where the goal is to decide what tasks the robot should do autonomously (act), what tasks should be delegated to a human (delegate) and what tasks the robot should be taught (learn) so as to complete all the given tasks with minimum effort. This planning problem results in a search tree that grows expo-nentially with nn - making standard graph search algorithms intractable. We address this by converting the problem into a mixed integer program that can be solved efficiently using off-the-shelf solvers with bounds on solution quality. To predict the benefit of learning, we use an approximate simulation model of the tasks to train a precondition model that is parameterized by the training task. Finally, we evaluate our approach on peg insertion and Lego stacking tasks- both in simulation and real-world, showing substantial savings in human effort.

Simulation is a crucial tool for accelerating the development of autonomous vehicles. Making simulation realistic requires models of the human road users who interact with such cars. Such models can be obtained by applying learning from demonstration (LfD) to trajectories observed by cars already on the road. However, existing LfD methods are typically insufficient, yielding policies that frequently collide or drive off the road. To address this problem, we propose Symphony, which greatly improves realism by combining conventional policies with a parallel beam search. The beam search refines these policies on the fly by pruning branches that are unfavourably evaluated by a discriminator. However, it can also harm diversity, i.e., how well the agents cover the entire distribution of realistic behaviour, as pruning can encourage mode collapse. Symphony addresses this issue with a hierarchical approach, factoring agent behaviour into goal generation and goal conditioning. The use of such goals ensures that agent diversity neither disappears during adversarial training nor is pruned away by the beam search. Experiments on both proprietary and open Waymo datasets confirm that Symphony agents learn more realistic and diverse behaviour than several baselines.