IROS 2022 Papers

To aid in real-world situations, soft robots need to be able to estimate their state and external interactions based on proprioceptive sensors. Estimating disturbances allows a soft robot to perform desirable force control. However, even in the case of rigid manipulators, force estimation at the end-effector is seen as a non-trivial problem. And indeed, current approaches to address this challenge have shortcomings that prevent their general application. They are often based on simplified soft dynamic models, such as the ones relying on a piece-wise constant curvature approximation or matched rigid-body models that do not represent enough details of the problem. This severely limits applications in complex human-robot interaction. Finite element method (FEM) based modeling allows for predictions of soft robot dynamics in a more generic fashion. Here, using the soft robot modeling capabilities of the frame-work SOFA, we built a detailed FEM model of a multi-segment soft continuum robotic arm composed of compliant deformable materials and fiber-reinforced pressurized actuation chambers. In addition, a model for sensors that provide orientation output is presented. This model is used to establish a state observer for the manipulator. The sensor model is adequate for representing the output of flexible bend sensors as well as orientations provided by IMUs or coming from tracking systems, all of which are popular choices in soft robotics. Model parameters were calibrated to match imperfections of the manual fabrication process using physical experiments. We then solve a quadratic programming inverse statics problem to compute the components of external force that explain the pose mismatch. Our experiments show an average force estimation error of around 1.2%. As the methods proposed are generic, these results are encouraging for the task of building soft robots exhibiting complex, reactive, sensor-based behavior that can be deployed in human-centered environments. Show More

We investigate how employing model learning methods in concert with model predictive control (MPC) can be used to automate obstacle reduction to mitigate risks to Combat Engineers operating construction equipment in an active battlefield. We focus on the task of earthen berm removal using a bladed vehicle. We introduce a novel data-driven formulation for earthmoving dynamics that enables prediction of the vehicle and detailed terrain state over a one second horizon. In a simulation environment, we first record demonstrations from a human operator and then train two different earthmoving models to produce predictions of the high-dimensional state using under six minutes of data. Optimization over the learned model is performed to select an action sequence, constrained to a 2D space of template action trajectories. Simple recovery controllers are implemented to improve controller performance when the model predictions degrade. This system yields near human-level performance on a berm removal task, indicating that model learning and predictive control is a promising data-efficient approach to autonomous earthmoving.

While both navigation and manipulation are chal-lenging topics in isolation, many tasks require the ability to both navigate and manipulate in concert. To this end, we propose a mobile manipulation system that leverages novel navigation and shape completion methods to manipulate an object with a mobile robot. Our system utilizes uncertainty in the initial estimation of a manipulation target to calculate a predicted next-best-view. Without the need of localization, the robot then uses the predicted panoramic view at the next-best-view location to navigate to the desired location, capture a second view of the object, create a new model that predicts the shape of object more accurately than a single image alone, and uses this model for grasp planning. We show that the system is highly effective for mobile manipulation tasks through simulation experiments using real world data, as well as ablations on each component of our system.

It has been shown that changing the coordinates describing a multi-body system to use absolute rather than relative angles produces a significant improvement in the tractability of trajectory optimization problems. This simplifies the equations of motion when modelling long kinematic chains. In this paper, we extend this idea by investigating whether a maximal coordinate system, which also describes the translational position of bodies using absolute coordinates, might lead to further performance improvements. We compare it to the relative translation, absolute orientation (RTAO) coordinate scheme using a batch of trajectory optimization trials selected with contact-implicit legged locomotion applications in mind. We find that maximal coordinates tend to shorten solving times for spatial problems, while the RTAO formulation still performs best in the case of planar motion.

Microspine grippers allow robots to ascend steep rocky slopes and cliff faces, enabling scientific exploration of exposed strata on Earth and other solar system bodies. Historically, the Shape Deposition Manufacturing (SDM) process has been used to fabricate multi-material suspensions for load-sharing among multiple microspines. We instead apply the Hybrid Deposition Manufacturing (HDM) process to microspine fabrication, and we further propose a novel 3D-printed microspine suspension design that can be manufactured via Fused Deposition Manufacturing (FDM) alone, using a single flexible material with an embedded fishhook. We use a model of microspine stiffness that allows designers to compensate for order-of-magnitude changes in material tensile modulus by adjusting geometric parameters of the design. The stiffness model and the FDM microspine design are validated through tensile testing, and mechanical properties of the HDM and FDM designs are compared against a standard SDM microspine design. We demonstrate that the FDM process can produce microspines with equivalent normal and axial stiffness and superior maximum load and fatigue response to SDM microspines, and discuss additional advantages of the FDM process for rapid prototyping and broader accessibility.

This paper presents a combined task and motion planner for a robot arm to carry out 3D metal wire curving tasks by collaborating with a bending machine. We assume a collaborative robot that is safe to work in a human environment but has a weak payload to bend objects with large stiffness, and developed a combined planner for the robot to use a bending machine. Our method converts a 3D curve to a bending set and generates the feasible bending sequence, machine usage, robotic grasp poses, and pick-and-place arm motion considering the combined task and motion level constraints. Compared with previous deformable linear object shaping work that relied on forces provided by robotic arms, the proposed method is suitable for the material with high stiffness. We evaluate the system using different tasks. The results show that the proposed system is flexible and robust to generate robotic motion to corporate with the designed bending machine.

The dynamic load attached to the load gravity imposes an excessive burden to human shoulders during load carriage, resulting in possible muscle injuries and additional physical exertion. This paper proposes an active suspension backpack, capable of transferring partial load from human shoulders to pelvis and alleviating the dynamic load through separated panels and motor actuation, to reduce pressure on human shoulders and improve walking metabolic efficiency. Based on the human body motion in the vertical direction, the dynamical model of the human-backpack system with shoulder interaction force measured by a soft ballonet with an embedded air pressure sensor is introduced, and an impedance controller has been implemented to maintain a relatively small and constant pressure on the shoulder. In an experimental case study, we presents preliminary results of three healthy subjects performing a treadmill walking with a 20kg load in ACTIVE configuration where the shoulder pressure shows a decrease by 30% along with a reduction of the metabolic energy consumption by 16.4%, compared with the load LOCKED case.

In this work, we present a memory-augmented approach for image-goal navigation. Earlier attempts, including RL-based and SLAM-based approaches have either shown poor generalization performance, or are heavily-reliant on pose/depth sensors. Our method is based on an attention-based end-to-end model that leverages an episodic memory to learn to navigate. First, we train a state-embedding network in a self-supervised fashion, and then use it to embed previously-visited states into the agent's memory. Our navigation policy takes advantage of this information through an attention mechanism. We validate our approach with extensive evaluations, and show that our model establishes a new state of the art on the challenging Gibson dataset. Furthermore, we achieve this impressive performance from RGB input alone, without access to additional information such as position or depth, in stark contrast to related work.

This letter presents a complete framework Meeting-Merging-Mission for multi-robot exploration under communication restriction. Considering communication is limited in both bandwidth and range in the real world, we propose a lightweight environment presentation method and an efficient cooperative exploration strategy. For lower bandwidth, each robot uses specific polytopes to maintain free space and to generate Super Frontier Information (SFI), which serves as the source for exploration decision-making. To reduce repeated exploration, we develop a mission-based protocol that drives robots to share collected information in stable rendezvous. We also design a complete path planning scheme for both centralized and decentralized cases. To validate that our framework is practical and generic, we present an extensive benchmark and deploy our system into multi-UGV and multi-UAV platforms.

Soft robots are capable of effortlessly adapting to their environment using elastic materials that impart structural compliance into their designs, allowing them to execute complex tasks with minimal sensing and control. However, soft robots cannot exert high forces and can only handle low deformation forces. These characteristics typically limit their applicabil-ity to tasks that require delicate interactions. In this work, we present a mechanically programmable, variable stiffness, jamming actuator based on an articulated mesh structure. The proposed actuator can elastically bend when it is not activated but compresses to attain a pre-programmed shape that is determined by the mesh geometry of the multi-layer jamming architecture when pressure is applied to the silicone pouch containing it. Unlike traditional jamming structures the utilisation of the articulated mesh structure facilitates elastic deformations past the yield point when jammed. The actuator can become >27 times stiffer than its relaxed configuration when exposed to only 90 kPa pressure. We demonstrate the efficiency of this actuator by developing variable stiffness joints that can be used to create: i) underactuated, tendon driven robotic grippers and soft, disposable robotic grippers that exhibit increased dexterity and ii) wearable, affordable, lightweight elbow exoskeleton systems that can assist humans in holding heavy objects with minimal effort.

In this paper, we introduce a mask-based grasping method that discerns multiple objects within the scene regard-less of transparency or specularity and finds the optimal grasp position avoiding clutter. Conventional vision-based robotic grasping approaches often fail to extend to the scenes containing transparent objects due to their different visual appearance. To handle the different visual characteristics, we first segment both transparent and opaque objects into instance masks, which serve as the domain-agnostic intermediate representation of both object types, using a neural network. While there exists no labelled training dataset that strongly represents both object types, we overcome the limitation by augmenting transparent objects on an existing large-scale dataset. Then, given the object instance masks, our method selects the top K discrete masks and robustly estimates grasp poses avoiding clutter. Through experiments, we verify that the instance masks are light-weight yet provide sufficient information for vision-based grasping agnostic of various appearances. On an unseen real-world test environment with complex objects, our method substantially outperforms previous methods without fine-tuning.

Suture needle localization is necessary for autonomous suturing. Previous approaches in autonomous suturing often relied on fiducial markers rather than markerless detection schemes for localizing a suture needle due to the in-consistency of markerless detections. However, fiducial markers are not practical for real-world applications and can often be occluded from environmental factors in surgery (e.g., blood). Therefore in this work, we present a robust tracking approach for estimating the 6D pose of a suture needle when using inconsistent detections. We define observation models based on suture needles' geometry that captures the uncertainty of the detections and fuse them temporally in a probabilistic fashion. In our experiments, we compare different permutations of the observation models in the suture needle localization task to show their effectiveness. Our proposed method outperforms previous approaches in localizing a suture needle. We also demonstrate the proposed tracking method in an autonomous suture needle regrasping task and ex vivo environments**The code is available at https://github.com/ucsdarclab/suture-needle-tracking..

Spatiotemporal maps are data-driven estimates of time changing phenomena. For environmental science, rather than collect data from an array of static sensors, a mobile sensor platform could reduce setup time and cost, maintain flexibility to be deployed to any area of interest, and provide active feedback during observations. While promising, mapping is challenging with mobile sensors because vehicle constraints limit not only where, but also when observations can be made. By assuming spatial and temporal correlations in the data through kernel functions, this paper uses Gaussian process regression (GPR) to generate a maximum likelihood estimate of the phenomenon while also tracking the estimate uncertainty. Spatiotemporal mapping by GPR is simulated for a single fixed-path mobile robot observing a latent spatiotemporal scalar field. The learned spatiotemporal map captures the structure of the latent scalar field with the largest uncertainties in areas the robot never visited.

Kinesthetic teaching of collaborative robots is applied for intuitive and flexible robot programming by demonstration. This enables non-experts to program such robots on the task-level. Multiple strategies exist to teach velocity- or torque-controlled robots and, thus, the maneuverability among commercial robots differs significantly. However, currently there exists no metric that quantifies how “well” the robot can be guided, e.g., how much effort is required to initiate a motion. In this paper, we propose standardized procedures to quantitatively assess robot manual maneuverability. First, we identify different motion phases during kinesthetic teaching. For each phase, we then propose metrics and experimental setups to evaluate them. The experimental protocols are applied to the proprietary teaching schemes of five commercial robots, namely the KUKA LWR iiwa 14, Yuanda Yu+, Franka Emika robot, and Universal Robot's UR5e and UR10e. The experimental comparison highlights distinct differences between the robots and shows that the proposed methods are a meaningful contribution to the performance and ergonomics assessment of collaborative robots.

The teleoperation of mobile manipulators may pose significant challenges, demanding complex interfaces and causing a substantial burden to the human operator due to the need to switch continuously from the manipulation of the arm to the control of the mobile platform. Hence, several works have considered to exploit shared control techniques to overcome this issue and, in general, to facilitate the task execution. This work proposes a manipulability-aware shared locoma-nipulation motion generation method to facilitate the execution of telemanipulation tasks with mobile manipulators. The method uses the manipulability level of the end-effector to control the generation of the mobile base and manipulator motions, facilitating their simultaneous control by the operator while executing telemanipulation tasks. Therefore, the operator can exclusively control the end -effector, while the underlying ar-chitecture generates the mobile platform commands depending on the end-effector manipulability level. The effectiveness of this approach is demonstrated with a number of experiments in which the CENTAURO robot, a hybrid leg-wheel platform with an anthropomorphic upper body, is teleoperated to execute a set of telemanipulation tasks.

This paper proposes a model of a class of regular virtual tubes that can generate safe, feasible, and smooth space for a robotics swarm in an obstacle-dense environment, especially for a drone swarm based on the flocking model. The regular principles are first proposed, and the regular conditions are then formulated based on the principles. A method to obtain a regular virtual tube is also presented based on trajectory planning and regular conditions. The proposed method's effectiveness and robustness are comprehensively demonstrated in a simulation environment with random obstacles.

Accurate and real-time position estimates are cru-cial for mobile robots. This work focuses on ranging-based positioning systems, which rely on distance measurements between known points, called anchors, and a tag to localize. The topology of the network formed by the anchors strongly influences the tag's localizability, i.e., its ability to be accurately localized. Here, the tag and some anchors are supposed to be carried by robots, which allows enhancing the positioning accuracy by planning the anchors' motions. We leverage Bayesian Cramer-Rao Lower Bounds (CRLBs) on the estimates' covariance in order to quantify the tag's localizability. This class of CRLBs can capture prior information on the tag's position and take it into account when deploying the anchors. We propose a method to decrease a potential function based on the Bayesian CRLB in order to maintain the localizability of the tag while having some prior knowledge about its position distribution. Then, we present a new experiment highlighting the link between the localizability potential and the precision expected in practice. Finally, two real-time anchor motion planners are demonstrated with ranging measurements in the presence or absence of prior information about the tag's position.

The magnetic medical microrobots are influenced by diverse factors such as the medium, the geometry of the microrobot, and the imaging procedure. It is worth noting that the size limitations make it difficult or even impossible to obtain reliable physical properties of the system. In this research, to achieve a precise microrobot control using minimum knowledge about the system, an Adaptive Fuzzy Sliding-Mode Control (AFSMC) scheme is designed for the motion control problem of the magnetically actuated microrobots in presence of input saturation constraint. The AFSMC input consists of a fuzzy system designed to approximate an unknown nonlinear dynamical system and a robust term considered for mismatch compensation. According to the designed adaptation laws, the asymptotic stability is proved based on the Lyapunov theorem and Barbalat's lemma. In order to evaluate the effectiveness of the proposed method, a comparative simulation study is conducted.

Aiming at the problem that it is difficult to calculate the force of permanent magnets in the magnetic field, this paper proposes a nonlinear mechanical model of linear array magnetic field based on radial basis function neural network (RBFNN). Combined with the linear Halbach array adsorption module of the wall-climbing robot, the three-dimensional geometric magnetic fields of four typical linear array permanent magnets were constructed, and the theoretical models of the interaction between the magnetic fields were given respectively. Further, the finite element simulation calculation of the magnetic force was carried out using COMSOL Multiphysics. According to the parametric scanning results of the orthogonal test, a nonlinear intelligent prediction model of the force between magnetic fields with local loss sensitivity is established by using the RBFNN numerical fitting method. The average deviation of the network test set is 1.19, and the standard deviation is 0.80. The intelligent prediction model has strong generalization performance, faster convergence speed and stronger flexibility, which provides a theoretical basis for the interaction and control of array magnetic fields.