IROS 2022 Papers

Performing a complex autonomous mission with a multi-robot system requires to integrate several deliberative approaches to perform task allocation, optimization, and execution control. Implementing such a deliberative architecture is a complex task: it requires the developer to master the decision algorithms themselves (e.g., automated planning models), to have a good knowledge of the involved robotic platforms, and to think about how these elements will be assembled as a system architecture. We propose a framework to help designing such deliberative architectures. The framework relies on the concept of a hierarchical structure of actors, each actor managing goals with specific planning or optimization approaches, and delegating sub-goals to other actors.

As soft, continuum robots see increasing areas of application, many scenarios have arisen where it is necessary to consider the geometric shape of the robot. The current approaches to robot kinematics, such as the piecewise constant-curvature (PCC) model, are effective in representing simple overall robot geometry and estimating the end-effector state, but they are less intuitive for planning robots that involve complex geometries. In this work, we propose a solution to the geometric design problem by a two-part approach: a freeform spline defines a “shape curve” that describes the overall geometry of the robot, and then a “kinematic curve” composed of shapes that are feasible to replicate with continuum robots is fitted to the shape curve. As an implementation of this approach, we specifically explore the application of piecewise cubic Bezier curves in designing the shape curve of the robot, and pairs of arcs to construct the kinematic curves. Finally, the approach is applied to a tip-extension “vine” robot that is designed and fabricated to “grow” along a designed path and access the top surface of an obstacle.

In this paper, we propose a novel and general method for autonomous robotic assembly of arbitrary and complex-shaped parts in the presence of 6-dimensional uncertainty. When a nominal assembly motion of the robot holding a part is stopped by contact due to uncertainty, our method finds the best estimate for the uncertainty and the contact configuration of the part based on sensed force/torque and uses that information to find a more accurate estimate of the goal configuration to guide a recovery motion of the part. It is based on a general, surface-based sphere tree representation of parts, a constrained optimization strategy to find the best estimate of the contact configuration under an uncertainty estimate, and a learned force/torque calibration model to relate computed force/torque and the sensed real force/torque. The method is applied and evaluated on different complex-shaped multi-peg-in-hole tasks. The results show that our method can achieve successful assembly with the presence of realistic 6-D uncertainties more than 10 times of the tight task clearances in terms of orientation clearance (< 0.015rad)(< 0.015rad) and position clearance (< 1.5mm)(< 1.5mm), in all the test cases.

Stroke survivors usually have dyskinesia, who have an urgent need for rehabilitation-assist training. To reduce the labor of rehabilitation therapists, this paper attempts to investigate an effective rehabilitation-assisted robot skill acquisition framework, which is inspired by the scheme of robot learning from demonstration (LfD). Since most of the current LfD methods were implemented with rigorous assumptions that the considering motion features are only represented on an individual manifold. Meanwhile, despite many advancements that have been achieved on time-position trajectories and position-velocity trajectories, those methods are restricted to Euclidean space and can not be applied to learn those dexterous and compliant rehabilitation-assisted robot skills such as position-orientation trajectories and force-stiffness trajectories, etc. In this paper, we propose a novel skill acquisition framework for rehabilitation-assisted robot using manifold-mappings and Gaussian processes, which allows the robot to 1) simultaneously considering the robot position, orientation, force as well as stiffness by manifold-mappings among d-dimensional Euclidean space \mathcal{R}^{d}\mathcal{R}^{d}, special orthogonal group S\mathcal{O}S\mathcal{O} (3), and Riemannian space \mathcal{M}\mathcal{M}, respectively, which resulting in accurate motion and compliant behavior; 2) retrieving skill representation by encap-sulating the variability of multiple high-dimensional demon-strations that with input-dependent noises; 3) implementing the via-points-based trajectory modulation by considering task constraints or environmental changes. To simplify the writing, we named the proposed framework as Multi-motion Features Fusion-based Robot Skill Learning (MF2 RoSL). To effectively evaluate the effectiveness of our proposed method, an upper limb rehabilitation training system with a collaborative Kinova robot is developed. The training exercises of our system are determined according to the Brunnstrom therapeutic approach t... Show More

Real-time control is an essential aspect of safe robot operation in the real world with dynamic objects. We present a framework for the analysis of object-aware con-trollers, methods for altering a robot's motion to anticipate and avoid possible collisions. This framework is focused on three design considerations: kinematics, motion profiles, and virtual constraints. Additionally, the analysis in this work relies on verification of robot behaviors using fundamental robot-obstacle experimental scenarios. To showcase the effectiveness of our method we compare three representative object-aware controllers. The comparison uses metrics originating from the design considerations. From the analysis, we find that the design of object-aware controllers often lacks kinematic considerations, continuity of control points, and stability in movement profiles. We conclude that this framework can be used in the future to design, compare, and benchmark obstacle avoidance methods.

This paper presents a framework for transferring surface finishing skills to new surface geometries while preserving the surface finish quality. The main idea is to estimate the contact area between the workpiece and the tool by using 3D point cloud approach and replicate a given material removal rate and the accumulated material removal, as these quantities are the main parameters for quality. The grinding motion trajectory is generated by solving a constrained optimization problem that minimizes the maximal point-wise deviation between actual and desired material removal and simultaneously minimizes the average deviation between actual and desired material removal rate. The proposed approach is verified in simulation to show the difference between direct replication of force/motion and the proposed replication of material removal. Finally, experimental results confirm that the quality of a surface finishing task can be transferred to new surface geometries with the proposed method.

Fast, robust, and flexible part feeding is essential for enabling automation of low volume, high variance assembly tasks. An actuated vision-based solution on a traditional vibratory feeder, referred to here as a vision trap, should in principle be able to meet these demands for a wide range of parts. However, in practice, the flexibility of such a trap is limited as an expert is needed to both identify manageable tasks and to configure the vision system. We propose a novel approach to vision trap design in which the identification of manageable tasks is automatic and the configuration of these tasks can be delegated to an automated feeder design system. We show that the trap's capabilities can be formalized in such a way that it integrates seamlessly into the ecosystem of automated feeder design. Our results on six canonical parts show great promise for autonomous configuration of feeder systems.

Traditional gimbal-based bionic eye systems usually use a multi-degree-of-freedom mechanical platform to move the camera freely, which makes the structure complex and bulky. The galvanometer-based reflective bionic eye system uses a galvanometer to replace the traditional mechanical rotation structure, which separates the camera from the gimbal system, greatly simplifying the structure. However, there are currently few methods for calibrating such systems, mostly for object detection and tracking. In this paper, a flexible method for high-precision calibration of a galvanometer-based reflective bionic eye system is proposed. In this method, a planar target is used for the calibration of the bionic eye system. The effectiveness and accuracy of the method are evaluated by the reprojection error of the control voltage and the spatial localization of the binocular system. Experiments show that the error of the control voltage after calibration is less than 0.2%. At an indoor distance of about 7 m, the RMSE of spatial visual localization is less than 0.3 cm.

In order to facilitate and assist the indoor mobility of people with special needs, the classically static objects in the environment, such as furniture, can be rendered mobile. The need for efficient and safe autonomous coordination of a mobile furniture swarm arises. We present a closed-form approach for mobile furniture obstacle avoidance and navigation within an indoor environment. The approach shows that each mobile furniture agent, defined by a polygonal surface, does not collide with any static or mobile obstacle (e.g., a person is moving around). All controllable mobile furniture converges towards a defined goal position and orientation. We showcase the application of this algorithm in simulation on mobile furniture for smart environments. Results demonstrate that the proposed method can coordinate a swarm of mobile furniture to get out of the way of a mobile agent representing a person with limited mobility passing through the room while avoiding obstacles and converging towards a predefined target pose.

Cleaning public areas like commercial complexes is challenging due to their sophisticated surroundings and the vast kinds of real-life dirt. Robots are required to distinguish dirts and apply corresponding cleaning strategies. In this work, we proposed an active-cleaning framework by utilizing deep-learning methods for both solid wastes detection and liquid stains segmentation. Our system consists of 4 components: a Perception module integrated with deep-learning models, a Post-processing module for projection, a Tracking module for map localization, and a Planning and Control module for cleaning strategies. Compared with classic approaches, our vision-based system significantly improves cleaning efficiency. Besides, we released the largest real-world indoor hybrid dirt cleaning dataset (HD10K) containing 10K labeled images, together with a track-level evaluation metric for better cleaning performance measurement. The proposed deep-learning based system is verified with extensive experiments on our dataset, and deployed to Gaussian Robotics's robots operating globally. Dataset is available at: https://gaussianopensource.github.io/projects/active_cleaning.

Most virtual reality (VR) applications use a commercial controller for interaction. However, a typical virtual reality controller (VRC) lacks positional precision and accu-racy in millimeter-scale scenarios. This lack of precision and accuracy is caused by built-in sensors drift. Therefore, the tracking performance of a VRC needs to be enhanced for millimeter-scale scenarios. Herein, we introduce a novel way of enhancing the tracking performance of a commercial VRC in a millimeter-scale environment using a deep learning (DL) al-gorithm. Specifically, we use a long short-term memory (LSTM) model trained with data collected from a linear motor, an IMU sensor, and a VRC. We integrate the virtual environment developed in Unity software with the LSTM model running in Python. We designed three experimental conditions: the VRC, Kalman filter (KF), and LSTM modes. Furthermore, we evaluate tracking performances in the three conditions and two other experimental scenarios, namely stationary and dynamic. In the stationary experimental scenario, the system is left motionless for 10 s. By contrast, in the dynamic experimental scenarios, the linear stage moves the system by 12 mm along the X, Y, and Z axes. The experimental results indicate that the deep learning model outperforms the standard controllers positional performance by 85.69 % and 92.14 % in static and dynamic situations, respectively.

Establishing a physics-based model capturing the kinetostatic behavior of concentric tube continuum robots is challenging as elastic interactions between the flexible tubes constituting the robot result in a highly non-linear problem. The Goldstandard physics-based model using the Cosserat theory of elastic rods achieves reasonable approximations with 1.5 - 3 % with respect to the robot's length, if well-calibrated. Learning-based models of concentric tube continuum robots have been shown to outperform the Goldstandard model with approximation errors below 1 %. Yet, the merits of learning-based models remain largely unexplored as no common dataset and benchmark exist. In this paper, we present a dataset captured from a three-tube concentric tube continuum robot for use in learning-based kinematics research. The dataset consists of 100 000 joint configurations and the corresponding four 6 dof sensors in SE(3) measured with an electromagnetic tracking system (github.com/ContinuumRoboticsLab/CRL-Dataset-CTCR-Pose). With our dataset, we empower the continuum robotics and machine learning community to advance the field. We share our insights and lessons learned on joint space representation, shape representation in task space, and sampling strategies. Furthermore, we provide benchmark results for learning the forward kinematics using a simple, shallow feedforward neural network. The benchmark results for the tip error are 0.74 mm w.r.t. position (0.4 % of total robot length) and 6.49° w.r.t. orientation.

Enlightened by the creeping gait of natural snakes, snake-like robots swing joints side to side at similar tracks for generating propelling forces. However, it is not always essential to control all joints of a snake-like robot to realize the creeping gait. Therefore, in this paper, a creeping snake-like robot with partially actuated joints has been investigated, towards reducing the redundancy caused by full actuation. Essentially, this approach is composed of the following two concepts: 1) joint equipped with torsion spring mechanism bridges the passive joint to generate rhythm oscillation, and 2) harmonic joint trajectories assist the robot in generating more efficient locomotion. We hereafter demonstrated that the actuated joint dominates passive dynamics of the system, which contributes to overall motion. Meanwhile, different spring stiffness affects the motion performance. Additionally, the interaction between robot and environment through Coulomb friction has been considered to reveal the contributing factors that assist the snake-like robot to yield better locomotion performance.

Control of an articulated spine is important for humanoids' dynamic and balanced motion. Although there have been many spinal structures for humanoids, their actuation is still limited due to the usage of geared motors for joints. This paper introduces position control of a distributed electrome-chanical spine in a vertical plane. The spine dynamics model is approximated as an open chain. Gravitational and spring torques are compensated for the control. Moreover, torque-to-current conversion for the actuator is developed. Experimental results show the implemented control of the electromechanical spine for undulatory motions.

Reinforcement learning shows great potential to solve complex contact-rich robot manipulation tasks. However, the safety of using RL in the real world is a crucial problem, since unexpected dangerous collisions might happen when the RL policy is imperfect during training or in unseen scenarios. In this paper, we propose a contact-safe reinforcement learning framework for contact-rich robot manipulation, which maintains safety in both the task space and joint space. When the RL policy causes unexpected collisions between the robot arm and the environment, our framework is able to immediately detect the collision and ensure the contact force to be small. Furthermore, the end-effector is enforced to perform contact-rich tasks compliantly, while keeping robust to external disturbances. We train the RL policy in simulation and transfer it to the real robot. Real world experiments on robot wiping tasks show that our method is able to keep the contact force small both in task space and joint space even when the policy is under unseen scenario with unexpected collision, while rejecting the disturbances on the main task.

Over the last years, research done in automation and industrial robotics has established the foundations for skill-oriented systems based on the OPC UA standard. Nevertheless, utilizing these advances in other areas of robotics research can be challenging and time consuming. We present a framework aiming to reduce this entry threshold. Our solution is an open source, easy to configure tool based on OPC UA, that provides with a hardware agnostic, skill-oriented, event driven interface to systems. The framework allows integrating external hardware and software by means of plugins. It also provides a mechanism for endowing skills with hardware agnostic motion control. We demonstrate how our framework can be combined with state of the art approaches to control in a simulated assembly task with multiple robots, conveyors, actuators and sensors involved.

Delicate industrial insertion tasks (e.g., PC board assembly) remain challenging for industrial robots. The chal-lenges include low error tolerance, delicacy of the components, and large task variations with respect to the components to be inserted. To deliver a feasible robotic solution for these insertion tasks, we also need to account for hardware limits of existing robotic systems and minimize the integration effort. This paper proposes a composable framework for efficient integration of a safe insertion policy on existing robotic platforms to accomplish these insertion tasks. The policy has an interpretable modularized design and can be learned efficiently on hardware and transferred to new tasks easily. In particular, the policy includes a safe insertion agent as a baseline policy for insertion, an optimal configurable Cartesian tracker as an interface to robot hardware, a probabilistic inference module to handle component variety and insertion errors, and a safe learning module to optimize the parameters in the aforementioned modules to achieve the best performance on designated hard-ware. The experiment results on a URIO robot show that the proposed framework achieves safety (for the delicacy of components), accuracy (for low tolerance), robustness (against perception error and component defection), adaptability and transferability (for task variations), as well as task efficiency during execution plus data and time efficiency during learning.

This paper presents a comprehensive comparative study of the multisensor-based force observers for accurate force control. A force controlled system which contains a force sensor for measuring force transmitted to the load by the motor and an encoder for measuring motor position is considered as the general multisensor-based motion system in this study. Even though these multisensor-based motion systems are emerging as potential motion systems as the demands for collaborative robots increase, there has been few studies that investigate their advantages and limitations. to address this issue, three types of observer-based force controllers that utilize the multisensors are designed and implemented. These controllers exploit the availability of force sensor, motor encoder, and motor torque information from the multisensor-based motion system to estimate accurate force which is later utilized to close the feedback loop. Mathematical and quantitative analyses are conducted to compare performances of the proposed observer-based force control and through this, their advantages and limitations are pointed out. Finally, simulation and an experimental case study with an actual robot are conducted to validate the force tracking performance of the designed force control systems.

Building humanoid robots with properties similar to those of humans in terms of strength and agility is a great and unsolved challenge. This work introduces a compact and lightweight wrist joint mechanism that is singularity-free and has large range of motion. The mechanism provides two degrees of freedom (DoF) and was developed for integration into a human scale humanoid robot arm. It is based on a parallel mechanism with rolling contact joint behaviour and remote actuation that facilitates a compact design with low mass and inertia. The mechanism's kinematics along with a solution of the inverse kinematics problem for the specific design, and the manipulability analysis are presented. The first prototype of the proposed mechanism shows the possible integration of actuation, sensing and electronics in small and narrow space. Experimental evaluations shows that the design feature unique performance regarding weight, speed, payload and accuracy.