IROS 2020 Papers

The exploration of spatially limited terrestrial or aquatic environments requires miniature and lightweight robots. Soft-bodied robot research is paving ways for a new class of small-scale robots that can navigate a variety of environments with minimum influence on the environment itself. However, it is generally challenging to design miniature soft-bodied robots that efficiently adapt to the change between viscous environments. A small-scale soft-bodied robot, which could slowly move on dry land, will need rapid motions to be able to swim in a wet environment. Although using snap-through buckling of a deformable body could help to create swift motions of the robot, merely applying the snap-through buckling does not improve the swimming speed of the robot so much. Here we propose a design of a stringy soft-bodied robot that can crawl on dry surfaces and swim in liquid environments. Besides taking advantage of the snap-through buckling using coil shape memory alloys (SMAs), we design the body of the robot with a geometrical overlapping of the active body segments and control the frequency of the undulation movement, which is crucial for the swimming locomotion. We evaluate the performance of the robot in different density and viscosity liquids such as cooking oil and Glycerin solution. We found that the robot needs to drastically change its undulation from low to high frequency when it moves from high to low viscosity environments. Our robot can swim at a speed of 3. 37 body-lengths per minute (BL/min) and crawl at a speed of 1. 74 BL/min. We anticipate our findings will help shed light on the design of soft-bodied robots that adapt to the changing environments efficiently.

AIRo-5.1 an in-pipe inspection robot comprised of two passive compliant joints and a single active compliant joint that is driven by a series elastic actuator (SEA) is presented in the course of this study. As an aid in pipeline maintenance, AIRo-5.1 controls joint angles and the torque of middle joints, to enable them to adapt to bend, branch, vertical pipes, and slippery surfaces. To sense the joint torques, an improved durable polyurethane rubber spring was installed. To smoothly pass through T-branches, the angle trajectory of middle joints was calculated based on the pipe geometry and thus, was interpolated using a cosine curve. Experiments to verify robot performance in bent and T-branch pipes, its joint angle and torque control was conducted.

This paper presents a framework providing a full pipeline to execute a complex physical interaction behaviour of a humanoid bipedal robot, both from a theoretical and a practical standpoint. Building from a multi-contact control architecture that combines contact planning and reactive force distribution capabilities, the main contribution of this work consists in the integration of a sample-based motion planning layer conceived for transitioning movements where obstacle and self-collisions avoidance is involved. To plan these motions we use Rapidly Exploring Random Tree (RRT) projected on the contacts manifold and validated through the Centroidal Statics (CS) model, to ensure static balance on non-coplanar surfaces. Finally, we successfully validate the presented planning and control architecture on the humanoid robot COMAN+ performing a wall-plank task.

A long-standing argument in model-based control of locomotion is about the level of complexity that a model should have to define a behavior such as running. Even though a goldilocks model based on biomechanical evidence is often sought, it is unclear what level of complexity qualifies to be such a model. This dilemma deepens further for bipedal robotic running with point feet, since these robots are underactuated. When center-of-mass (COM) trajectories defined by the spring-loaded inverted pendulum (SLIP) model are fully tracked, angular coordinates of the robot's trunk become uncontrolled. Existing work in the literature approach this problem either by trading off COM trajectory tracking against upright trunk posture during stance or by adopting more detailed models that include effects of trunk angular dynamics. In this paper, we present a new approach based on modifying foot placement targets of the SLIP model. Theoretical analysis and numerical results show that the proposed approach can be alternative to existing strategies.

Constructing a spatial map of an indoor environment, e.g., a typical office environment with glass surfaces, is a difficult and challenging task. Current state-of-the-art, e.g., camera- and laser-based approaches are unsuitable for detecting transparent surfaces. Hence, the spatial map generated with these approaches are often inaccurate. In this paper, a method that utilizes echolocation with sound in the audible frequency range is proposed to robustly localize the position of an acoustic reflector, e.g., walls, glass surfaces etc., which could be used to construct a spatial map of an indoor environment as the robot moves. The proposed method estimate the acoustic reflector's position, using only a single microphone and a loudspeaker that are present on many socially assistive robot platforms such as the NAO robot. The experimental results show that the proposed method could robustly detect an acoustic reflector up to a distance of 1.5 m in more than 60% of the trials and works efficiently even under low SNRs. To test the proposed method, a proof-of-concept robotic platform was build to construct a spatial map of an indoor environment.

This paper presents novel control techniques for passivation and stabilisation of floating-base systems with contacts, whose dynamical models comprise both joint-space, and Cartesian floating-base coordinates. The aforementioned results are achieved using both minimally model-based, and completely model-free controllers that employ power-shaping signals. Model-free control is permitted through usage of a decoupled dynamical model, procured via coordinate transformation operations. It is demonstrated that even though passive closed-loop systems are attainable without utilisation of exteroceptive feedback, global stabilisation of a floating-base robot necessitates direct usage of either measured or estimated external forces. The presented asymptotical stabilisation results pertain to both the set-point regulation, and trajectory-tracking cases, thereby ensuring suitability for static balancing, and dynamical locomotion tasks. To ensure practicability and production of feasible input signals, a variable impedance control, power-shaping term is appended to the original design, wherein it circumstantially serves as either a power-dissipating, or power-injecting element. This enhancement provably preserves closed-loop stability, by appositely shaping the system's power. Experiments involving a metamorphic, quadrupedal walking robot, corroborate the theoretical analysis, as they attest to the system's ability to stably execute locomotory tasks using a single, unified, model-free control scheme.

Truss structures can be found in many buildings and civil infrastructure such as bridges and towers. But as these architectures age, their maintenance is required to keep them structurally sound. A legged robotic solution capable of climbing these structures for maintenance is sought, but determining the size and shape of such a robot to maximise structure coverage is a challenging task. This paper proposes a model in which the size of a multi-legged robot is optimised for coverage in a truss structure. A detailed representation of a truss structure is presented, which forms the novel framework for constraint modelling. With this framework, the overall truss structure coverage is modelled, given a robot's size and its climbing performance constraints. This is set up as an optimisation problem, such that its solution represents the optimum size of the robot that satisfies all constraints. Three case studies of practical climbing applications are conducted to verify the model. By intuitive analysis of the model's output data, the results show that the model accurately applies these constraints in a variety of truss structures.

In this paper, we present a multimodal mobile teleoperation system that consists of a novel vision-based hand pose regression network (Transteleop) and an IMU (inertial measurement units)-based arm tracking method. Transteleop observes the human hand through a low-cost depth camera and generates not only joint angles but also depth images of paired robot hand poses through an image-to-image translation process. A keypoint-based reconstruction loss explores the resemblance in appearance and anatomy between human and robotic hands and enriches the local features of reconstructed images. A wearable camera holder enables simultaneous hand-arm control and facilitates the mobility of the whole teleoperation system. Network evaluation results on a test dataset and a variety of complex manipulation tasks that go beyond simple pick-and-place operations show the efficiency and stability of our multimodal teleoperation system.

To allow wheelchair (electronic or manual) users to practice driving in different safe, repeatable and controlled scenarios, the use of simulator as a training tool is considered here. In this context, the capabilities of providing high fidelity motions for users of the simulator is highlighted as one of the most important aspects for the effectiveness of the tool. For this purpose, the motion cueing algorithm (MCA) is studied in our work to regenerate wheelchair motion cues by transforming motions of the real or simulated wheelchair into the simulator motion. The studied algorithm is developed based on Model Predictive Control (MPC) approach to efficiently optimize the motions of the platform. The overall problem is formulated using mixed-integer quadratic programming (MIQP) which involves not only the vestibular model, strict constraints of the platform but also the perception threshold in the optimization cost function. In the end, the performance assessment of the system using different control techniques is analyzed, showing the effectiveness of the proposed approach in the simulation environment.

Robotic manipulators can be found today in most industries, from autonomous warehouses to advanced assembly lines in factories. Most of these industrial robots are characterized by having non-flexible and highly rigid links. In dense and complex environments these manipulators require many degrees of freedom (DOFs) which complicates the mechanical structure of the manipulator, as well as the control and path planning algorithms. In this work we present a minimalistic approach to reduce the number of active DOFs by using non-rigid, Hyper-Flexible Manipulators (HFM). We introduce a dynamic model of the HFM as well as a control scheme to bring the end-effector to a desired position from known initial configuration. Finally, we present experiments that support the analytic part and simulative results of this paper.

Restoring tactile sensation is essential to enable in-hand manipulation and the smooth, natural control of upper-limb prosthetic devices. Here we present a platform to contribute to that long-term vision, combining an anthropomorphic robot hand (QB SoftHand) with a neuromorphic optical tactile sensor (neuroTac). Neuromorphic sensors aim to produce efficient, spike-based representations of information for bio-inspired processing. The development of this 5-fingered, sensorized hardware platform is validated with a customized mount allowing manual control of the hand. The platform is demonstrated to succesfully identify 4 objects from the YCB object set, and accurately discriminate between 4 directions of shear during stable grasps. This platform could lead to wide-ranging developments in the areas of haptics, prosthetics and telerobotics.

A practical robotic bin-picking system requires a high grasp success rate for various objects. Also, the system must be capable of coping with various constraints and their changes flexibly. To resolve these issues, this study proposes a novel deep learning-based method that exploits a simulator to generate desired grasping actions. The features of this method are as follows: (1) Grasping conditions for any object can be flexibly customizable in the simulated environment to improve the real-world grasping actions. (2) Sensor input (RGB image) is directly regressed to grasping actions by using convolutional processing. Owing to these features, the system using the proposed method can grasp objects with geometric variations, semi-transparent objects, and objects with a biased center of gravity. Experimental results on a real robot system show that the proposed method exhibits a high grasp success rate for four types of objects with different physical and geometric properties as well as additional constraints of grasping condition.

Surgical knot tying is one of the most fundamental and important procedures in surgery, and a high-quality knot can significantly benefit the postoperative recovery of the patient. However, a longtime operation may easily cause fatigue to surgeons, especially during the tedious wound closure task. In this paper, we present a vision-based method to automate the suture thread grasping, which is a sub-task in surgical knot tying and an intermediate step between the stitching and looping manipulations. To achieve this goal, the acquisition of a suture's three-dimensional (3D) information is critical. Towards this objective, we adopt a transfer-learning strategy first to fine-tune a pre-trained model by learning the information from large legacy surgical data and images obtained by the onsite equipment. Thus, a robust suture segmentation can be achieved regardless of inherent environment noises. We further leverage a searching strategy with termination policies for a suture's sequence inference based on the analysis of multiple topologies. Exact results of the pixel-level sequence along a suture can be obtained, and they can be further applied for a 3D shape reconstruction using our optimized shortest path approach. The grasping point considering the suturing criterion can be ultimately acquired. Experiments regarding the suture 2D segmentation and ordering sequence inference under environmental noises were extensively evaluated. Results related to the automated grasping operation were demonstrated by simulations in V-REP and by robot experiments using Universal Robot (UR) together with the da Vinci Research Kit (dVRK) adopting our learning-driven framework.

We propose a human-robot interface based on potentials recorded through surface Electroencephalographic sensors, aiming to decode human visual attention into motion in three-dimensional space. Low-frequency components are extracted and processed in real time, and subspace system identification methods are used to derive the optimal, in mean squared sense, linear dynamics generating the position vectors. This results in a human-robot interface that can be used directly in robot teleoperation or as part of a shared-control robotic manipulation scheme, feels natural to the user, and is appropriate for upper extremity amputees, since it requires no limb movement. We validate our methodology by teleoperating a redundant, anthropomorphic robotic arm in real time. The system's performance outruns similar EMG-based systems, and shows low long-term model drift, indicating no need for frequent model re-training.

Rigid-terrain unmanned ground vehicles(UGV) can run under the field environment by the advanced adaptive ability. This paper presents a novel horse inspired rigid-terrain eight-wheel vehicle with four-swing arms. This unmanned ground vehicle is drived by distributed hydraulic motors. By cooperating with four-swing arms and eight wheels, the vehicle has the ability to work like a horse climbs an obstacle under the complex ground. The mechanism, bionic obstacle surmounting algorithm and operation strategy are analyzed in detail. The posture planning of wheel arms and the kinematic model of the UGV are studied. Automatic Dynamic Analysis of Mechanical Systems (ADAMS) simulation results and prototype experiments are executed to verify the analysis and strategy. The results show that this type of unmanned ground vehicle has good performance on crossing the obstacle and running on the rigid-terrain ground.

We present a method for optimal coordination of multiple vehicle teams when multiple endpoint configurations are equally desirable, such as seen in the autonomous assembly of formation flight. The individual vehicles’ positions in the formation are not assigned a priori and a key challenge is to find the optimal configuration assignment along with the optimal control and trajectory. Commonly, assignment and trajectory planning problems are solved separately. We introduce a new multi-vehicle coordination paradigm, where the optimal goal assignment and optimal vehicle trajectories are found simultaneously from a viscosity solution of a single Hamilton–Jacobi (HJ) partial differential equation (PDE), which provides a necessary and sufficient condition for global optimality. Intrinsic in this approach is that individual vehicle dynamic models need not be the same, and therefore can be applied to heterogeneous systems. Numerical methods to solve the HJ equation have historically relied on a discrete grid of the solution space and exhibits exponential scaling with system dimension, preventing their applicability to multiple vehicle systems. By utilizing a generalization of the Hopf formula, we avoid the use of grids and present a method that exhibits polynomial scaling in the number of vehicles.

We consider the problem of visual imitation learning without human kinesthetic teaching or teleoperation, nor access to an interactive reinforcement learning training environment. We present a geometric perspective to this problem where geometric feature correspondences are learned from one training video and used to execute tasks via visual servoing. Specifically, we propose VGS-IL (Visual Geometric Skill Imitation Learning), an end-to-end geometry-parameterized task concept inference method, to infer globally consistent geometric feature association rules from human demonstration video frames. We show that, instead of learning actions from image pixels, learning a geometry-parameterized task concept provides an explainable and invariant representation across demonstrator to imitator under various environmental settings. Moreover, such a task concept representation provides a direct link with geometric vision based controllers (e.g. visual servoing), allowing for efficient mapping of high-level task concepts to low-level robot actions.

Interactive decision-making and motion planning are important to safety-critical autonomous agents, particularly when they interact with humans. Many different interaction strategies can be exploited by humans. For instance, they might ignore the autonomous agents, or might behave as selfish optimizers by treating the autonomous agents as opponents, or might assume themselves as leaders and the autonomous agents as followers who should take responsive actions. Different interaction strategies can lead to quite different closed-loop dynamics, and misalignment between the human's policy and the autonomous agent's belief over the policy will severely impact both safety and efficiency. Moreover, a human's interaction policy can change as interaction goes on. Hence, autonomous agents need to be aware of such uncertainties on the human policy, and integrate such information into their decision-making and motion planning algorithms. In this paper, we propose a policy-aware interaction strategy based on game theory. The goal is to allow autonomous agents to estimate humans' interactive policies and respond consequently. We validate the proposed algorithm with a roundabout scenario with real traffic data. The results show that the proposed algorithm can yield trajectories that are more similar to the ground truth than those with fixed policies. Also, we estimate how humans adjust their interaction strategies statistically based on the proposed algorithm.

The objective of this paper is to present a personalisable human ergonomics framework that integrates a method for real-time identification of a human model and an ergonomics monitoring function. The human model is based on a floating base structure and on a Statically Equivalent Serial Chain (SESC) model used for the estimation of the whole-body centre of Mass (CoM). A recursive linear regression algorithm (i.e., Kalman filter) is developed to achieve the online identification of the SESC parameters. A visual feedback provides a minimum set of suggested human poses to speed up the identification process, while enhancing the model accuracy based on a convergence value. The online ergonomics monitoring function computes and displays the overloading effects on body joints in heavy lifting tasks. The overloading joint torques are calculated based on the displacement of the Center of Pressure (CoP) between the measured one and the estimated one. Unlike our previous work, the entire process, from the model identification (personalisation) to ergonomics monitoring, is performed in real-time. We evaluated the efficacy of the proposed method through human experiments during model identification and load lifting tasks. Results demonstrate the high exploitation potential of the framework in industrial settings, due to its fast personalisation and ergonomics monitoring capacity.