IROS 2022 Papers

In this paper, we propose a novel torque controller for the implementation virtual remote center of motion. The controller allows the system to implement the required behavior and guarantees the satisfaction of the remote center of motion constraint. Exploiting the Udwadia-Kalaba equation for constrained dynamic systems, the controller is synthesized considering the dynamic effect the constraint produces on the manipulator, achieving more effective control with respect to kinematic strategies, and allowing the implementation of compliance behaviors. Simulations and experimental validation with a KUKA LWR 4+ with 7 degrees of freedom has been performed to check the performances of the proposed controller. Results show the effectiveness of the proposed controller with different control action, and the capability to interact with the environment by implementing compliant motion control.

In this paper, we introduce an event-based visual odometry and mapping framework that relies on decaying event-based corners. Event cameras, unlike conventional cam-eras, can provide sensor data during high-speed motions or in scenes with high dynamic ranges. Rather than providing intensity information at a global shutter rate, events are trig-gered asynchronously depending on whether there is a change in brightness at the pixel location. This novel sensing paradigm calls for unconventional ego-motion estimation techniques to address these new challenges. The key aspect of our framework is the use of a continuous representation of inertial measurements to characterise the system's motion which accommodates the asynchronous nature of the event data while estimating a discrete state in an optimisation-based approach. The proposed method relies on corners extracted from events-only data and associates them with a spatio-temporal locality scheme based on exponential decay. Event tracks are then tightly coupled with temporally accurate preintegrated inertial measurements, allowing for the estimation of ego-motion and a sparse map. The proposed method is evaluated on the Event Camera Dataset showing performance against the state-of-art in event-based visual-inertial odometry.

In this paper, we discuss a framework for teaching bimanual manipulation tasks by imitation. To this end, we present a system and algorithms for learning compliant and contact-rich robot behavior from human demonstrations. The presented system combines insights from admittance control and machine learning to extract control policies that can (a) recover from and adapt to a variety of disturbances in time and space, while also (b) effectively leveraging physical contact with the environment. We demonstrate the effectiveness of our approach using a real-world insertion task involving multiple simultaneous contacts between a manipulated object and insertion pegs. We also investigate efficient means of collecting training data for such bimanual settings. To this end, we conduct a human-subject study and analyze the effort and mental demand as reported by the users. Our experiments show that, while harder to provide, the additional force/torque information available in teleoperated demonstrations is crucial for phase estimation and task success. Ultimately, force/torque data substantially improves manipulation robustness, resulting in a 90% success rate in a multipoint insertion task. Code and videos can be found at https://bimanualmanipulation.com/

This paper presents a method of drawing pipeline routes using a sensing unit with a rotary encoder and IMU (Inertial Measurement Unit), which is towed by a self-propelled in-pipe inspection robot. However, the IMU information generally contains integration errors, making it difficult to draw accurate pipeline routes. In this study, we propose a method combining gradient descent using a gyroscopic sensor and an accelerometer, and the correction of the route based on the standard information of the pipe. First, the method of identifying the start point, end point, direction of straight pipes, and bending direction of curved pipes is explained. Then, an experiment is conducted using the developed robot system on a 11.6-meter-long pipeline course that includes nine curved pipes and horizontal and vertical straight pipes. Consequently, the mean absolute error of the route dimension was reduced to 2.74 %.

This paper is about fast slosh-free fluid transportation. Existing approaches are either computationally heavy or only suitable for specific robots and container shapes. We model the end effector as a point mass suspended by a spherical pendulum and study the requirements for slosh-free motion and the validity of the point mass model. In this approach, slosh-free trajectories are generated by controlling the pendulum's pivot and simulating the motion of the point mass. We cast the trajectory optimization problem as a quadratic program-this strategy can be used to obtain valid control inputs. Through simulations and experiments on a 7 DoF Franka Emika Panda robot we validate the effectiveness of the proposed approach.

Mobile manipulator throwing is a promising method to increase the flexibility and efficiency of dynamic manipulation in factories. Its major challenge is to efficiently plan a feasible throw under a wide set of task specifications. We show that the mobile manipulator throwing problem can be simplified to a planar problem, hence greatly reducing the computational costs. Using machine learning approaches, we build a model of the object's inverted flying dynamics and the robot's kinematic feasibility, which enables throwing motion generation within 1 ms for given query of target position. Thanks to the computational efficiency of our method, we show that the system is adaptive under disturbance, via replanning on the fly for alternative solutions, instead of sticking to the original throwing plan.

This paper presents a proof-of-concept soft robotic glove that provides haptic feedback to the surgeon's hand during interventional endoscopy procedures, specifically colonoscopy. The glove is connected to a force sensing soft robotic sleeve that is mounted onto a colonoscope. The glove consists of pneumatic actuators that inflate in proportion to the incident forces on the soft robotic sleeve. Thus, the glove is capable of alerting the surgeon of potentially dangerous forces exerted on the colon wall by the colonoscope during the navigation. The proposed glove is adaptable to a variety of hand sizes. It features modular actuators that facilitate convenient and rapid assembly and attachment before the procedure and removal afterward. The glove is calibrated to respond to incident forces on the soft robotic sleeve ranging from 0–3 N. The glove's actuators are able to reach an internal pressure of 53 kPa and exert forces up to 20 N, thereby relaying and amplifying the force exerted by the colonoscope on the colon to the surgeon's hand.

Achieving compact and biocompatible actuators with sensing capabilities is a key challenge for the safety critical and highly patient-specific biomedical field. In this study, a compact and versatile soft fluidic sensor-actuator capable of measuring both force and displacement in static and dynamic conditions is presented. Pressure and resistance are shown to be interchangeable in predicting load and sensor-actuator height, and showed good repeatability and distinction between the loaded and constrained conditions tested. Furthermore the sensor-actuator is demonstrated in a probe application and showed comparable findings to a tensile test machine when tested on three objects of varying stiffness. Overall, this sensor-actuator has the potential to be a key building block for biomedical robots that require large expansion, as well as continuous monitoring of both displacement and force.

Upper limb impairments and weakness are com-mon post-stroke and with advanced aging. Rigid exoskeletons have been developed as a potential solution, but have had limited impact. In addition to user concerns about safety, their weight and appearance, the rigid attachment and typical anchoring methods can result in skin damage. In this paper, we present a soft, fabric-based pneumatic sleeve, which can shrink from a loose fit to a tight fit in order to anchor to the limbs temporarily, thereby enabling the application of mechanical assistance only when needed. The sleeve is comfortable, ergonomic and can be embedded unobtrusively with clothing. A mathematical model is built to simulate and design sleeves with different geometric parameters. The best sleeve was capable of generating a friction force of 98 N on the limb when inflated to 25 kPa. This sleeve was used to create a wearable assistive device, integrated with a cable-driven actuator. This device was able to lift a 1.44 kg forearm rig up to 95 degree at low pressure of 20 kPa. The device was tested with six healthy participants, in terms of fit, comfort and assistive functionality. The average acceptable sleeve pressure was found to be 33±4.7 kPa. All participants liked the appearance of the sleeve, with a high average perceived assistance score of 7.33±1.6 (out of 10). The shrink-to-fit sleeve is expected to significantly increase the development and adoption of soft robotic assistive devices and emerging powered clothing.

As with all actuated mechanical systems, the development of a control scheme for robotic systems must take actuator saturation into account. Furthermore, in order to properly control a robotic arm, other types of mechanical limitations must also be considered (e.g. limited operating range of the joint, speed limitations). In this paper, we propose a saturation-aware trajectory-based Explicit Reference Governor, a lightweight constrained control scheme with no online optimization. Additionally, we tested the effectiveness of the proposed control architecture on a 7 degree-of-freedom KUKA LBR IIWA14 R820. Both the performance and the computational time of our proposed solution were tested against other constrained control solutions available in the literature.

Planning for autonomous operation in unknown environments poses a number of technical challenges. The agent must ensure robustness to unknown phenomena, un-predictable variation in execution, and uncertain resources, all while maximizing its objective. These challenges are ex-acerbated in the context of space missions where uncertainty is often higher, long communication delays necessitate robust autonomous execution, and severely constrained computational resources limit the scope of planning techniques that can be used. We examine this problem in the context of a Europa Lander concept mission where an autonomous lander must collect valuable data and communicate that data back to Earth. We model the problem as a hierarchical task network, framing it as a utility maximization problem constrained by a strictly monotonically decreasing energy resource. We propose a novel deterministic planning framework that uses periodic replanning and sampling-based optimization to better handle model uncertainty and execution variation, while remaining computationally tractable. We demonstrate the efficacy of our framework through simulations of a Europa Lander concept mission in which our approach outperforms several baselines in utility maximization and robustness.

Navigation through unknown, cluttered environments is a fundamental and challenging task for autonomous vehicles as they must deal with a myriad of obstacle configurations typically unknown a priori. Challenges arise because obstacles of unknown shapes and dimensions can create occlusions limiting sensor field of view and leading to uncertainty in motion planning. In this paper we propose to leverage such occlusions to quickly explore and cover unknown cluttered environments. Specifically, this work presents a novel occlusion-aware frontier-based approach that estimates gaps in point cloud data and shadows in the field of view to generate waypoints to navigate. Our scheme also proposes a breadcrumbing technique to save states of interest during exploration that can be exploited in future missions. For the latter aspect we focus primarily on the generation of the minimum number of breadcrumbs that will increase coverage and visibility of an explored environment. Extensive simulations and experiment results on an unmanned ground vehicle (UGV) are demonstrated to validate the proposed technique, showing improvements over traditional state of the art frontier-based exploration methods.

Last-mile delivery robots are usually required to navigate on the sidewalk through a fixed route. The current solutions heavily rely on the image-based perception and GPS localization to successfully complete delivery tasks. However, it is prone to fail and become unreliable when the robot runs in challenging conditions, such as operating in different illuminations, or under canopies of trees or buildings. To address these issues, this paper proposes a novel robust sidewalk navigation method for the last-mile delivery robots with an affordable sparse LiDAR, which consists of two main modules: Semantic Point Cloud Network (SegPCn) and Reactive Nav-igation Network (RNn), as shown in Fig. 1. More specifically, SegPCn takes the raw 3D point cloud as input and predicts the point-wise segmentation labels, presenting a robust perception capability even in the night. Then, the semantic point clouds are fed to RNn to generate an angular velocity to navigate the robot along the sidewalk, where the localization of the robot is not required. Moreover, an autolabeling mechanism is developed to reduce the labor involved in data preparation as well. And the LSTM neural network is explored to effectively leverage the historical context and derive correct decisions. Extensive experiments have been carried out to verify the efficacy of this method, and the results show that this method enables the robot to navigate on the sidewalk robustly during day and night. We open source the code and the data set on https://github.com/lukewenMX/Robust-Navigation-Method.

Frequently, ground robots are hampered by debris and objects on the ground, and safely surpassing them is not always trivial. On the contrary, a robot capable of flying is intrinsically immune to such obstacles and, therefore, greatly enhances the possibility of inspecting and intervening in adverse surroundings for humans. This work introduces a novel teleoperated aerial platform for inspection and intervention in unstructured environments. The robot is composed of an aerial base, two arms, and a two-degrees-of-freedom head that consent the access of human operators in any workplace in total safety. The arms are designed with a joint structure of tendons and are held by elastic components. This composition considerably improves the robustness by inserting softness and redistributing the weights to lessen the actions on the drone. Moreover, the aerial platform employs two soft hands capable of adapting to the shape of the objects under grasp, increasing the manipulation performance. We presented the mechanical and control design, a gazebo simulation employed to test the controllers, and a physical structure for the experimental validation of the system. The system is available as Open-Source material.

Robots are increasingly being called on to operate in settings and on tasks originally designed for humans, or where humans are also expected to work. Accordingly, the hardware and tools to be packaged, operated, or maintained are typically designed for use by humans, not robots. Robot autonomy in such cases can be expedited by a “robot factors” approach to the design of hardware, analogous to ergonomics for humans, taking typical current robot capabilities into account during the design process. In this paper, we present two case studies of redesigning mission-critical hardware in space habitats to facilitate autonomous robot operation. In both cases, hardware that previously required dexterous bi-manual manipulation is redesigned such that the entire maintenance task can be completed by a single robotic arm with a standard parallel jaw gripper. We demonstrate successful autonomous replacement of modules in the two hardware systems, and characterize how orientation and compliance of a grasp helps compensate for positioning errors. Based on our findings, we identify several key design strategies that underpin the robot factors approach to designing robot-friendly hardware, including consolidating compound actions into simpler mechanisms, constraining required motions to a single axis, and introducing mechanical compliance to mitigate the effects of pose uncertainties.

Dynamic motions of humans and robots are widely driven by posture-dependent nonlinear interactions between their degrees of freedom. However, these dynamical effects remain mostly overlooked when studying the mechanisms of human movement generation. Inspired by recent works, we hypothesize that human motions are planned as sequences of geodesic synergies, and thus correspond to coordinated joint movements achieved with piecewise minimum energy. The underlying computational model is built on Riemannian geometry to account for the inertial characteristics of the body. Through the analysis of various human arm motions, we find that our model segments motions into geodesic synergies, and successfully predicts observed arm postures, hand trajectories, as well as their respective velocity profiles. Moreover, we show that our analysis can further be exploited to transfer arm motions to robots by reproducing individual human synergies as geodesic paths in the robot configuration space.

Proprioception, or the perception of the configuration of one's body, is challenging to achieve with soft robots due to their infinite degrees of freedom and incompatibility with most off-the-shelf sensors. This work explores the use of inertial measurement units (IMUs), sensors that output orientation with respect to the direction of gravity, to achieve soft robot proprioception. A simple method for estimating the shape of a soft continuum robot arm from IMUs mounted along the arm is presented. The approach approximates a soft arm as a serial chain of rigid links, where the orientation of each link is given by the output of an IMU or by spherical linear interpolation of the output of adjacent IMUs. In experiments conducted on a 660mm long real-world soft arm, this approach provided estimates of its end effector position with a median error of less than 10% of the arm's length. This demonstrates the potential of IMUs to serve as inexpensive off-the-shelf sensors for soft robot proprioception.

We present a portable multiscopic camera system with a dedicated model for novel view and time synthesis in dynamic scenes. Our goal is to render high-quality images for a dynamic scene from any viewpoint at any time using our portable multiscopic camera. To achieve such novel view and time synthesis, we develop a physical multiscopic camera equipped with five cameras to train a neural radiance field (NeRF) in both time and spatial domains for dynamic scenes. Our model maps a 6D coordinate (3D spatial position, 1D temporal coordinate, and 2D viewing direction) to view-dependent and time-varying emitted radiance and volume density. Volume rendering is applied to render a photo-realistic image at a specified camera pose and time. To improve the robustness of our physical camera, we propose a camera parameter optimization module and a temporal frame interpolation module to promote information propagation across time. We conduct experiments on both real-world and synthetic datasets to evaluate our system, and the results show that our approach outperforms alternative solutions qualitatively and quantitatively. Our code and dataset are available at https://yuenfuilau.github.io/.

Percutaneous coronary intervention (PCI) involves the delivery of a flexible submillimeter guidewire and existing x- ray based approaches impose significant ironing radiation. The use of magnetic resonance imaging (MRI) for intraoperative guidance has the advantages of not only being safe but also having high positioning accuracy and excellent tissue contrast. This paper develops a pneumatically driven MR-conditional delivery mechanism for the ease of manipulation of the guidewire in vivo. It incorporates newly developed rotary pneumatic step motors and a pneumatic slip ring for actuation and decoupling of translational and rotational motions. An effective clamping mechanism for the locking and releasing of the guidewire is also incorporated. The proposed pneumatic slip ring mechanism decouples six gas lines, where four are used to supply a pneumatic step motor for translational motion and two for the clamping mechanism. High friction sil sleeve is used to hold the guidewire firmly. The rotary pneumatic motor has excellent sealing and stability, providing an output torque of 15.75 Nm/MPa. Experiments show that the average error of translational motion is 0.37 mm. Real-time MRI-guided endovascular intervention is performed in a vascular phantom with pulsatile flows to validate its potential clinical use. The imaging artifact test under MRI shows no noticeable distortion and the loss of Signal-to-Noise Ratio (SNR) is less than 2%.