ICRA 2019 Papers

This article describes an innovative design of variable stiffness soft actuator, which can potentially be utilized for manipulation and locomotion of soft robots. The new actuator is a combination of two types of actuations: soft pneumatic actuation and muscle-like supercoiled polymer (SCP) actuation. Soft pneumatic actuator has two roles: first is to generate bending motions and second is to increase the stiffness of the whole actuator together with SCP artificial muscles. SCP artificial muscles are exploited to generate pre-load to resist the whole actuator from (excessive) deformation when external load is applied. These two types of actuations are arranged antagonistically to realize stiffness tuning of the whole actuator. At a given bending position, stiffness of the actuator could be tuned by controlling the pressure inside the air chamber and the tension on the SCP artificial muscles. In experimental section, tests are conducted to characterize the applied SCP artificial muscles before they are applied to the proposed actuator. Afterwards, tests of proposed actuator are performed to examine its variable stiffness capability. From experimental results, the proposed actuator can achieve 3.47 times stiffness variation ratio from 0.0312 N/mm(40kPa air pressure and no SCP actuation) to 0.1083 N/mm(82kPa air pressure and SCP actuation at 0.143 W/cm) at the same position (bending angle of 56 degree). This study exhibits the potential of applying SCP artificial muscles to promote the performance of soft robots.

In this paper, we present a novel design concept for a robot arm link with variable stiffness. Variable stiffness links are intended to grant a robot the safety benefits of compliance and the performance benefits of stiffness. Our compact design actively modulates stiffness via parallel, rotating beams actuated by simple servomotors. It achieves a lateral stiffness ratio greater than ten with a minimum stiffness under 0.2 N/mm. Our novel design offers many benefits over existing variable stiffness link solutions in its compactness, simplicity, and speed of actuation. One challenge of this research lies in the mechanics modeling of variable stiffness. Here we propose a comprehensive mechanics model that considers mechanical compliances due to deflections of parallel guided beams, column buckling, and bearing at the beam roots. By comparing with experimental testing data, we show that our analytical model accurately predicts the lateral stiffness of the robotic link. This model can be used as a design tool in future iterations, including for scaling the design.

Surface finishing of freeform surfaces is predominately a manual operation that requires a considerable amount of operator skill; automation of this process has many benefits, including consistent surface quality, preventing hazardous exposure to particulate, etc. A novel robotic surface finishing system, consisting of a robot and an end-effector that includes a force sensor, finishing tool, and proximity laser sensor, is developed in this paper to automate the surface finishing process. The laser sensor is treated as an additional link, and based on it a novel perception system is developed for real-time scanning of the surface that provides the surface profile mesh and the corresponding normal vectors which can be used directly by the robot closed-loop control system for pose tracking. A unique feature of the perception system is that the geometry of the surface profile and normal vectors are all obtained in real-time in the robot base coordinate system, thus eliminating issues such as precise registration of the work piece in the fixture and its location with respect to the robot base coordinates. An impedance-type closed-loop control algorithm is developed for pose tracking. The proposed system and control algorithm are employed to conduct surface finishing experiments on wooden surfaces. A representative sample of the results and measurement images of surface finish are provided to illustrate the capabilities of the robotic surface finishing system. A video of the system in operation is also provided.

Perforations in flexible endoscopy are life-threatening. Defect closure or suturing in flexible endoscopy has long been a critical challenge due to the confined space of the access routes and surgical sites, high dexterity and force demands of suturing tasks, as well as critical size and strength requirements of wound closure. This paper introduces a novel robotic suturing system for flexible endoscopic surgery. This system features a flexible, through-the-scope, five-degree-of-freedom robotic suturing instrument. This instrument allows the surgeon to endoscopically manipulate a needle via a master console to create running stitches and knots in flexible endoscopy, which is not possible with existing devices. Successful ex-vivo trials were conducted inside porcine colons to show how surgical stitches and knots can be endoscopically created and secured in a completely new way. This new technology will change the way how surgeons close defects or perforations in flexible endoscopic surgery.

In this paper, a novel revolute joint of adjustable stiffness with reconfigurability (JASR) is presented. The JASR is designed with zero-length base link four-bar linkage, and allows adjusting its stiffness to achieve soft- and hard-spring behaviour. The new joint has a compact and light-weight structure and can be integrated in robot and transmissions for different applications. In the paper, mathematical models are developed for the JASR, with which influences of design parameters on stiffness performance are analyzed. A prototype of JASR is constructed and preliminary test results demonstrate the compliance properties of the new joint.

For tiny obstacle discovery in a monocular image, edge is a fundamental visual element. Nevertheless, because of various reasons, e.g., noise and similar color distribution with background, it is still difficult to detect the edges of tiny (b) obstacles at long distance. In this paper, we propose an obstacle-aware discovery method to recover the missing contours of these obstacles, which helps to obtain obstacle proposals as much as possible. First, by using visual cues in monocular images, several multi-layer regions are elaborately inferred to reveal the distances from the camera. Second, several novel obstacle-aware occlusion edge maps are constructed to well capture the contours of tiny obstacles, which combines cues from each layer. Third, to ensure the existence of the tiny obstacle proposals, the maps from all layers are used for proposals extraction. Finally, based on these proposals containing tiny obstacles, a novel obstacle-aware regressor is proposed to generate an obstacle occupied probability map with high confidence. The convincing experimental results with comparisons on the Lost and Found dataset demonstrate the effectiveness of our approach, achieving around 9.5% improvement on the accuracy than FPHT and PHT, it even gets comparable performance to MergeNet. Moreover, our method outperforms the state-of-the-art algorithms and significantly improves the discovery ability for tiny obstacles at long distance.

Laser scalpels are utilized across a variety of surgical and dermatological procedures due to their precision and non-contact nature. This paper presents a novel laser scalpel system for superficial laser therapy applications. The system integrates a RGB-D camera, a 3D triangulation sensor and a carbon dioxide ( CO _{2})( CO _{2}) laser scalpel for computer-assisted laser surgery. To accurately ablate targets chosen from the color image, a 3D extrinsic calibration method between the RGB-D camera frame and the laser coordinate system is implemented. The accuracy of the calibration method is tested on phantoms with planar and cylindrical surfaces. Positive error and negative error, as defined as undershooting and overshooting over the target area, are reported for each test. For 60 total test cases, the root-mean-square of the positive and negative error in both planar and cylindrical phantoms is less than 1.0 mm, with a maximum absolute error less than 2.0 mm. This work demonstrates the feasibility of automated laser therapy with surgeon oversight via our sensor system.

Soft robots attract research interests worldwide. However, its control remains challenging due to the difficulty in sensing and accurate modeling. In this paper, we propose a novel iterative learning model predictive control (ILMPC) method for soft bending actuators. The uniqueness of our approach is the ability to improve model accuracy gradually. In this method, a pseudo-rigid-body model is used to take an initial guess of the bending behavior of the actuator and the model accuracy is improved with iterative learning. Compared with conventional model free iterative learning control (ILC), the proposed method significantly reduces the learning curve. Compared with the model predictive control (MPC), the proposed method does not rely on an accurate model and it will output a satisfactory model after the learning process. A soft-elastic composite actuator (SECA) is used to validate the proposed method. Both simulation and experimental results show that the proposed method outperforms the conventional MPC and ILC.

We develop two cooperative robots for power transmission lines (PTLs) inspection - a light climbing robot (CBR) which can stably move on the overhead ground wire (OGW) for sensor data collection and an unmanned aerial vehicle (UAV) with a grabbing mechanism, which can automatically put the CBR on the OGW and take it off. In order to guarantee the safety, the mechanical structures of the connectors are designed in the shape of a trumpet. Further, a self-locked structure of the CBR is developed to automatically seize and release the OGW. For autonomous navigation, the UAV is equipped with a movable sliding rail and a 2D Laser Range Finder (LRF). The LRF can not only detect the position and orientation of the OGW but also detect the top beam of the CBR and the grabbing position in it. Furthermore, the action of the grabbing mechanism is automatically triggered by a microswitch. Finally, by the developed UAV and CBR platforms, we test the whole loading and unloading strategy in an artificially constructed PTLs environment outdoors and achieve an encouraging result1. Combining the flexible motion of the UAV and the high inspection accuracy of the CBR, the CBR can negotiate any obstacle by flying and abandon the traditional heavy obstacle crossing mechanism to effectively realize close-proximity inspection. Due to the light weight and low power consumption, the CBRs can be deployed once in many power corridors to conduct a long-term inspection.

Fully insertable laparoscopic cameras feature more locomotive flexibility in a larger workspace compared to conventional trocar-based laparoscopes and thus represent a promising future of minimally invasive surgery. These cameras are principally anchored and actuated by transabdominal magnetic coupling. Although several proof-of-concept prototypes have shown the technical feasibility in terms of camera actuation and laparoscopic imaging, none of them are getting close to clinical practice due to concerns about safety. One common problem lies in that the interaction force between the camera and the abdominal wall tissue is completely unknown and not controlled. The camera is being manipulated in an open loop which exposes the patient to a high risk of being injured. In this paper, a noninvasive real-time camera-tissue interaction force measurement approach for an insertable laparoscopic camera is proposed, implemented, and validated.Ex-vivo experiments using a simulated abdominal cavity have demonstrated the effectiveness of this approach during anchoring, translation, and rotation camera behaviors. Potential surgical impacts enabled by the force feedback have also been exemplified by a robotic-assisted camera control experiment using shared autonomy.

Among the numerous risk factors associated to work-related musculoskeletal disorders (WMSD), repetitive and monotonous movements with light-weight tools are one of the most frequently cited. Such tasks may indeed result in the excessive accumulation of local muscle fatigue, causing severe injuries in human joints. Accordingly, this paper proposes a new whole-body fatigue model to evaluate the cumulative effect of the overloading torque induced on the joints over time by light payloads. The proposed model is then integrated into a human-robot collaboration (HRC) framework to set the timing of a body posture optimisation procedure guided by the robot assistance, by the time fatigue overcomes a threshold in any joint. Our overloading fatigue model is based on an estimation method we developed in a previous work, to monitor joint torque variations due to external forces in real-time. To account for individuals' different perception of fatigue, the fatigue ratio parameter in the model is computed experimentally for each subject. The proposed model is first studied on ten subjects by means of an electromyography analysis. Next, its performance is assessed in a painting task and finally evaluated within the HRC framework, which is proved to be able to reduce the risk of injuries caused by excessive fatigue accumulation.

A new solution to improving the poor endurance of the existing hybrid aerial underwater vehicle (HAUV) is proposed in this paper. The proposed multimodal hybrid aerial underwater vehicle (MHAUV) merges the design concept of the fixed-wing unmanned aerial vehicle (UAV), the multirotor, and the underwater glider (UG) and has a novel lightweight pneumatic buoyancy adjustment system. MHAUV is well suited for moving in distinct medium and can achieve extended endurance for long distance travel in both air and water. The mathematical model is given based on Newton-Euler formalism. Necessary design principles of the vehicle's physical parameters are obtained through different gliding equilibrium points. Then, a control scheme composed of two separate proportional-integral-derivative (PID) is employed for the vehicle's motion control in multi-domain simulation. The simulation results are presented to verify the multi-domain mobility and the mode switch ability of the proposed vehicle intuitively. Finally, a prototype, NEZHA, is introduced to be the experimental platform. The success of the flight test, the hovering test, the underwater glide test, and the medium transition test all contribute to prove the feasibility of the proposed concept of the novel MHAUV.

In the future, human-robot interaction will include collaboration in close-quarters where the environment geometry is partially unknown. As a means for enabling such interaction, this paper presents a multi-modal sensor array capable of contact detection and localization, force sensing, proximity sensing, and mapping. The sensor array integrates Hall effect and time-of-flight (ToF) sensors in an I2C communication network. The design, fabrication, and characterization of the sensor array for a future in-situ collaborative continuum robot are presented. Possible perception benefits of the sensor array are demonstrated for accidental contact detection, mapping of the environment, selection of admissible zones for bracing, and constrained motion control of the end effector while maintaining a bracing constraint with an admissible rolling motion.

Generating multi-vehicle trajectories from existing limited data can provide rich resources for autonomous vehicle development and testing. This paper introduces a multi-vehicle trajectory generator (MTG) that can encode multi-vehicle interaction scenarios (called driving encounters) into an interpretable representation from which new driving encounter scenarios are generated by sampling. The MTG consists of a bi-directional encoder and a multi-branch decoder. A new disentanglement metric is then developed for model analyses and comparisons in terms of model robustness and the independence of the latent codes. Comparison of our proposed MTG with β-VAE and InfoGAN demonstrates that the MTG has stronger capability to purposely generate rational vehicle-to-vehicle encounters through operating the disentangled latent codes. Thus the MTG could provide more data for engineers and researchers to develop testing and evaluation scenarios for autonomous vehicles.

Geometric models are crucial for many robotics applications. Current robotic 3D reconstruction systems only focus on specific reconstruction goals which make them hard to adapt to different tasks. In this paper we present a next-best-view framework which allows robots to construct a geometric model incrementally through consecutive sensing actions. Instead of limiting the type and total number of sensors, in each sensing step we evaluate actions from all available sensors and pick the best to execute. Our framework is more comprehensive since the model building process can be designed to best accomplish different tasks. The system has been demonstrated in two experiments on 3D reconstruction and weld seam inspection, yielding promising results.

Visual Place Recognition (VPR) is an important component in both computer vision and robotics applications, thanks to its ability to determine whether a place has been visited and where specifically. A major challenge in VPR is to handle changes of environmental conditions including weather, season and illumination. Most VPR methods try to improve the place recognition performance by ignoring the environmental factors, leading to decreased accuracy decreases when environmental conditions change significantly, such as day versus night. To this end, we propose an end-to-end conditional visual place recognition method. Specifically, we introduce the multi-domain feature learning method (MDFL) to capture multiple attribute-descriptions for a given place, and then use a feature detaching module to separate the environmental condition-related features from those that are not. The only label required within this feature learning pipeline is the environmental condition. Evaluation of the proposed method is conducted on the multi-season NORDLAND dataset, and the multi-weather GTAV dataset. Experimental results show that our method improves the feature robustness against variant environmental conditions.

In this work, we present a decentralized motion planning and control architecture for the cooperative loading task using heterogeneous robotic agents operating in a cluttered workspace with static obstacles. Initially, we tackle the problem of calculating a set of feasible loading configurations via a Probabilistic Road Maps technique. Next, an optimal loading configuration is selected considering the connectivity of the space and the Euclidean distance between the robotic agents. A motion control scheme for each agent is designed and implemented in order to autonomously guide each robot to the desired loading configuration with guaranteed obstacle avoidance and convergence properties. The performance and the applicability of the proposed strategy is experimentally verified in a variety of loading scenarios using a redundant static manipulator and a mobile platform.

We develop a miniature suction-gripper with the goal to realize the novel robotic surgical instrument that can grip slippery and flexible peripheral nerves. In developing the instrument, we place a priority on devising the method that can robustly grip the nerve bundles during the surgical operation for the peripheral nerve. Also, we concentrate to investigate the working principle being able to minimize nerve damages that might be caused when manipulating the nerve. In this study, as the most suitable method to achieve the goal, we scheme to utilize the suction mechanism. Because it can non-invasively grip the nerve based on negative pressure, no external force is applied to the nervous tissues. Therefore the peripheral nerve can be manipulated without serious nerve damage (e.g. crush injury and stretch injury). To improve the gripping ability of the proposed suction gripper, two different types of microneedle arrays are applied to the suction-tips: passive-microneedle (PMN) arrays and active-microneedle (AMN) arrays. Since the most outer membrane of the nerve can be anchored by the penetrated PMN and AMN, the gripper can grip the nerve more robustly. The designed suction-gripper is fabricated as a functional prototype, and its working performances are assessed with in-vitro and in-vivo animal experiments. The experimental results well demonstrate the practical effectiveness of the proposed method and its applicability to the neurosurgical robot for the peripheral nerve.

Whether it is object detection, model reconstruction, laser odometry, or point cloud registration: Plane extraction is a vital component of many robotic systems. In this paper, we propose a strictly probabilistic method to detect finite planes in organized 3-D laser range scans. An agglomerative hierarchical clustering technique, our algorithm builds planes from bottom up, always extending a plane by the point that decreases the measurement likelihood of the scan the least. In contrast to most related methods, which rely on heuristics like orthogonal point-to-plane distance, we leverage the ray path information to compute the measurement likelihood. We evaluate our approach not only on the popular SegComp benchmark, but also provide a challenging synthetic dataset that overcomes SegComp's deficiencies. Both our implementation and the suggested dataset are available at [1].