ICRA 2023 Papers

In this paper, we look into the minimum obstacle displacement (MOD) planning problem from a mobile robot motion planning perspective. This problem finds an optimal path to goal by displacing movable obstacles when no path exists due to collision with obstacles. However this problem is computationally expensive and grows exponentially in the size of number of movable obstacles. This work looks into approximate solutions that are computationally less intensive and differ from the optimal solution by a factor of the optimal cost.

We analyze and model a belt transmission system with non-circular timing pulleys. Using a 3D printer as a proof-of-concept device, experiments consisting of tracking the pose data of a printer nozzle and its pulleys are conducted. A computational model from our previous work is validated with the experimental data and expanded to model more complex systems with multiple non-circular timing pulleys as well as slippage and non-ideal tensions. Finally, an example with two non-circular timing pulleys is presented and simulated utilizing the proposed method.

Between 2% to 5% of children are affected by Developmental Coordination Disorders in Canada and have been diagnosed with upper limb impairments, which affect their daily lives and reduces their autonomy. Motor impairments can be part of progressive disorders, so despite regular therapy, progress remains fleeting. Affected individuals therefore consistently face many barriers, including entertainment opportunities, as availability of off-the-shelf inclusive technology is very limited. Our long-term goal is to develop a play-mediator robot, which would facilitate play between children with motor impairments and their peers or family members. Here, games that the robot can play are remotely controlled by the participants, using appropriate interfaces (e.g. joysticks). In this paper, we take the first step towards that goal and develop an adaptive joystick controller that can compensate for individual deficits. We monitor movement statistics to determine if re-calibration of the controller is necessary. Moreover, we propose a computational model of data ‘distortion’, as a tool for developers to test their technology in the very early stages of prototype development, without requiring access to participants. This work is validated with data from healthy adults and children with upper limb impairments.

The main advantages of legged robots over wheeled ones are their abilities to traverse on uneven terrain due to the use of intermittent contacts and an ability to shift the center of mass relative to the contact location. A robot's leg design can be implemented by using an open-chain mechanism actuated with high-density torque actuators though this solution needs a vast energy budget. An alternative way to design a leg mechanism is the application of morphological computation principle. According to the principle, most of the desired robot's behavior can be delegated to the mechanics with minimum control effort needed to excite, stabilize or augment it. Within this paper, we have proposed a method to synthesize a leg for hopping robots. Due to optimization of mechanical structure, geometric parameters, mass distribution, and elasticity allocation, our method allows getting an energy-efficient robot with minimal control system complexity, which is accomplished via series elastic allocation and active variable length link. Based on this approach, we have designed a hopping robot with two low performance actuators that can achieve hopping, running, and, in the case of a biped or quadruped robot, walking motion. The paper describes a synthesized leg linkage and overviews prototype design, control strategy, and test results of a physical prototype.

We propose a computational approach for designing fully-integrated compliant mechanisms with bio-inspired joints that are stabilized and actuated by elastic elements. Similar to human knees or finger phalanges, our mechanisms leverage sliding between pairs of contacting surfaces to generate complex motions. Due to the vast design space, however, finding surface shapes that lead to ideal approximations of given target motions is a challenging and time-consuming task. To assist users in this process, our computational design tool combines forward and inverse simulation strategies that allow for guided and automated exploration of the parameter space. We demonstrate the potential of our method on a set of compliant mechanism with different joint geometries and validate our simulation results on 3D-printed prototypes.

Miniaturization of robotic grippers enables precise manipulation of small-size objects. However, most microgrippers are actuated by rigid actuators, and thus retain challenges such as micro-fabrication, complex structure, and lack of compliance. Here, we present a compliant microgripper driven by a soft polymer actuator. The proposed millimeter-scale soft polymer actuator can produce a linear displacement and output force with a fast operation. Then, we designed the gripper linkage to convert the linear displacement of the actuator into a gripping motion. Fabricated compliant microgripper has a size of \boldsymbol{10\times 10\times 10}\ \mathbf{mm}^{3}\boldsymbol{10\times 10\times 10}\ \mathbf{mm}^{3} and a weight of 0.36 g, with a maximum gripping width of 8 mm. Demonstration of the gripper shows the feasibility of gripping various sub-millimeter scale objects regardless of their shape owing to its compliance.

Underactuated tendon-driven fingers are a simple, yet effective solution, for realizing robotic grippers and hands. The lack of controllable degrees of actuation and precise sensing is compensated by the deformable structure of the finger, which is able to adapt to the objects to be grasped and manipulated, and also to implement grasping strategies based on environmental constraint exploitation. One of the main drawbacks of these robotic fingers is that, due to the limited number of actuators, they can only realize a limited number of movements. Finger closure motion realized by activating the tendon depends on finger mechanical properties, and in particular on elastic joint stiffness. In this paper, we introduce a passive elastic joint to be implemented in monolithic fingers in which the stiffness can be actively regulated by applying a pre-compression to the structure, controlled by a twisted-string actuator (TSA). The paper describes the working principle of the joint, investigates the relationship between pre-compression and flexural stiffness, and finally shows its application to a robotic finger composed of three phalanges.

The rotation orthonormalization on the special orthogonal group \text{SO}(n)\text{SO}(n), also known as the high dimensional nearest rotation problem, has been revisited. A new generalized simple iterative formula has been proposed that solves this problem in a completely rational manner. Rational operations allow for efficient implementation on various platforms and also significantly simplify the synthesis of large-scale circuitization. The developed scheme is also capable of designing efficient fundamental rational algorithms, for example, quaternion normalization, which outperforms long-exisiting solvers. Furthermore, an \text{SO}(n)\text{SO}(n) neural network has been developed for further learning purpose on the rotation group. Simulation results verify the effectiveness of the proposed scheme and show the superiority against existing representatives. Applications show that the proposed orthonormalizer is of potential in robotic pose estimation problems, e.g., hand-eye calibration.

Model Free Reinforcement Learning (MFRL) has shown significant promise for learning dexterous robotic manipulation tasks, at least in simulation. However, the high number of samples, as well as the long training times, prevent MFRL from scaling to complex real-world tasks. Model- Based Reinforcement Learning (MBRL) emerges as a potential solution that, in theory, can improve the data efficiency of MFRL approaches. This could drastically reduce the training time of MFRL, and increase the application of RL for real- world robotic tasks. This article presents a study on the feasibility of using the state-of-the-art MBRL to improve the training time for two real-world dexterous manipulation tasks. The evaluation is conducted on a real low-cost robot gripper where the predictive model and the control policy are learned from scratch. The results indicate that MBRL is capable of learning accurate models of the world, but does not show clear improvements in learning the control policy in the real world as prior literature suggests should be expected.

This paper addresses the task of joint multi-agent perception and planning, especially as it relates to the real-world challenge of collision-free navigation for connected self-driving vehicles. For this task, several communication-enabled vehicles must navigate through a busy intersection while avoiding collisions with each other and with obstacles. To this end, this paper proposes a learnable costmap-based planning mechanism, given raw perceptual data, that is (1) distributed, (2) uncertainty-aware, and (3) bandwidth-efficient. Our method produces a costmap and uncertainty-aware entropy map to sort and fuse candidate trajectories as evaluated across multiple-agents. The proposed method demonstrates several favorable performance trends on a suite of open-source overhead datasets as well as within a novel communication-critical simulator. It produces accurate semantic occupancy forecasts as an intermediate perception output, attaining a 72.5% average pixel-wise classification accuracy. By selecting the top trajectory, the multi-agent method scales well with the number of agents, reducing the hard collision rate by up to 57% with eight agents compared to the single-agent version.

Visual relocalization is a fundamental problem in computer vision and robotics. Recently, regression-based methods become popular and they can be categorized into two classes: absolute pose regression and scene coordinate regression. In this work, we present a combined regression network that jointly learns scene coordinate regression and absolute pose regression for single-image visual relocalization. The proposed network composes of a feature encoder and two regression branches with uncertainty modeling. In particular, we design a deep feature conditioning module, aiming at propagating the coarse pose information in absolute pose regression to inform the predictions in scene coordinate regression. The proposed network is trained in an end-to-end fashion to learn both regression tasks. Moreover, we propose an uncertainty-driven RANSAC algorithm that incorporates the predicted scene coordinates and their uncertainties to solve the camera pose during inference. To the best of our knowledge, this work is the first to combine scene coordinate regression and pose regression in a hierarchical framework for visual relocalization. Experiments on indoor and outdoor benchmarks demonstrate the effectiveness and the superiority of the proposed method over the state-of-the-art methods.

We present a robust method to visually segment scenes into objects based on motion and appearance. Both these cues provide complementary information that we fuse using two interconnected recursive estimators: One estimates object segmentation from motion as a probabilistic clustering of tracked 3D points, and the other estimates object segmentation from appearance as a probabilistic image segmentation. The interconnected estimators provide a probabilistic and consistent object segmentation in real time, which makes them well suited for many downstream robotic tasks. We evaluate our method on one such task, kinematic structure estimation, on a dataset of interactions with articulated objects and show that our fusion improves object segmentation by 70% and in turn estimated kinematic joints by 26% over a purely motion-based approach. Furthermore, we show the necessity of probabilistic modeling for downstream robotic tasks, achieving 339% of the performance of a recent multimodal but deterministic RNN for object segmentation on the estimation of kinematic structure.

Dubins coverage has been extensively researched to address the coverage path planning (CPP) problem of a known environment for the curvature-constrained robot. However, its fixed-speed assumption prevents the robot from accelerating to reduce the time and limits its flexibility to avoid obstacles. Therefore, this paper presents a collision-free CPP approach (CFC) for the obstacle-constrained environment, which enhances time efficiency by constructing the variable-speed Dubins paths and ensures robot safety by building a risk potential surface for representing the possibility of collision. Furthermore, CFC models the CPP problem as an asymmetric traveling salesman problem (ATSP) and utilizes a graph pruning strategy to reduce the computational cost. Comparison tests with other Dubins coverage methods demonstrate that CFC provides shorter coverage times and better runtimes than the other Dubins coverage methods while preventing collision risk between the robot and obstacles. Physical experiments in a laboratory setting demonstrate the applicability of CFC to the physical robot.

In this paper, we address the problem of estimating the in-hand 6D pose of an object in contact with multiple vision-based tactile sensors. We reason on the possible spatial configurations of the sensors along the object surface. Specifically, we filter contact hypotheses using geometric reasoning and a Convolutional Neural Network (CNN), trained on simulated object-agnostic images, to promote those that better comply with the actual tactile images from the sensors. We use the selected sensors configurations to optimize over the space of 6D poses using a Gradient Descent-based approach. We finally rank the obtained poses by penalizing those that are in collision with the sensors. We carry out experiments in simulation using the DIGIT vision-based sensor with several objects, from the standard YCB model set. The results demonstrate that our approach estimates object poses that are compatible with actual object-sensor contacts in 87.5% of cases while reaching an average positional error in the order of 2 centimeters. Our analysis also includes qualitative results of experiments with a real DIGIT sensor.

In physical human-robot interaction (pHRI) it is essential to reliably estimate and localize contact forces between the robot and the environment. In this paper, a complete contact detection, isolation, and reaction scheme is presented and tested on a new 6-dof industrial collaborative robot. We combine two popular methods, based on monitoring energy and generalized momentum, to detect and isolate collisions on the whole robot body in a more robust way. The experimental results show the effectiveness of our implementation on the LARA 5 cobot, that only relies on motor current and joint encoder measurements. For validation purposes, contact forces are also measured using an external GTE CoboSafe sensor. After a successful collision detection, the contact point location is isolated using a combination of the residual method based on the generalized momentum with a contact particle filter (CPF) scheme. We show for the first time a successful implementation of such combination on a real robot, without relying on joint torque sensor measurements.

Collaborative scheduling is an essential ability for a team of heterogeneous robots to collaboratively complete complex tasks, e.g., in a multi-robot assembly application. To enable collaborative scheduling, two key problems should be addressed, including allocating tasks to heterogeneous robots and adapting to robot failures in order to guarantee the completion of all tasks. In this paper, we introduce a novel approach that integrates deep bipartite graph matching and imitation learning for heterogeneous robots to complete complex tasks as a team. Specifically, we use a graph attention network to represent attributes and relationships of the tasks. Then, we formulate collaborative scheduling with failure adaptation as a new deep learning-based bipartite graph matching problem, which learns a policy by imitation to determine task scheduling based on the reward of potential task schedules. During normal execution, our approach generates robot-task pairs as potential allocations. When a robot fails, our approach identifies not only individual robots but also subteams to replace the failed robot. We conduct extensive experiments to evaluate our approach in the scenarios of collaborative scheduling with robot failures. Experimental results show that our approach achieves promising, generalizable and scalable results on collaborative scheduling with robot failure adaptation.

The diagnostic value of biopsies is highly dependent on the placement of needles. Robotic trajectory guidance has been shown to improve needle positioning, but feedback for real-time navigation is limited. Haptic display of needle tip forces can provide rich feedback for needle navigation by enabling localization of tissue structures along the insertion path. We present a collaborative robotic biopsy system that combines trajectory guidance with kinesthetic feedback to assist the physician in needle placement. The robot aligns the needle while the insertion is performed in collaboration with a medical expert who controls the needle position on site. We present a needle design that senses forces at the needle tip based on optical coherence tomography and machine learning for real-time data processing. Our robotic setup allows operators to sense deep tissue interfaces independent of frictional forces to improve needle placement relative to a desired target structure. We first evaluate needle tip force sensing in ex-vivo tissue in a phantom study. We characterize the tip forces during insertions with constant velocity and demonstrate the ability to detect tissue interfaces in a collaborative user study. Participants are able to detect 91 percent of ex-vivo tissue interfaces based on needle tip force feedback alone. Finally, we demonstrate that even smaller, deep target structures can be accurately sampled by performing post-mortem in situ biopsies of the pancreas.

This paper presents a collaborative control method based on payload-leading for the multi-quadrotor transportation systems. The goal is to keep the relative distance between the quadrotors and the payload as constant as possible during the transportation, so as to ensure the stable attitude of the payload. The control mechanism consists of a guidance control law that generates the common desired velocity for the quadrotors, an internal feedback controller for each quadrotor, and a decentralized formation controller. The stability of the control structure is proved by Lyapunov theory. Finally, the experimental platform of the multi-quadrotor transportation system is built to verify the effectiveness of the control method. Experimental results show that the proposed method has an excellent control effect.

Communication latency in any delicate telerobotic operation (such as remote surgery over distance) would impose a significant challenge due to the temporal degradation of visual perception and can substantially affect the outcomes. Less is known, however, about the neurophysiological basis of how operators adapt/react to delayed visual feedback. Identification of such neural markers might provide novel ways for future applications to monitor the mental workload (MW). In this study, we recorded electroencephalography (EEG) data from nine users while performing a peg transfer task using the da Vinci Research Kit with three levels of induced visual delay in the video feedback. Our results suggest that spectral EEG-based features can provide markers of the operator's MW modulated by arbitrary visual delay. We also show that the exposure to different visual delays could be successfully classified/detected solely from EEG data, using a Riemannian geometry-based classifier, which highlights the utility of EEG signals for detecting the effect of visual delay on brain activity.