IROS 2023 论文列表

In this study, we introduce an online monocular lane mapping approach that solely relies on a single camera and odometry for generating spline-based maps. Our proposed technique models the lane association process as an assignment issue utilizing a bipartite graph, and assigns weights to the edges by incorporating Chamfer distance, pose uncertainty, and lateral sequence consistency. Furthermore, we meticulously design control point initialization, spline parameterization, and optimization to progressively create, expand, and refine splines. In contrast to prior research that assessed performance using self-constructed datasets, our experiments are conducted on the openly accessible OpenLane dataset. The experimental outcomes reveal that our suggested approach enhances lane association and odometry precision, as well as overall lane map quality. We have open-sourced out code11https://github.com/HKUST-Aerial-Robotics/MonoLaneMapping for this project.

Service robots are expected to assist users in a constantly growing range of environments and tasks. People may be unique in many ways, and online adaptation of robots is central to personalized assistance. We focus on collaborative tasks in which the human collaborator may not be fully ablebodied, with the aim for the robot to automatically determine the best level of support. We propose a methodology for online adaptation based on Reinforcement Learning and Bayesian inference. As the Reinforcement Learning process continuously adjusts the robot's behavior, the actions that become part of the improved policy are used by the Bayesian inference module as local evidence of human capability, which can be generalized across the state space. The estimated capabilities are then used as pre-conditions to collaborative actions, so that the robot can quickly disable actions that the person seems unable to perform. We demonstrate and validate our approach on two simulated tasks and one real-world collaborative task across a range of motion and sensing capabilities.

This paper proposes an online estimation model for 2D arm stiffness in humans. The proposed model is based on recent physiological findings which suggest that: (1) joint stiffness is linearly related to the magnitude of joint torque and increases to compensate for environmental disturbances; and (2) the endpoint stiffness of the arm is proportional to grasp force. To validate the proposed model, perturbation experiments were conducted under different grasp forces. The model parameters were identified and the accuracy of the model was assessed. The results showed that the proposed model has advantages over previous models for estimating human arm endpoint stiffness, in the sense of simplicity and robustness. The proposed model was also used to design a variable stiffness controller for peg-in-hole tasks, demonstrating the potential of the model for human-robot collaboration.

Vision systems mounted on home robots need to interact with unseen classes in changing environments. Robots have limited computational resources, labelled data and storage capability. These requirements pose some unique challenges: models should adapt without forgetting past knowledge in a data- and parameter-efficient way. We characterize the problem as few-shot (FS) online continual learning (OCL), where robotic agents learn from a non-repeated stream of few-shot data updating only a few model parameters. Additionally, such models experience variable conditions at test time, where objects may appear in different poses (e.g., horizontal or vertical) and environments (e.g., day or night). To improve robustness of CL agents, we propose RobOCLe, which; 1) constructs an enriched feature space computing high order statistical moments from the embedded features of samples; and 2) computes similarity between high order statistics of the samples on the enriched feature space, and predicts their class labels. We evaluate robustness of CL models to train/test augmentations in various cases. We show that different moments allow RobOCLe to capture different properties of deformations, providing higher robustness with no decrease of inference speed.

In recent years, deep neural networks have shown remarkable progress in dense disparity estimation from dynamic scenes in monocular structured light systems. However, their performance significantly drops when applied in unseen environments. To address this issue, self-supervised online adaptation has been proposed as a solution to bridge this performance gap. Unlike traditional fine-tuning processes, online adaptation performs test-time optimization to adapt networks to new domains. Therefore, achieving fast convergence during the adaptation process is critical for attaining satisfactory accuracy. In this paper, we propose an unsupervised loss function based on long sequential inputs. It ensures better gradient directions and faster convergence. Our loss function is designed using a multi-frame pattern flow, which comprises a set of sparse trajectories of the projected pattern along the sequence. We estimate the sparse pseudo ground truth with a confidence mask using a filter-based method, which guides the online adaptation process. Our proposed framework significantly improves the online adaptation speed and achieves superior performance on unseen data. The code is available on https://github.com/CodePointer/TIDENet.

We present One-Shot Affordance Learning (OSAL): a unified pipeline that learns manipulation for articulated objects by observing human demonstration only once. The key idea of our method is to embody affordance of articulated objects with an open-loop trajectory conditioned on a certain area of the object's surface. It serves as a simplified object-centric manipulation representation, which can be easily transferred into robot motion, while traditional methods fail to deal with the configuration difference between human hands and robot end effectors. Our system extracts the embodied affordance by focusing on hand action's effect on the object, and further grounds such affordance into object visual features through self-supervised learning for novel object configurations. We evaluated our method on a collection of real-life objects and furniture and demonstrated high success rates. With our system, humans only need to manipulate a novel object once with any gesture to transfer that manipulation skill to the robot, which we believe to be a highly efficient and user-friendly paradigm oriented for future real-life robots.

A fundamental task in robotics is to navigate between two locations. In particular, real-world navigation can require long-horizon planning using high-dimensional RGB images, which poses a substantial challenge for end-to-end learning-based approaches. Current semi-parametric methods instead achieve long-horizon navigation by combining learned modules with a topological memory of the environment, often represented as a graph over previously collected images. However, using these graphs in practice requires tuning a number of pruning heuristics. These heuristics are necessary to avoid spurious edges, limit runtime memory usage and maintain reasonably fast graph queries in large environments. In this work, we present One-4-All (O4A), a method leveraging self-supervised and manifold learning to obtain a graph-free, end-to-end navigation pipeline in which the goal is specified as an image. Navigation is achieved by greedily minimizing a potential function defined continuously over image embeddings. Our system is trained offline on non-expert exploration sequences of RGB data and controls, and does not require any depth or pose measurements. We show that 04A can reach long-range goals in 8 simulated Gibson indoor environments and that resulting embeddings are topologically similar to ground truth maps, even if no pose is observed. We further demonstrate successful real-world navigation using a Jackal UGV platform.aaProject page https://montrealrobotics.ca/o4a/.

Achieving fluent flocking, similar to those observed in birds and fish, on robotic swarms in a desired direction while avoiding obstacles using onboard sensing and computation remains a challenge. In a previous study (Önür et al, Proc. of ANTS'2022), we proposed a predictive flocking model as a computationally efficient method to generate smoother and more robust motion of the swarm. In this study, we extend this model to achieve safe flocking in cluttered environments in the presence of faulty robots that get immobilized during flocking. Systematical evaluation of the model in simulation with different swarm sizes and different faulty robot ratios has shown that safe flocking can be achieved even when 40% of the robots malfunction during flocking. Finally, we validate the model on a swarm of five micro quadcopters using only onboard range and bearing sensors and computation in a distributed manner without any communication11Videos of the experiments are available here: https://www.youtube.com/playlist?list=PLY04vZs6xGr8U5Y8KWMD056ZRjaUVMh63..

Robot learning is often difficult due to the expense of gathering data. The need for large amounts of data can, and should, be tackled with effective algorithms and leveraging expert information on robot dynamics. Bayesian reinforcement learning (BRL), thanks to its sample efficiency and ability to exploit prior knowledge, is uniquely positioned as such a solution method. Unfortunately, the application of BRL has been limited due to the difficulties of representing expert knowledge as well as solving the subsequent inference problem. This paper advances BRL for robotics by proposing a specialized framework for physical systems. In particular, we capture this knowledge in a factored representation, then demonstrate the posterior factorizes in a similar shape, and ultimately formalize the model in a Bayesian framework. We then introduce a sample-based online solution method, based on Monte-Carlo tree search and particle filtering, specialized to solve the resulting model. This approach can, for example, utilize typical low-level robot simulators and handle uncertainty over unknown dynamics of the environment. We empirically demonstrate its efficiency by performing on-robot learning in two human-robot interaction tasks with uncertainty about human behavior, achieving near-optimal performance after only a handful of real-world episodes. A video of learned policies is at https://youtu.be/H9xp60ngOes.

This study proposes a novel diagnosis framework to decrease the early detection miss rate of colorectal cancer (CRC) polyps by using a hypersensitive vision-based tactile sensor (HySenSe) and a deep residual neural network. The HySenSe generates high-resolution 3D textural images of 160 realistic polyp phantoms for accurate classification via the proposed deep learning (DL) architecture. The DL module explores lightweight dilated convolutions, residual neural network architecture, and transfer learning to overcome the challenge of a small dataset of 229 images. Results show that the proposed architecture outperforms state-of-the-art DL models (i.e., EfficientNet and DenseNet) with a 94% accuracy, offering a promising solution for improving early detection of CRC polyps. The proposed framework can be used as a diagnostic module within tele-assessment medical robots, highlighting the potential of advanced technology and deep learning to revolutionize the early detection and treatment of CRC.

The generation of energy-efficient and dynamic-aware robot motions that satisfy constraints such as joint limits, self-collisions, and collisions with the environment remains a challenge. In this context, Riemannian geometry offers promising solutions by identifying robot motions with geodesics on the so-called configuration space manifold. While this manifold naturally considers the intrinsic robot dynamics, constraints such as joint limits, self-collisions, and collisions with the environment remain overlooked. In this paper, we propose a modification of the Riemannian metric of the configuration space manifold allowing for the generation of robot motions as geodesics that efficiently avoid given regions. We introduce a class of Riemannian metrics based on barrier functions that guarantee strict region avoidance by systematically generating accelerations away from no-go regions in joint and task space. We evaluate the proposed Riemannian metric to generate energy-efficient, dynamic-aware, and collision-free motions of a humanoid robot as geodesics and sequences thereof.

Telemanipulation is widely used in robotics applications, ranging from maintenance of various industrial systems to search and rescue response in remote and/or hazardous environments. Human operators are often responsible for the control of such robotic systems. However, these remote interactions require highly trained and experienced operators owing to their complex nature. Semi-autonomous systems are presented as an alternative to complex and counter-intuitive manual systems, combining decoded user intentions with autonomous control modules. This paper proposes a semi-autonomous framework for robotic telemanipulation that employs Electromyography (EMG) based motion decoding and potential fields to execute complex object stacking tasks with a dexterous robot arm-hand system. Even though simple EMG-based teleoperation is promising, the signals are often noisy leading to induced randomness and control errors. To assist the user during task executions, potential fields are utilized to avoid obstacles and guide the robot end-effector toward the objects of interest, thus reducing the cognitive load on the user and the need for accurate predictions. The user's motion is decoded from the myoelectric activations of the human upper arm and upper torso using a Random Forest-based regression methodology. The objects are detected in the environment with an external camera that provides their goal poses to the potential fields scheme. EMG control and potential fields work in a synergistic manner simplifying the system's operation. The framework performance is experimentally validated in real-time experiments involving complex cube and cylinder stacking tasks.

Substantial research and development on the design and control of robotic ankle-foot prostheses have aimed to restore normal function and movement capacity for people with gait impairments and lower limb amputations. However, prostheses controllers usually fail to incorporate information pertaining to the properties of the walking terrain, such as ground stiffness. There is therefore a need for a framework that adjusts the prostheses parameters according to the user's intent to transition to a variable impedance terrain. To achieve this, we need to incorporate the human wearer in the control loop of the prosthesis. This work proposes an advanced, high-level controller framework for powered ankle-foot prostheses that combines subject-specific pattern recognition (PR) and classification strategies to predict whether the next step will be on a rigid or compliant surface. Comparing the Support Vector Machine (SVM) and k-Nearest Neighbors (k-NN) classification algorithms for this task, we conclude that by combining a k-NN implementation with a Pattern Recognition Neural Network (PR NN), our method can accurately forecast upcoming surface stiffness transitions in time to allow for prompt adaptation to the new walking terrain. We also show that the sensor fusion of kinematic and surface electromyographic (EMG) data outperforms single-source inputs producing the best prediction results for all subjects with an accuracy of up to 87.5%.

As robots become more prevalent, optimizing their design for better performance and efficiency is becoming increasingly important. However, current robot design practices overlook the impact of perception and design choices on a robot's learning capabilities. To address this gap, we propose a comprehensive methodology that accounts for the interplay between the robot's perception, hardware characteristics, and task requirements. Our approach optimizes the robot's morphology holistically, leading to improved learning and task execution proficiency. To achieve this, we introduce a Morphology-AGnostIc Controller (MAGIC), which helps with the rapid assessment of different robot designs. The MAGIC policy is efficiently trained through a novel PRIvileged Single-stage learning via latent alignMent (PRISM) framework, which also encourages behaviors that are typical of robot onboard observation. Our simulation-based results demonstrate that morphologies optimized holistically improve the robot performance by 15-20% on various manipulation tasks, and require 25x less data to match human-expert made morphology performance. In summary, our work contributes to the growing trend of learning-based approaches in robotics and emphasizes the potential in designing robots that facilitate better learning. The project's website can be found at learning-robot.github.io

In this paper, an immune average consensus behavior of distributed trajectory generators given in the form of a multi-agent system is presented. Starting with the well-known results of linear consensus protocols, we propose a decomposition of the invariant consensus value to enable a distributed cyber-attacks detection and mitigation mechanism among the connected agents over mainly undirected communication links. This decomposition suggests one preferred propagation of the invariant quantity along communication links of the multi-agent systems under study. Despite its simplicity, the effectiveness of this mechanism in detecting and mitigating various types of cyber-attacks is evident through a numerical simulation. Interestingly, the resulting defense mechanism will not be passive, rather it can initiate its counter-attack measures by pretending that the attack process was a success. Moreover, the trajectory generators can operate under stealth mode where the communication links get silenced or totally disconnected without affecting the intended behavior after having the consensus value locked.

With the rapidly changing climate and an increase in extreme weather events, it is necessary to have better methods to monitor and study the impacts of these phenomena on urban river environments. Multi-robot environmental monitoring has long focused on strategies that assign individual robots to distinct regions or task objectives. While these methods have seen success for Autonomous Surface Vehicles (ASVs), the spatial expanse and temporal variability of rivers impose an increased burden on existing techniques, necessitating computationally intensive replanning. Alternative methods aim to model and control teams of robots by prescribing global constraints on the system, using the insight that robots' transitions between tasks are stochastic and time-based. These methods do not require replanning because robots will perform different tasks achieving the overall desired system state, focusing on temporal switching alone limits their overall descriptive power. In this paper, we present a method that considers collaborations between robots to inform task switching based on spatial proximity. Our results suggest that in unknown environments macroscopic models provide increased flexibility for individual robot task execution as compared to coverage control methods.

Applying Reinforcement Learning to solve real-world optimization problems presents significant challenges because of the large amount of data normally required. A popular solution is to train the algorithms in a simulation and transfer the weights to the real system. However, sim-to-real approaches are prone to fail when the Reality Gap is too big, e.g. in robotic systems with complex and non-linear dynamics. In this work, we propose the use of Offline Reinforcement Learning as a viable alternative to sim-to-real policy transfer to address such instances. On the example of a small quadrotor, we show that the ground effect causes problems in an otherwise functioning zero-shot sim-to-real framework. Our sim-to-real experiments show that, even with the explicit modelling of the ground effect and the employing of popular transfer techniques, the trained policies fail to capture the physical nuances necessary to perform a real-world take-off maneuver. Contrariwise, we show that state-of-the-art Offline Reinforcement Learning algorithms represent a feasible, reliable and sample efficient alternative in this use case.

Localisation with Frequency-Modulated Continuous-Wave (FMCW) radar has gained increasing interest due to its inherent resistance to challenging environments. However, complex artefacts of the radar measurement process require appropriate uncertainty estimation - to ensure the safe and reliable application of this promising sensor modality. In this work, we propose a multi-session map management system which constructs the “best” maps for further localisation based on learned variance properties in an embedding space. Using the same variance properties, we also propose a new way to introspectively reject localisation queries that are likely to be incorrect. For this, we apply robust noise-aware metric learning, which both leverages the short-timescale variability of radar data along a driven path (for data augmentation) and predicts the downstream uncertainty in metric-space-based place recognition. We prove the effectiveness of our method over extensive cross-validated tests of the Oxford Radar RobotCar and MulRan dataset. In this, we outperform the current state-of-the-art in radar place recognition and other uncertainty-aware methods when using only single nearest-neighbour queries. We also show consistent performance increases when rejecting queries based on uncertainty over a difficult test environment, which we did not observe for a competing uncertainty-aware place recognition system.

Domestic service robots are becoming increasingly popular due to their ability to help people with household tasks. These robots often encounter the challenge of manipulating objects in cluttered environments (MoC), which is difficult due to the complexity of effective planning and control. Previous solutions involved designing specific action primitives and planning paradigms. However, the pre-coded action primitives can limit the agility and task-solving scope of robots. In this paper, we propose a general approach for MoC called the Object-Oriented Option Framework (O3F), which uses the option framework (OF) to learn planning and control. The standard OF discovers options from scratch based on reinforcement learning, which can lead to collapsed options and hurt learning. To address this limitation, O3F introduces the concept of an object-oriented option space for OF, which focuses specifically on object movement and overcomes the challenges associated with collapsed options. Based on this, we train an object-oriented option planner to determine the option to execute and a universal object-oriented option executor to complete the option. Simulation experiments on the Ginger XR1 robot and robot arm show that O3F is generally applicable to various types of robot and manipulation tasks. Furthermore, O3F achieves success rates of 72.4% and 90% in grasping and object collecting tasks, respectively, significantly outperforming baseline methods.