IROS 2020 论文列表

The increasing interest in autonomous robots with a high number of degrees of freedom for industrial applications and service robotics have also increased the demand for efficient control algorithms. The unstructured environment these robots operate in often impose constraints on the joint motion, an important type being the joint limits of the robot itself. These circumstances demand control algorithms to handle multiple tasks as well as constraints efficiently. This paper shows that both kinematic and torque control of redundant robots under hard joint constraints can be formulated in a single framework as a constrained optimization problem. To solve said problem, a generalization of the Fast-SNS algorithm to weighted pseudoinverses is proposed, which fulfills our demand of efficiently and reliably handling joint constraints.

Using robotic manipulators to remotely perform real-world complex contact tasks is challenging whether tasks are known (due to uncertainty) or unknown a priori (lack of motion waypoints, force profiles, etc.). For known tasks we can integrate and utilize Affordance Templates with a selective compliance jogger to remotely perform high dimensional velocity/force tasks - such as turning valves, opening doors, etc. Affordance Templates (ATs) contain virtual visual representations of task-relevant objects and waypoints for interacting with visualized objects. Operators and/or developers align pre-defined ATs with real-world objects to complete complex tasks, potentially reducing the operator's input dimension to a single initiation command. In this work, we integrate a compliant controller with existing ATs to reduce the operator's burden by 1) reducing the dimension of commanded inputs, 2) internally managing contact forces even for complex tasks, and 3) providing situational awareness in the task frame. Since not all tasks can be modeled for general teleoperation, we also introduce Affordance Primitives which reduce the command dimensionality of complex spatial tasks to as low as 1-dimensional input gestures as demonstrated for this effort. To enable reduction of the command input's dimension, the same compliant jogger used to robustly handle uncertainty with ATs is used with Affordance Primitives to autonomously maintain force constraints associated with complex contact tasks. Both Affordance Templates and Affordance Primitives - when used in tandem with a compliant jogger - provide a safe, intuitive, and efficient teleoperation system for general use including using primitives to easily develop new Affordance Templates from newly completed teleoperation tasks.

We present a method to reconstruct the 3D trajectory of an airborne robotic system only from videos recorded with cameras that are unsynchronized, may feature rolling shutter distortion, and whose viewpoints are unknown. Our approach enables robust and accurate outside-in tracking of dynamically flying targets, with cheap and easy-to-deploy equipment. We show that, in spite of the weakly constrained setting, recent developments in computer vision make it possible to reconstruct trajectories in 3D from unsynchronized, uncalibrated networks of consumer cameras, and validate the proposed method in a realistic field experiment. We make our code available along with the data, including cm-accurate groundtruth from differential GNSS navigation.

Tuning the stiffness of soft robots is essential in order to extend usability and control the maneuverability of soft robots. In this paper, we propose a novel mechanism that can reconfigure the stiffness of tubular structures, using pinching to induce highly directional changes in stiffness. When pinched, these tubes can be then utilized as flexure hinges to create virtual joints on demand; the orientation of the hinge axis can additionally be selected via control of the distribution of pinch forces on the surface of the tube. Through proper material and geometry selection, passive shape recovery is observed when pinching forces are removed; a proposed active shape recovery technique can further assist the tube to recover its initial shape in order to re-configure the hinge in a new orientation. The proposed mechanism has been validated in FEA as well as experimentally, looking specifically at the relation between pinching force and curvature change, as well as comparing tube stiffness between pinched and unpinched configurations. The experimental prototype detailed in this paper - and demonstrated in the associated video - is capable of controlling the generation and recovery of flexure hinges at multiple orientations around the radial axis of tubes on demand.

This paper describes the development of a system for lifelike gaze in human-robot interactions using a humanoid Audio-Animatronics® bust. Previous work examining mutual gaze between robots and humans has focused on technical implementation. We present a general architecture that seeks not only to create gaze interactions from a technological standpoint, but also through the lens of character animation where the fidelity and believability of motion is paramount; that is, we seek to create an interaction which demonstrates the illusion of life. A complete system is described that perceives persons in the environment, identifies persons-of-interest based on salient actions, selects an appropriate gaze behavior, and executes high fidelity motions to respond to the stimuli. We use mechanisms that mimic motor and attention behaviors analogous to those observed in biological systems including attention habituation, saccades, and differences in motion bandwidth for actuators. Additionally, a subsumption architecture allows layering of simple motor movements to create increasingly complex behaviors which are able to interactively and realistically react to salient stimuli in the environment through subsuming lower levels of behavior. The result of this system is an interactive human-robot experience capable of human-like gaze behaviors.

In this paper, we present a real-time non-linear model-predictive control (NMPC) framework to perform minimum-time motion planning for autonomous racing cars. We introduce an innovative kineto-dynamical vehicle model, able to accurately predict non-linear longitudinal and lateral vehicle dynamics. The main parameters of this vehicle model can be tuned with only experimental or simulated maneuvers, aimed to identify the handling diagram and the maximum performance G-G envelope. The kineto-dynamical model is adopted to generate on-line minimum time trajectories with an indirect optimal control method. The motion planning framework is applied to control an autonomous 1:8 RC vehicle near the limits of handling along a test circuit. Finally, the effectiveness of the proposed algorithms is illustrated by comparing the experimental results with the solution of an off-line minimum-time optimal control problem.

Real-time 3D perception of the environment is crucial for the adoption and deployment of reliable autonomous harvesting robots in agriculture. Using data collected with RGB-D cameras under farm field conditions, we present two methods for processing 3D data that reliably detect mature broccoli heads. The proposed systems are efficient and enable real-time detection on depth data of broccoli crops using the organised structure of the point clouds delivered by a depth sensor. The systems are tested with datasets of two broccoli varieties collected in planted fields from two different countries. Our evaluation shows the new methods outperform state-of-the-art approaches for broccoli detection based on both 2D vision-based segmentation techniques and depth clustering using the Euclidean proximity of neighbouring points. The results show the systems are capable of accurately detecting the 3D locations of broccoli heads relative to the vehicle at high frame rates.

With the development of robotic technology, the demand for state-of-the-art technology in the field of rehabilitation is rapidly increasing for the elderly and people with disabilities. In this paper, we propose a real-time virtual coach to assist physical therapists with the end-effector-based robot-assisted gait training for stroke survivors using Long Short-Term Memory (LSTM) networks. Our proposed virtual coach consists of the sensor module for data gathering and dataset generation, real-time classification of the pathologic patient gait during the training using LSTM networks, and delivery of the coaching recommendations in an audiovisual form. Our preliminary study determined the selection of coaching recommendations. LSTM networks are trained to provide the selected coaching recommendations. The performance of the proposed virtual coach is verified using classification simulation of an able-bodied person on the rehabilitation robot, G-EO System. The usability was verified through a satisfaction survey of five professional physical therapists.

Distracted driving is one of the main contributors to traffic accidents. This paper proposes a deep learning approach to detecting multiple distracted driving behaviors. In order to obtain more accurate detection results, a synchronized image recognition system based on two cameras is designed, by which the body movements and face of the driver are monitored respectively. The images captured from driver's body and face areas are fed to two Convolutional Neural Networks (CNNs) simultaneously to ensure the performance of classification. The data collection and validation processes of the proposed distraction detection approach were conducted on a laboratory-based assisted driving testbed to provide near-realistic driving experiences. Our dataset includes distracted and safe driving images of the drivers. Furthermore, we developed a meaningful and practical application of a voice-alert system that alerts the distracted driver to focus on the driving task. We evaluated VGG-16, ResNet, and MobileNet-v2 networks for the proposed approach. Experimental results show that by using two cameras and VGG-16 networks, we can achieve a recognition accuracy of 96.7% with a computation speed of 8 fps.

The intuitive collaboration of humans and intelligent robots (embodied AI) in the real-world is an essential objective for many desirable applications of robotics. Whilst there is much research regarding explicit communication, we focus on how humans and robots interact implicitly, on motor adaptation level. We present a real-world setup of a human-robot collaborative maze game, designed to be non-trivial and only solvable through collaboration, by limiting the actions to rotations of two orthogonal axes, and assigning each axes to one player. This results in neither the human nor the agent being able to solve the game on their own. We use deep reinforcement learning for the control of the robotic agent, and achieve results within 30 minutes of real-world play, without any type of pre-training. We then use this setup to perform systematic experiments on human/agent behaviour and adaptation when co-learning a policy for the collaborative game. We present results on how co-policy learning occurs over time between the human and the robotic agent resulting in each participant's agent serving as a representation of how they would play the game. This allows us to relate a person's success when playing with different agents than their own, by comparing the policy of the agent with that of their own agent.

Compressing massive LiDAR point clouds in real-time is critical to autonomous machines such as drones and self-driving cars. While most of the recent prior work has focused on compressing individual point cloud frames, this paper proposes a novel system that effectively compresses a sequence of point clouds. The idea to exploit both the spatial and temporal redundancies in a sequence of point cloud frames. We first identify a key frame in a point cloud sequence and spatially encode the key frame by iterative plane fitting. We then exploit the fact that consecutive point clouds have large overlaps in the physical space, and thus spatially encoded data can be (re-)used to encode the temporal stream. Temporal encoding by reusing spatial encoding data not only improves the compression rate, but also avoids redundant computations, which significantly improves the compression speed. Experiments show that our compression system achieves 40× to 90× compression rate, significantly higher than the MPEG's LiDAR point cloud compression standard, while retaining high end-to-end application accuracies. Meanwhile, our compression system has a compression speed that matches the point cloud generation rate by today LiDARs and out-performs existing compression systems, enabling real-time point cloud transmission.

In this paper, we propose a novel algorithm that estimates the pose of the robot end effector using depth vision. The input to our system is the segmented robot hand point cloud from a depth sensor. Then a neural network takes a point cloud as input and outputs the position and orientation of the robot end effector in the camera frame. The estimated pose can serve as the input of the controller of the robot to reach a specific pose in the camera frame. The training process of the neural network takes the simulated rendered point cloud generated from different poses of the robot hand mesh. At test time, one estimation of a single robot hand pose is reduced to 10ms on gpu and 14ms on cpu, which makes it suitable for close loop robot control system that requires to estimate hand pose in an online fashion. We design a robot hand pose estimation experiment to validate the effectiveness of our algorithm working in the real situation. The platform we used includes a Kinova Jaco 2 robot arm and a Kinect v2 depth sensor. We describe all the processes that use vision to improve the accuracy of pose estimation of the robot end-effector. We demonstrate the possibility of using point cloud to directly estimate the robot's end-effector pose and incorporate the estimated pose into the controller design of the robot arm.

This paper introduces a Wearable SLAM system that performs indoor and outdoor SLAM in real time. The related project is part of the MALIN challenge which aims at creating a system to track emergency response agents in complex scenarios (such as dark environments, smoked rooms, repetitive patterns, building floor transitions and doorway crossing problems), where GPS technology is insufficient or inoperative. The proposed system fuses different SLAM technologies to compensate the lack of robustness of each, while estimating the pose individually. LiDAR and visual SLAM are fused with an inertial sensor in such a way that the system is able to maintain GPS coordinates that are sent via radio to a ground station, for real-time tracking. More specifically, LiDAR and monocular vision technologies are tested in dynamic scenarios where the main advantages of each have been evaluated and compared. Finally, 3D reconstruction up to three levels of details is performed.

This paper presents a constrained nonlinear model predictive control (NMPC) framework for legged locomotion. The framework assumes a legged robot as a floating base single rigid body with contact forces being applied to the body as external forces. With consideration of orientation dynamics evolving on the rotation manifold SO(3), analytic Jacobians which are necessary for constructing the gradient and the Gauss-Newton Hessian approximation of the objective function are derived. This procedure also includes the reparameterization of the robot orientation on SO(3) to orientation error in the tangent space of that manifold. Obtained gradient and Gauss-Newton Hessian approximation are utilized to solve nonlinear least squares problems formulated from NMPC in a computationally efficient manner. The proposed algorithm is verified on various types of legged robots and gaits in a simulation environment.

The paper presents a receding horizon planning strategy for a quadrotor-type mav to navigate through an unknown cluttered environment at high speed. Utilizing a lightweight on-board short-range sensor that generates point-clouds within a narrow Field of View (FOV), the reported approach generates safe and dynamically feasible trajectories within the fov of the sensor, which the mav uses to navigate without relying on any global planner or prior information about the environment. The effectiveness of this planner-controller combination is demonstrated in both indoor and outdoor tests featuring speeds of up to of 3.5 m/s. With minor adjustments, the local motion planner can be utilized for interception and tracking of a moving target; evidence to this effect are provided in the form of numerical (Gazebo) simulations. Given the absence of any global information about the robot's workspace, the extent to which the local planner can provide convergence guarantees is limited; when complemented by a global planner and/or target tracker, the reported lower-level, sensor-driven reactive motion control strategy completes the autonomous mav navigation stack, enabling navigation in dynamic, uncertain, and partially-known environments with guaranteed convergence to any static or dynamic target.

Learning-enabled components have been widely deployed in autonomous systems. However, due to the weak interpretability and the prohibitively high complexity of large-scale machine learning models such as neural networks, reliability has been a crucial concern for safety-critical autonomous systems. This work proposes an online monitor called Reach-Flow for fault prevention of waypoint-following tasks for self-driving cars. It mainly consists of two components: (a) an online verification tool which conservatively checks the safety of the system behavior in the near future, and (b) a fallback controller which steers the system back to a desired state when the system is potentially unsafe. We implement ReachFlow in a self-driving racing car governed by a reinforcement learning-based controller. We demonstrate the effectiveness by rigorously verifying a safe waypoint-following control and providing a fallback control for an unsafe situation in which a large deviation from the planned path is predicted.

Learning adaptable policies is crucial for robots to operate autonomously in our complex and quickly changing world. In this work, we present a new meta-learning method that allows robots to quickly adapt to changes in dynamics. In contrast to gradient-based meta-learning algorithms that rely on second-order gradient estimation, we introduce a more noise-tolerant Batch Hill-Climbing adaptation operator and combine it with meta-learning based on evolutionary strategies. Our method significantly improves adaptation to changes in dynamics in high noise settings, which are common in robotics applications. We validate our approach on a quadruped robot that learns to walk while subject to changes in dynamics. We observe that our method significantly outperforms prior gradient-based approaches, enabling the robot to adapt its policy to changes based on less than 3 minutes of real data.

This paper shows how CasADi’s state-of-the-art implementation of algorithmic differentiation can be leveraged to formulate and efficiently solve gait optimization problems, enabling rapid gait design for high-dimensional biped robots. Comparative studies on a 7-DOF planar biped show that CasADi generates optimal gaits 4 times faster than another existing advanced optimization package. The framework is also applied to simultaneously generate a gait and a feedback controller for 2 spatial bipeds: a 12-DOF model and a 20DOF model. Results suggest that CasADi’s unprecedented efficiency could provide a practical path toward real-time gait optimization for high-dimensional biped robots.

A semantic understanding of the environment is needed to enable high level autonomy in robotic systems. Recent results have demonstrated rapid progress in underlying technology areas, but few results have been reported on end-to-end systems that enable effective autonomous perception in complex environments. In this paper, we describe an approach for rapidly and autonomously mapping unknown environments with integrated semantic and geometric information. We use surfel-based RGB-D SLAM techniques, with incremental object segmentation and classification methods to update the map in realtime. Information theoretic and heuristic measures are used to quickly plan sensor motion and drive down map uncertainty. Preliminary experimental results in simple and cluttered environments are reported.