IROS 2019 论文列表

We explore the possibility of using a single monocular camera to forecast the time to collision between a suitcase-shaped robot being pushed by its user and other nearby pedestrians. We develop a purely image-based deep learning approach that directly estimates the time to collision without the need of relying on explicit geometric depth estimates or velocity information to predict future collisions. While previous work has focused on detecting immediate collision in the context of navigating Unmanned Aerial Vehicles, the detection was limited to a binary variable (i.e., collision or no collision). We propose a more fine-grained approach to collision forecasting by predicting the exact time to collision in terms of milliseconds, which is more helpful for collision avoidance in the context of dynamic path planning. To evaluate our method, we have collected a novel dataset of over 13,000 indoor video segments each showing a trajectory of at least one person ending in a close proximity (a near collision) with the camera mounted on a mobile suitcase-shaped platform. Using this dataset, we do extensive experimentation on different temporal windows as input using an exhaustive list of state-of-the-art convolutional neural networks (CNNs). Our results show that our proposed multi-stream CNN is the best model for predicting time to near-collision. The average prediction error of our time to near-collision is 0.75 seconds across the test videos. The project webpage can be found at https://aashi7.github.io/NearCollision.html.

The use of hand tools presents a challenge for robot manipulation in part because it calls for motions requiring continuous force application over a whole trajectory, usually involving large joint-angle excursions. The feasible application of a tool, such as pulling a nail with a hammer claw, requires careful coordination of the choice of grasp and joint trajectories to ensure kinematic and force limits are not exceeded - in the grasp as well as the robot mechanism. In this paper, we formulate this type of problem as choosing the values of decision variables in the presence of various constraints. We evaluate the impact of the various constraints in some representative instances of tool use. To aid others in further investigating this class of problems, we have released materials such as printable tool models and experimental data. We hope that these can serve as the basis of a benchmark problem for investigating tasks that involve many kinematic, actuation, friction, and environment constraints.

For a variety of applications remote navigation of an unmanned aerial vehicle (UAV) along a flight trajectory is an essential task. For instance, during search and rescue missions in outdoor scenes, an important goal is to ensure safe navigation. Assessed by the remote operator, this could mean avoiding collisions with obstacles, but moreover avoiding hazardous flight areas. State of the art approaches enable navigation along trajectories, but do not allow for indirect manipulation during motion. In addition, they suggest to use egocentric views which could limit understanding of the remote scene. With this work we introduce a novel indirect manipulation method, based on gravitational law, to recover safe navigation in the presence of hazardous flight areas. The indirect character of our method supports manipulation at far distances where common direct manipulation methods typically fail. We combine it with an immersive exocentric view to improve understanding of the scene. We designed three flavors of our method and compared them during a user study in a simulated scene. While with this method we present a first step towards a more extensive navigation interface, as future work we plan experiments in dynamic real-world scenes.

To realize a robust automobile driving behavior of musculoskeletal tendon-driven humanoids, we developed a six-axis force measurement module with a core-shell structure. This sensor enables space saving, high load capacity and wholebody sensing at the same time. By developing a foot unit incorporating a core-shell structural force sensor on its toe, we realized behaviors of depressing a pedal and recovering from foot slipping during the depressing with a lifesized musculoskeletal humanoid ”Musashi”.

Many robot applications being explored involve robots leading humans during navigation. Developing effective robots for this task requires a way for robots to understand and model a human's following behavior. In this paper, we present results from a user study of how humans follow a guide robot in the halls of an office building. We then present a data-driven Markovian model of this following behavior, and demonstrate its generalizability across time interval and trajectory length. Finally, we integrate the model into a global planner and run a simulation experiment to investigate the benefits of coupled human-robot planning. Our results suggest that the proposed model effectively predicts how humans follow a robot, and that the coupled planner, while taking longer, leads the human significantly closer to the target position.

In this paper, we investigate the problem of enabling a drone to fly through a tilted narrow gap, without a traditional planning and control pipeline. To this end, we propose an end-to-end policy network, which imitates from the traditional pipeline and is fine-tuned using reinforcement learning. Unlike previous works which plan dynamical feasible trajectories using motion primitives and track the generated trajectory by a geometric controller, our proposed method is an end-to-end approach which takes the flight scenario as input and directly outputs thrust-attitude control commands for the quadrotor. Key contributions of our paper are: 1) presenting an imitate-reinforce training framework. 2) flying through a narrow gap using an end-to-end policy network, showing that learning based method can also address the highly dynamic control problem as the traditional pipeline does (see attached video1). 3) propose a robust imitation of an optimal trajectory generator using multilayer perceptrons. 4) show how reinforcement learning can improve the performance of imitation learning, and the potential to achieve higher performance over the model-based method.

Robotic pollinators not only can aid farmers by providing more cost effective and stable methods for pollinating plants but also benefit crop production in environments not suitable for bees such as greenhouses, growth chambers, and in outer space. Robotic pollination requires a high degree of precision and autonomy but few systems have addressed both of these aspects in practice. In this paper, a fully autonomous robot is presented, capable of precise pollination of individual small flowers. Experimental results show that the proposed system is able to achieve a 93.1% detection accuracy and a 76.9% `pollination' success rate tested with high-fidelity artificial flowers.

FlightGoggles is a photorealistic sensor simulator for perception-driven robotic vehicles. The key contributions of FlightGoggles are twofold. First, FlightGoggles provides photorealistic exteroceptive sensor simulation using graphics assets generated with photogrammetry. Second, it provides the ability to combine (i) synthetic exteroceptive measurements generated in silico in real time and (ii) vehicle dynamics and proprioceptive measurements generated in motio by vehicle(s) in flight in a motion-capture facility. FlightGoggles is capable of simulating a virtual-reality environment around autonomous vehicle(s) in flight. While a vehicle is in flight in the Flight-Goggles virtual reality environment, exteroceptive sensors are rendered synthetically in real time while all complex dynamics are generated organically through natural interactions of the vehicle. The FlightGoggles framework allows for researchers to accelerate development by circumventing the need to estimate complex and hard-to-model interactions such as aerodynamics, motor mechanics, battery electrochemistry, and behavior of other agents. The ability to perform vehicle-in-the-loop experiments with photorealistic exteroceptive sensor simulation facilitates novel research directions involving, e.g., fast and agile autonomous flight in obstacle-rich environments, safe human interaction, and flexible sensor selection. FlightGoggles has been utilized as the main test for selecting nine teams that will advance in the AlphaPilot autonomous drone racing challenge. We survey approaches and results from the top AlphaPilot teams, which may be of independent interest. FlightGoggles is distributed as open-source software along with the photorealistic graphics assets for several simulation environments, under the MIT license at http://flightgoggles.mit.edu.

Micro Aerial Vehicles (MAVs) rely on onboard attitude and position sensors for autonomous flight. Due to their size, weight, and power (SWaP) constraints, most modern MAVs use miniaturized inertial measurement units (IMUs) to provide attitude feedback, which is critical for flight stabilization and control. However, recent adversarial attack studies have demonstrated that many commonly used IMUs are vulnerable to attacks exploiting their physical characteristics. Conventional redundancy-based approaches are not effective against such attacks because redundant IMUs have the same or similar physical vulnerabilities. In this paper, we present a novel fault-tolerant solution for IMU compromised scenarios, using separate position and heading information to restore the failed attitude states. Rather than adding more IMU alternatives for recovery, the proposed method is intended to minimize any modifications to the existing system and control program. Thus, it is particularly useful for vehicles that have tight SWaP constraints while requiring simultaneous high performance and safety demands. To execute the recovery logic properly, a robust estimator was designed for fine-grained detection and isolation of the faulty sensors. The effectiveness of the proposed approach was validated on a quadcopter MAV through both simulation and experimental flight tests.

Flexures provide precise motion control without friction or wear. Variable impedance mechanisms enable adaptable and robust interactions with the environment. This paper combines the advantages of both approaches through layer jamming. Thin sheets of complaint material are encased in an airtight envelope, and when connected to a vacuum, the bending stiffness and damping increase dramatically. Using layer jamming structures as flexure elements leads to mechanical systems that can actively vary stiffness and damping. This results in flexure mechanisms with the versatility to transition between degrees of freedom and degrees of constraint and to tune impact response. This approach is used to create a 2-DOF, jamming-based, tunable impedance robotic wrist that enables passive hybrid force/position control for contact tasks. Keywords: Compliant Joint/Mechanism, Compliance and Impedance Control, Mechanism Design.

In this paper, we propose a visual-inertial framework able to efficiently estimate the camera poses of a non-rigid trinocular baseline for long-range depth estimation on-board a fast moving aerial platform. The estimation of the time-varying baseline is based on relative inertial measurements, a photometric relative pose optimizer, and a probabilistic wing model fused in an efficient Extended Kalman Filter (EKF) formulation. The estimated depth measurements can be integrated into a geo-referenced global map to render a reconstruction of the environment useful for local replanning algorithms. Based on extensive real-world experiments we describe the challenges and solutions for obtaining the probabilistic wing model, reliable relative inertial measurements, and vision-based relative pose updates and demonstrate the computational efficiency and robustness of the overall system under challenging conditions.

Fiducials are artificial features with a variety of uses in computer vision such as object tracking and localization. We propose the idea of flexible tag layouts for visual fiducial systems. In contrast to traditional square tags, flexible tag layouts allow circular, annular, or other shapes as desired. One use of layout flexibility is to increase the data density of standard square shaped tags. In addition, we describe a detector that is faster and has higher recall than both the AprilTag 2 and ArUco detectors while maintaining precision.

The hybrid zero dynamics (HZD) approach has become a powerful tool for the gait planning and control of bipedal robots. This paper aims to extend the HZD methods to address walking, ambling and trotting behaviors on a quadrupedal robot. We present a framework that systematically generates a wide range of optimal trajectories and then provably stabilizes them for the full-order, nonlinear and hybrid dynamical models of quadrupedal locomotion. The gait planning is addressed through a scalable nonlinear programming using direct collocation and HZD. The controller synthesis for the exponential stability is then achieved through the Poincaré sections analysis. In particular, we employ an iterative optimization algorithm involving linear and bilinear matrix inequalities (LMIs and BMIs) to design HZD-based controllers that guarantee the exponential stability of the fixed points for the Poincaré return map. The power of the framework is demonstrated through gait generation and HZD-based controller synthesis for an advanced quadruped robot, - Vision 60, with 36 state variables and 12 control inputs. The numerical simulations as well as real world experiments confirm the validity of the proposed framework.

CNNs have excelled at performing place recognition over time, particularly when the neural network is optimized for localization in the current environmental conditions. In this paper we investigate the concept of feature map filtering, where, rather than using all the activations within a convolutional tensor, only the most useful activations are used. Since specific feature maps encode different visual features, the objective is to remove feature maps that detract from the ability to recognize a location across appearance changes. Our key innovation is to filter the feature maps in an early convolutional layer, but then continue to run the network and extract a feature vector using a later, more viewpoint invariant layer in the same network. Our approach improves the condition and viewpoint invariance of a pre-trained network, using as little as a single training image pair from the deployment environment. An exhaustive experimental analysis is performed to determine the full scope of causality between early layer filtering and late layer extraction. For validity, we use three datasets: Oxford RobotCar, Nordland, and Gardens Point, achieving overall superior performance to NetVLAD. The work provides a number of new avenues for exploring CNN optimizations, without requiring any re-training of the network weights.

Accurate manipulation of a deformable body such as a piece of fabric is difficult because of its many degrees of freedom and unobservable properties affecting its dynamics. To alleviate these challenges, we propose the application of feedback-based control to robotic fabric strip folding. The feedback is computed from the low dimensional state extracted from a camera image. We trained the controller using reinforcement learning in simulation which was calibrated to cover the real fabric strip behaviors. The proposed feedback-based folding was experimentally compared to two state-of-the-art folding methods and our method outperformed both of them in terms of accuracy.

The computational power of mobile robots is currently insufficient to achieve torque level whole-body Model Predictive Control (MPC) at the update rates required for complex dynamic systems such as legged robots. This problem is commonly circumvented by using a fast tracking controller to compensate for model errors between updates. In this work, we show that the feedback policy from a Differential Dynamic Programming (DDP) based MPC algorithm is a viable alternative to bridge the gap between the low MPC update rate and the actuation command rate. We propose to augment the DDP approach with a relaxed barrier function to address inequality constraints arising from the friction cone. A frequency-dependent cost function is used to reduce the sensitivity to high-frequency model errors and actuator bandwidth limits. We demonstrate that our approach can find stable locomotion policies for the torque-controlled quadruped, ANYmal, both in simulation and on hardware.

While rehabilitation of upper-limb motor function with human-interactive robots has been met with success, robot-aided locomotor rehabilitation has proven challenging. To inform more effective approaches to robotic gait therapy, it is important to understand neuro-mechanical dynamics and control of unimpaired locomotion. Our previous studies reported that human gait entrained to periodic mechanical perturbations at the ankle when the perturbation period was close to preferred walking cadence. Moreover, entrainment was accompanied by synchronizing the perturbations to a constant gait phase, the same for all subjects, where they provided mechanical assistance. To test the generality of entrainment-based assistance, the present study evaluated the behavior of live unimpaired subjects who walked overground while wearing a hip exoskeleton robot. Periodic torque pulses were applied to the subjects' hips, with a period different from, but close to, their preferred stride cadence. Results indicated that unimpaired subjects entrained their gait to periodic mechanical perturbations at the hip. Convergence of relative phase between gait and perturbations was observed, but clustered around two distinct gait phases, in contrast to the single converged phase observed in entrainment to periodic ankle torques. These entrainment studies quantify important aspects of the nonlinear neuro-mechanical dynamics underlying the control of walking, which will inform the development of effective approaches to robotic walking therapy.

Applying deep reinforcement learning to physical systems, as opposed to learning in simulation, presents additional challenges in terms of sample efficiency and safety. Collecting large amounts of hardware demonstration data is time-consuming and the exploratory behavior of reinforcement learning algorithms may lead the system into dangerous states, especially during the early stages of training. To address these challenges, we apply transfer learning to reuse a previously learned policy instead of learning from scratch. In this paper, we propose a method where given a source policy, policy adaptation is performed via transfer learning to produce a target policy suitable for real-world deployment. For policy adaptation, alignment-based transfer learning is applied to trajectories generated by the source policy and their corresponding safe target trajectories. We apply this method to manipulators and show that the proposed method is applicable to both inter-task and inter-robot transfer whilst considering safety. We also show that the resulting target policy is robust and can be further improved with reinforcement learning.

This paper introduces a variant of the Singular Value Decomposition with Phase Transform (SVD-PHAT), named Difference SVD-PHAT (DSVD-PHAT), to achieve robust Sound Source Localization (SSL) in noisy conditions. Experiments are performed on a Baxter robot with a four-microphone planar array mounted on its head. Results show that this method offers similar robustness to noise as the state-of-the-art Multiple Signal Classification based on Generalized Singular Value Decomposition (GSVD-MUSIC) method, and considerably reduces the computational load by a factor of 250. This performance gain thus makes DSVD-PHAT appealing for real-time application on robots with limited on-board computing power.