ICRA 2019 Papers

This paper examines the problem of dynamic traffic scene classification under space-time variations in viewpoint that arise from video captured on-board a moving vehicle. Solutions to this problem are important for realization of effective driving assistance technologies required to interpret or predict road user behavior. Currently, dynamic traffic scene classification has not been adequately addressed due to a lack of benchmark datasets that consider spatiotemporal evolution of traffic scenes resulting from a vehicle's ego-motion. This paper has three main contributions. First, an annotated dataset is released to enable dynamic scene classification that includes 80 hours of diverse high quality driving video data clips collected in the San Francisco Bay area. The dataset includes temporal annotations for road places, road types, weather, and road surface conditions. Second, we introduce novel and baseline algorithms that utilize semantic context and temporal nature of the dataset for dynamic classification of road scenes. Finally, we showcase algorithms and experimental results that highlight how extracted features from scene classification serve as strong priors and help with tactical driver behavior understanding. The results show significant improvement from previously reported driving behavior detection baselines in the literature.

Contact destabilization after an impact that occurs at high-speed, e.g. when a robot steps on an obstacle of unknown height, can be tackled by injecting angular momentum damping for a short time interval immediately after the impact. This is done by making use of the motion from within the reaction null-space (RNS). The angular momentum damping results in an appropriate arm motion that stabilizes the contacts. An impact at high-speed occurs when the stepping time is very short. In this case, conventional controllers cannot handle the reaction stemming from the swing leg dynamics. A general whole-body controller is designed that makes use of the relative angular acceleration control component to inject the angular momentum damping. The proposed control method is robust; it can deal with obstacles of various height and inclination without altering the feedback gains. The controller is fast since iterative optimization is avoided. The performance is examined via a simulated dynamic stepping.

In this paper, we examine the problem of navigating cluttered environments without explicit object detection and tracking. We introduce the dynamic risk density to map the congestion density and spatial flow of the environment to a cost function for the agent to determine risk when navigating that environment. We build upon our prior work, wherein the agent maps the density and motion of objects to an occupancy risk, then navigate the environment over a specified risk level set. Here, the agent does not need to identify objects to compute the occupancy risk, and instead computes this cost function using the occupancy density and velocity fields around them. Simulations show how this dynamic risk density encodes movement information for the ego agent and closely models the object-based congestion cost. We implement our dynamic risk density on an autonomous wheelchair and show how it can be used for navigating unstructured, crowded and cluttered environments.

This study examined strategies humans chose to manipulate an object with complex (nonlinear, underactuated) dynamics, such as liquid sloshing in a cup of coffee. The problem was simplified to the well-known cart-and-pendulum system moving on a horizontal line. This model was implemented in a virtual environment and human subjects manipulated the object via a robotic manipulandum. The task was to maneuver the system from rest to arrive at a target position such that no residual oscillations of the pendulum bob remained. Our goal was to test whether humans simplified control by employing dynamic primitives, specifically submovements. Experimental velocity profiles of the human movements were compared to those predicted by three different control models. Two models used continuous optimization-based control, the third control model was based on Input Shaping. Input Shaping is a method for controlling flexible objects by convolving a motion profile with impulses of appropriate amplitude and timing. To evaluate whether humans used Input Shaping, we decomposed the velocity profiles recorded from humans into submovements, as proxies for the convolved impulses. Comparing the motion profiles from the 3 models with the experimentally measured human profiles showed superior performance of the Input Shaping model. These initial results are consistent with our hypothesis that combining dynamic primitives, submovements, is a competent description of human performance and may provide a simpler alternative to computationally complex optimization-based methods of robot control.

This research explored the generation of period-two dynamic running motion in a robot, based on the passive dynamic period-two motion of the reduced order, rolling spring-loaded inverted pendulum (R-SLIP) model. Each cycle of period-two motion consists of two stance phases separated by two flight phases. The distribution of the period-two fixed points of the model was analyzed using a return map. Models with the same or different landing angles per motion cycle were studied, and two sets of period-two motion trajectories were implemented in a robot for experimental evaluation. Without sensory feedback or control, this evaluation relied on the open loop trajectory of the model. Based on the experiments, the robot was capable of performing dynamic period-two motion.

Nowadays, robots can navigate complex and dynamic environments such as air, water, and different terrain. However, moving into the underground, and especially into the soil, is still a challenge. Soil is a complex environment, and its exploration and monitoring is a crucial aspect in different engineering fields. Although some robotic solutions for mapping the soil are available, none of them can navigate into it. In this work, we propose a new solution for dynamic obstacles detection by embedding a 6-axis force torque sensor into a plant-inspired robot for soil exploration. We measured the forces acting on the apical part of the robot while it penetrates the soil by growing. We tested the system in different configurations, and at different depths. Results show that it is possible to identify the relative position of the obstacle before touching it with the robot. By using the proposed method as control feedback it is possible to move toward the development of novel robotic systems for navigating in complex and dynamic environments, such as the soil.

The study of animals and insects have led to realization that animals select their gaits, patterns of leg movement, according to speed. For proper gait planning, the legs must be controlled for proper foot placement with respect to the body motion and ground interactions. However, in small scale robotic platforms gait planning through foot placement control is neither cost effective nor easily attainable due to a lack of available sensors. Thus, even though a desired gait is envisioned at the design phase, it is not known whether the gait is optimum. In this work, we present the comprehensive dynamic model of the miniature foldable robot, MinIAQ-II, which has four independently actuated legs. Dynamic model is used to perform gait analysis, to investigate the difference between the intended gait and the achieved gait in the absence of foot placement control. The model is verified through slow speed walking experiments on flat terrain. The work presented can be modified for different miniature robots with passive legs to predict their locomotion under no foot placement control.

This paper proposes a novel configurable wheel that exhibits desired properties of varied radius wheels. Positional manipulation of the centre hub is proposed and tested to achieve these desired characteristics of `virtual' wheels in a physical system. The centre hub is manipulated via the use of pneumatic actuators mounted to and constricted by the outer rim of the wheel, which allows for fast and accurate control to enable the vehicle ride height and gear ratio to be adjusted continuously and be maintained during the wheels' full rotation. Experiments are presented, validating this ability of the system. We envision uses for this system to extend from off-road robotics to space exploration as these wheels exhibit novel characteristics not demonstrated by other platforms.

For dynamic manipulation of flexible objects, we propose an acquisition method of a flexible object motion equation model using a deep neural network and a control method to realize a target state by calculating an optimized time-series joint torque command. By using the proposed method, any physics model of a target object is not needed, and the object can be controlled as intended. We applied this method to manipulations of a rigid object, a flexible object with and without environmental contact, and a cloth, and verified its effectiveness.

This paper addresses the problem of learning instantaneous occupancy levels of dynamic environments and predicting future occupancy levels. Due to the complexity of most real environments, such as urban streets or crowded areas, the efficient and robust incorporation of temporal dependencies into otherwise static occupancy models remains a challenge. We propose a method to capture the uncertainty of moving objects and incorporate this uncertainty information into a continuous occupancy map represented in a rich high-dimensional feature space. This data-efficient model not only allows us to learn the occupancy states incrementally, but also makes predictions about what the future occupancy states will be. Experiments performed using 2D and 3D laser data collected from crowded unstructured outdoor environments show that the proposed methodology can accurately predict occupancy states for areas of around 1000 m2 at 10 Hz, making the proposed framework ideal for online applications under real-time constraints.

Real-time navigation in dense human environments is a challenging problem in robotics. Most existing path planners fail to account for the dynamics of pedestrians because introducing time as an additional dimension in search space is computationally prohibitive. Alternatively, most local motion planners only address imminent collision avoidance and fail to offer long-term optimality. In this work, we present an approach, called Dynamic Channels, to solve this global to local quandary. Our method combines the high-level topological path planning with low-level motion planning into a complete pipeline. By formulating the path planning problem as graph-searching in the triangulation space, our planner is able to explicitly reason about the obstacle dynamics and capture the environmental change efficiently. We evaluate efficiency and performance of our approach on public pedestrian datasets and compare it to a state-of-the-art planning algorithm for dynamic obstacle avoidance. Completeness proofs are provided in the supplement at http://caochao.me/files/proof.pdf. An extended version of the paper is available on arXiv.

Object detection methods fall into two categories, i.e., two-stage and single-stage detectors. The former is characterized by high detection accuracy while the latter usually has a considerable inference speed. Hence, it is imperative to fuse their merits for a better accuracy vs. speed trade-off. To this end, we propose a dual refinement network (DRN) to boost the performance of the single-stage detector. Inheriting from the advantages of two-stage approaches (i.e., two-step regression and accurate features for detection), anchor refinement and feature offset refinement are conducted in a novel anchor-offset detection, where the detection head is comprised of deformable convolutions. Moreover, to leverage contextual information for describing objects, we design a multi-deformable head, in which multiple detection paths with different receptive field sizes devote themselves to detecting objects. Extensive experiments on PASCAL VOC and ImageNet VID datasets are conducted, and we achieve a state-of-the-art detection performance in terms of both accuracy and inference speed.

We develop a drift-free roll and pitch attitude estimation scheme for monopedal jumping robots. The estimator uses only onboard rate gyroscopes and encoders and does not rely on external sensing or processing. It is capable of recovering from attitude estimate disturbances and, together with onboard velocity estimation, enables fully autonomous stable hopping control. The estimator performs well on a small untethered robot capable of large jumps and extreme stance accelerations. We demonstrate that the robot can follow a rectangular path using onboard dead-reckoning with less than 2 meters of drift over 200 seconds and 300 jumps covering 60 m. We also demonstrate that the robot can operate untethered outdoors under human wireless joystick direction.

This paper presents a holistic approach for door opening with a cartesian impedance controlled mobile robot, a KUICA KMR iiwa. Based on a given map of the environment, the robot autonomously detects the door handle, opens doors and traverses doorways without knowledge of a door model or the door's geometry. The door handle detection uses a convolutional neural network (CNN)-based architecture to obtain the handle's bounding box in a RGB image that works robustly for various handle shapes and colors. We achieve a detection rate of 100% for an evaluation set of 38 different door handles, by always selecting for highest confidence score. Registered depth data segmentation defines the door plane to construct a handle coordinate frame. We introduce a control structure based on the task frame formalism that uses the handle frame for reference in an outer loop for the manipulator's impedance controller. It runs in soft real-time on an external computer with approximately 20 Hz since access to inner controller loops is not available for the KMR iiwa. With the approach proposed in this paper, the robot successfully opened and traversed for 22 out of 25 trials at five different doors.

Accurate state estimation is a fundamental component of robotic control. In robotic manipulation tasks, as is our focus in this work, state estimation is essential for identifying the positions of objects in the scene, forming the basis of the manipulation plan. However, pose estimation typically requires expensive 3D cameras or additional instrumentation such as fiducial markers to perform accurately. Recently, Tobin et al. introduced an approach to pose estimation based on domain randomization, where a neural network is trained to predict pose directly from a 2D image of the scene. The network is trained on computer generated images with a high variation in textures and lighting, thereby generalizing to real world images. In this work, we investigate how to improve the accuracy of domain randomization based pose estimation. Our main idea is that active perception - moving the robot to get a better estimate of pose- can be trained in simulation and transferred to real using domain randomization. In our approach, the robot trains in a domain-randomized simulation how to estimate pose from a sequence of images. We show that our approach can significantly improve the accuracy of standard pose estimation in several scenarios: when the robot holding an object moves, when reference objects are moved in the scene, or when the camera is moved around the object.

We propose an efficient multi-robot motion planning algorithm for missions captured by linear temporal logic (LTL) specifications, in the presence of bounded disturbances and denial-of-service (DoS) attacks against the communication between robots and base stations. Given an LTL formula Ψ, our goal is to construct robot trajectories, and associated control strategies, to satisfy Ψ and continuously establish communication paths between robots and base stations despite the DoS attacks and the disturbances on the robot states. Our approach combines and extends results from robust control and efficient motion planning via satisfiability modulo convex programming (SMC). We first compute a feedback controller that rejects the disturbance together with a perturbation of the DoS-free workspace that accounts for the worst-case disturbance scenario. On the perturbed workspace, we formulate the planning problem as a feasibility problem over Boolean and convex constraints, respectively capturing the DoS-resilient mission constraints and the constraints on the nominal, disturbance-free, robot dynamics. Numerical results show the effectiveness of our algorithm in providing DoS-resilient plans that are robust to disturbances and support the execution of complex missions.

This paper presents design of a robust controller with disturbance compensation for an indoor blimp robot and its realization. The movement of blimp in horizontal plane is modeled as a slider-like nonlinear system complemented with uncertain bounded disturbances. To design the output feedback controller, a homogeneous differentiator is used as an observer. Then the method for disturbance evaluation is designed, the perturbation estimate is next used in the controller for cancellation of the influence of exogenous disturbances. Control scheme is implemented on a concrete blimp, finally, the performance of blimp disturbance compensation based controller is verified in experiments.

In this paper, we propose a novel distributional reinforcement learning (RL) method which models the distribution of the sum of rewards using a mixture density network. Recently, it has been shown that modeling the randomness of the return distribution leads to better performance in Atari games and control tasks. Despite the success of the prior work, it has limitations which come from the use of a discrete distribution. First, it needs a projection step and softmax parametrization for the distribution, since it minimizes the KL divergence loss. Secondly, its performance depends on discretization hyperparameters such as the number of atoms and bounds of the support which require domain knowledge. We mitigate these problems with the proposed parameterization, a mixture of Gaussians. Furthermore, we propose a new distance metric called the Jensen-Tsallis distance, which allows the computation of the distance between two mixtures of Gaussians in a closed form. We have conducted various experiments to validate the proposed method, including Atari games and autonomous vehicle driving.

This paper details the system and methods designed to enable the autonomous estimation of distributed nuclear radiation fields within complex and possibly GPS-denied environments. A sensing apparatus consisting of three radially placed Thallium-doped Cesium Iodide (CsI(Tl)) scintillators and Silicon Photomultipliers (SiPm) combined with custom- built pulse counting circuitry is designed and the provided readings are pose-annotated using LiDAR-based localization. Given this capacity, a method that utilizes the radiation intensity readings to first calculate the immediate field gradient and then combine this information to update and co-estimate the believed field intensity and gradient across the whole environment is developed. The strategy propagates the effect of each local measurement through field gradient co-estimation and simultaneously derives a model of the underlying uncertainty. To further support the need for informative data gathering, especially in the framework of emergency and rapid reconnaissance missions, a path planning strategy is also developed that first utilizes the field intensity and uncertainty estimates to select its new waypoint and then performs terrain traversability analysis to derive admissible paths. The complete system is evaluated both in simulation and experimentally. The experimental results refer to the autonomous exploration and field estimation inside an indoor facility within which actual radioactive uranium and thorium ore sources have been distributed.