IROS 2020 Papers

ROSflight is a lean, open-source autopilot system developed with the primary goal of supporting the needs of researchers working with micro aerial vehicle systems. The project consists of firmware designed to run on low-cost, readily available flight controller boards, as well as ROS packages for interfacing between the flight controller and application code and for simulation. The core objectives of the project are as follows: maintain a small, easy-to-understand code base; provide high-bandwidth, low-latency communication between the flight controller and application code; provide a straightforward interface to research application code; allow for robust safety pilot integration; and enable true software-in-the-loop simulation capability.

This paper proposes ROS-lite, a robot operating system (ROS) development framework for embedded many- core platforms based on network-on-chip (NoC) technology. Many-core platforms support the high processing capacity and low power consumption requirement of embedded systems. In this study, a self-driving software platform module is parallelized to run on many-core processors to demonstrate the practicality of embedded many-core platforms. The experimental results show that the proposed framework and the parallelized applications have met the deadline for low-speed self-driving systems.

Recognizing and understanding conversational groups, or F-formations, is a critical task for situated agents designed to interact with humans. F-formations contain complex structures and dynamics, yet are used intuitively by people in everyday face-to-face conversations. Prior research exploring ways of identifying F-formations has largely relied on heuristic algorithms that may not capture the rich dynamic behaviors employed by humans. We introduce REFORM (REcognize F-FORmations with Machine learning), a data-driven approach for detecting F-formations given human and agent positions and orientations. REFORM decomposes the scene into all possible pairs and then reconstructs F-formations with a voting-based scheme. We evaluated our approach across three datasets: the SALSA dataset, a newly collected human-only dataset, and a new set of acted human-robot scenarios, and found that REFORM yielded improved accuracy over a state-of-the-art F-formation detection algorithm. We also introduce symmetry and tightness as quantitative measures to characterize F-formations.

Human-automation collaborations, like automated driving assistance and piloting drones, have become prevalent as these technologies become more commonplace. Designers need tools that help them understand how and why design interventions may change the strategies of operators in such complex human supervisory control systems. To this end, we demonstrate that when the divergence metric is applied to Hidden Markov Model (HMM) comparisons, it can accurately capture statistical differences between operator strategies for interfaces that embody different tasks. However, the use of such an approach is problematic when used to compare HMM strategy models with non-equivalent observations. To address this limitation, we developed an observation reduction approach and conducted a sensitivity analysis to assess the impact of this approach. Our results show that when comparing two non-equivalent interfaces, our observation reduction approach does not fundamentally change the divergence metric, thus allowing for direct model comparison. The results further show that HMMs from different interfaces produce a much higher divergence metric than model comparison from the same people who repeatedly use the same interface. Future work will examine if this method can detect differences in models with different tasks or modified interfaces.

Can we design motion primitives for complex legged systems uniformly for different terrain types without neglecting modeling details? This paper presents a method for rapidly generating quadrupedal locomotion on sloped terrains-from modeling to gait generation, to hardware demonstration. At the core of this approach is the observation that a quadrupedal robot can be exactly decomposed into coupled bipedal robots. Formally, this is represented through the framework of coupled control systems, wherein isolated subsystems interact through coupling constraints. We demonstrate this concept in the context of quadrupeds and use it to reduce the gait planning problem for uneven terrains to bipedal walking generation via hybrid zero dynamics. This reduction method allows for the formulation of a nonlinear optimization problem that leverages low-dimensional bipedal representations to generate dynamic walking gaits on slopes for the full-order quadrupedal robot dynamics. The result is the ability to rapidly generate quadrupedal walking gaits on a variety of slopes. We demonstrate these walking behaviors on the Vision 60 quadrupedal robot; in simulation, via walking on a range of sloped terrains of 13°, 15°, 20°, 25°, and, experimentally, through the successful locomotion of 13° and 20° ~ 25° sloped outdoor grasslands.

Large special-events parking involves various parking scenarios, e.g., temporary parking and on-street parking. Their occupancy detection is challenging as it is unrealistic to construct gates/stations for temporary parking areas or build a sensor-based detection system to cover every single street. To address this issue, this study develops a quadrotor-enabled autonomous parking occupancy detection system. A camera-equipped quadrotor is flying over the parking lot first; then the images are captured by the on-board camera of the quadrotor and transferred to the ground station; finally, the ground station will process and release the occupancy information to the driver's mobile devices. The decision tree learning algorithm is adopted to determine the optimal flying speed for the quadrotor to balance the trade-off between the detection efficiency and accuracy. In order to tackle the complex environment in real-life parking, a convolutional neural network (CNN)-based vehicle detection model has been trained and implemented, where the realistic factors, e.g., passing pedestrians and tree blocking, are considered. Experiments are conducted to illustrate the effectiveness of the proposed system.

Humans perceive and describe their surroundings with qualitative statements (e.g., "Alice's hand is in contact with a bottle."), rather than quantitative values (e.g., 6-D poses of Alice's hand and a bottle). Qualitative spatial representation (QSR) is a framework that represents the spatial information of objects in a qualitative manner. Region connection calculus (RCC), qualitative trajectory calculus (QTC), and qualitative distance calculus (QDC) are some popular QSR calculi. With the recent development of computer vision, it is important to compute QSR calculi from the visual inputs (e.g., RGB-D images). In fact, many QSR application domains (e.g., human activity recognition (HAR) in robotics) involve visual inputs. We propose a qualitative spatial representation network (QSRNet) that computes the three QSR calculi (i.e., RCC, QTC, and QDC) from the RGB-D images. QSRNet has the following novel contributions. First, QSRNet models the dependencies among the three QSR calculi. We introduce the dependencies as kinematics for QSR because they are analogous to the kinematics in classical mechanics. Second, QSRNet applies the 3-D point cloud instance segmentation to compute the QSR calculi. The experimental results show that QSRNet improves the accuracy in comparison to the other state-of-the-art techniques.

We present PufferBot, an aerial robot with an expandable structure that may expand to protect a drone's propellers when the robot is close to obstacles or collocated humans. PufferBot is made of a custom 3D-printed expandable scissor structure, which utilizes a one degree of freedom actuator with rack and pinion mechanism. We propose four designs for the expandable structure, each with unique characterizations for different situations. Finally, we present three motivating scenarios in which PufferBot may extend the utility of existing static propeller guard structures.

This paper introduces two simple techniques to improve off-policy Reinforcement Learning (RL) algorithms. First, we formulate off-policy RL as a stochastic proximal point iteration. The target network plays the role of the variable of optimization and the value network computes the proximal operator. Second, we exploits the two value functions commonly employed in state-of-the-art off-policy algorithms to provide an improved action value estimate through bootstrapping with limited increase of computational resources. Further, we demonstrate significant performance improvement over state-of-the-art algorithms on standard continuous-control RL benchmarks.

We present ProxEmo, a novel end-to-end emotion prediction algorithm for socially aware robot navigation among pedestrians. Our approach predicts the perceived emotions of a pedestrian from walking gaits, which is then used for emotion-guided navigation taking into account social and proxemic constraints. To classify emotions, we propose a multi-view skeleton graph convolution-based model that works on a commodity camera mounted onto a moving robot. Our emotion recognition is integrated into a mapless navigation scheme and makes no assumptions about the environment of pedestrian motion. It achieves a mean average emotion prediction precision of 82.47% on the Emotion-Gait benchmark dataset. We outperform current state-of-art algorithms for emotion recognition from 3D gaits. We highlight its benefits in terms of navigation in indoor scenes using a Clearpath Jackal robot.

Real-world environments are inherently uncertain, and to operate safely in these environments robots must be able to plan around this uncertainty. In the context of motion planning, we desire systems that can maintain an acceptable level of safety as the robot moves, even when the exact locations of nearby obstacles are not known. In this paper, we solve this chance-constrained motion planning problem using a sequential convex optimization framework. To constrain the risk of collision incurred by planned movements, we employ geometric objects called ε-shadows to compute upper bounds on the risk of collision between the robot and uncertain obstacles. We use these ε-shadow-based estimates as constraints in a nonlinear trajectory optimization problem, which we then solve by iteratively linearizing the non-convex risk constraints. This sequential optimization approach quickly finds trajectories that accomplish the desired motion while maintaining a user-specified limit on collision risk. Our method can be applied to robots and environments with arbitrary convex geometry; even in complex environments, it runs in less than a second and provides provable guarantees on the safety of planned trajectories, enabling fast, reactive, and safe robot motion in realistic environments.

Estimation of a quadruped's state is fundamentally important to its operation. In this paper we develop a low-level state estimator for quadrupedal robots that includes attitude, odometry, ground reaction forces, and contact detection. The state estimator is divided into three parts. First, a nonlinear observer estimates attitude by fusing inertial measurements. The attitude estimator is globally exponentially stable and is able to initialize with large errors in the initial state estimates whereas a state-of-the-art EKF would diverge. This is practical for situations when the robot has fallen over and needs to start from its side. Second, leg odometry is calculated with encoders, force sensors, and torque sensors in the robot's joints. Lastly, the leg odometry and inertial measurements are fused to obtain linear position and velocity. We experimentally validate the state estimator using a novel dataset from the HyQ robot. For the entirety of the experiment the estimated attitude matched the ground truth data and had a root mean square error (RMSE) of [2 1 5] deg, the velocity estimates has a RMSE of [0.11 0.15 0.04] m/s, and the position estimates, which are unobservable, drifted on average [2 1 8] mm/s.

This paper presents a teaching by demonstration method for contact tasks with periodic movement on planar surfaces of unknown pose. To learn the motion on the plane, we utilize frequency oscillators with periodic movement primitives and we propose modified adaptation rules along with an extraction method of the task’s fundamental frequency by automatically discarding near-zero frequency components. Additionally, we utilize an online estimate of the normal vector to the plane, so that the robot is able to quickly adapt to rotated hinged surfaces such as a window or a door. Using the framework of progressive automation for compliance adaptation, the robot transitions seamlessly and bi-directionally between hand guidance and autonomous operation within few repetitions of the task. While the level of automation increases, a hybrid force/position controller is progressively engaged for the autonomous operation of the robot. Our methodology is verified experimentally in surfaces of different orientation, with the robot being able to adapt to surface orientation perturbations.

Recent advancements in statistical learning and computational abilities have enabled autonomous vehicle technology to develop at a much faster rate. While many of the architectures previously introduced are capable of operating under highly dynamic environments, many of these are constrained to smaller-scale deployments, require constant maintenance due to the associated scalability cost with highdefinition (HD) maps, and involve tedious manual labeling. As an attempt to tackle this problem, we propose to fuse image and pre-built point cloud map information to perform automatic and accurate labeling of static landmarks such as roads, sidewalks, crosswalks, and lanes. The method performs semantic segmentation on 2D images, associates the semantic labels with point cloud maps to accurately localize them in the world, and leverages the confusion matrix formulation to construct a probabilistic semantic map in bird’s eye view from semantic point clouds. Experiments from data collected in an urban environment show that this model is able to predict most road features and can be extended for automatically incorporating road features into HD maps with potential future work directions.

Simultaneous localization and mapping (SLAM) is essential in numerous robotics applications such as autonomous navigation. Traditional SLAM approaches infer the metric state of the robot along with a metric map of the environment. While existing algorithms exhibit good results, they are still sensitive to measurement noise, sensors quality, data association and are still computationally expensive. Alternatively, we note that some navigation and mapping missions can be achieved using only qualitative geometric information, an approach known as qualitative spatial reasoning (QSR). In this work we contribute a novel probabilistic qualitative localization and mapping approach, which extends the state of the art by inferring also the qualitative state of the camera poses (localization), as well as incorporating probabilistic connections between views (in time and in space). Our method is in particular appealing in scenarios with a small number of salient landmarks and sparse landmark tracks. We evaluate our approach in simulation and in a real-world dataset, and show its superior performance and low complexity compared to state of the art.

Trajectory prediction is crucial for autonomous vehicles. The planning system not only needs to know the current state of the surrounding objects but also their possible states in the future. As for vehicles, their trajectories are significantly influenced by the lane geometry and how to effectively use the lane information is of active interest. Most of the existing works use rasterized maps to explore road information, which does not distinguish different lanes. In this paper, we propose a novel instance-aware representation for lane representation. By integrating the lane features and trajectory features, a goal-oriented lane attention module is proposed to predict the future locations of the vehicle. We show that the proposed lane representation together with the lane attention module can be integrated into the widely used encoder-decoder framework to generate diverse predictions. Most importantly, each generated trajectory is associated with a probability to handle the uncertainty. Our method does not suffer from collapsing to one behavior modal and can cover diverse possibilities. Extensive experiments and ablation studies on the benchmark datasets corroborate the effectiveness of our proposed method. Notably, our proposed method ranks third place in the Argoverse motion forecasting competition at NeurIPS 2019 1.

The process of planning views to observe a scene is known as the Next Best View (NBV) problem. Approaches often aim to obtain high-quality scene observations while reducing the number of views, travel distance and computational cost. Considering occlusions and scene coverage can significantly reduce the number of views and travel distance required to obtain an observation. Structured representations (e.g., a voxel grid or surface mesh) typically use raycasting to evaluate the visibility of represented structures but this is often computationally expensive. Unstructured representations (e.g., point density) avoid the computational overhead of maintaining and raycasting a structure imposed on the scene but as a result do not proactively predict the success of future measurements. This paper presents proactive solutions for handling occlusions and considering scene coverage with an unstructured representation. Their performance is evaluated by extending the density-based Surface Edge Explorer (SEE). Experiments show that these techniques allow an unstructured representation to observe scenes with fewer views and shorter distances while retaining high observation quality and low computational cost.

Passive sensing with ambient WiFi signals is a promising technique that will enable new types of human-robot interactions while preserving users' privacy. Here, we present PresSense, a system for human respiration sensing in noisy environments. Unlike existing WiFi-based respiration sensors, we employ a human presence detector, improving the robustness in scenarios where no human is present in an Area Of Interest (AOI). We also integrate our novel feature, Peak Distance Histogram (PDH), with other classic WiFi features to achieve better accuracy when someone is present in the AOI. We tested our system using commodity WiFi devices in an office room. Our PresSense outperforms the state of the arts in both respiration rate estimation and presence detection.

We present a predictive runtime monitoring technique for estimating future vehicle positions and the probability of collisions with obstacles. Vehicle dynamics model how the position and velocity change over time as a function of external inputs. They are commonly described by discrete-time stochastic models. Whereas positions and velocities can be measured, the inputs (steering and throttle) are not directly measurable in these models. In our paper, we apply Bayesian inference techniques for real-time estimation, given prior distribution over the unknowns and noisy state measurements. Next, we pre-compute the set-valued reachability analysis to approximate future positions of a vehicle. The pre-computed reachability sets are combined with the posterior probabilities computed through Bayesian estimation to provided a predictive verification framework that can be used to detect impending collisions with obstacles. Our approach is evaluated using the coordinated-turn vehicle model for a UAV using on-board measurement data obtained from a flight test of a Talon UAV. We also compare the results with sampling-based approaches. We find that precomputed reachability analysis can provide accurate warnings up to 6 seconds in advance and the accuracy of the warnings improve as the time horizon is narrowed from 6 to 2 seconds. The approach also outperforms sampling in terms of on-board computation cost and accuracy measures.