ICRA 2019 Papers

This paper presents a new method of maximizing the free space for a robot operating in a constrained environment under operator supervision. The objective is to make the resulting trajectories more robust to operator commands and/or changes in the environment. To represent the volume of free space, the constrained manipulability polytopes are used. These polytopes embed the distance to obstacles, the distance to joint limits and the distance to singular configurations. The volume of the resulting Cartesian polyhedron is used in an optimization-based motion planner to create the trajectories. Additionally, we show how fast collision-free inverse kinematic solutions can be obtained by exploiting the pre-computed inequality constraints. The proposed algorithm is validated in simulation and experimentally.

Belief space planning (BSP) is a fundamental problem in robotics. Determining an optimal action quickly grows intractable as it involves calculating the expected accumulated cost (reward), where the expectation accounts for all future measurement realizations. State of the art approaches therefore resort to simplifying assumptions and approximations to reduce computational complexity. Importantly, while in robotics re-planning is essential, these approaches calculate each planning session from scratch. In this work we contribute a novel approach, iX-BSP, that is based on the key insight that calculations in consecutive planning sessions are similar in nature and can be thus re-used. Our approach performs incremental calculation of the expectation by appropriately re-using computations already performed in a precursory planing session while accounting for the information obtained in inference between the two planning sessions. The formulation of our approach considers general distributions and accounts for data association aspects. We evaluate iX-BSP in statistical simulation and show that incremental expectation calculations significantly reduce runtime without impacting performance.

Many popular advanced sonar systems provide accurate and reliable measurements containing crucial info needed by robotic applications such as range, bearing and reflection strength of the objects in the field of view. While these sensor systems provide these crucial pieces of information accurately, they are often limited by a lack of processing power and/or size which leads to them needing an external computing device to process all the information generated by the microphone array on the sensor. In this paper we present two versions of a novel fully embedded 3D sonar sensor which have different sensing architectures which enable 3D perception for robotic application in harsh conditions using ultrasound at low cost. Experimental results taken from an office environment will show the 3D localization capabilities and performance of the sensor, showing the sensor has a large field-of-view (FoV) with accurate 3D localization combined with real-time capabilities.

Flapping-wing micro-aerial vehicles rely on subtle changes in the kinematics of high-frequency wing flapping to produce roll, pitch, and yaw torques. To generate yaw torque, the Harvard RoboBee changes the ratio of upstroke to downstroke speed (“split-cycling”) by applying a second harmonic to the fundamental flapping signal for each wing. However, since flapping typically occurs near resonance (for efficiency), these higher harmonics are filtered out by the transmission and actuator dynamics. Therefore, reliable yaw control authority has proven elusive. We propose a method to generate yaw torque sufficient for in-flight control by using split-cycle flapping in an “iso-lift” regime, to mitigate resonant filtering by decreasing the flapping frequency and increasing the drive voltage, which produces lift identical to typical flight conditions. We model the expected torque at iso-lift conditions and apply this method to the physical RoboBee, achieving reliable, controllable yaw torque. Finally, we demonstrate yaw control with a simple heading controller, achieving a step response with a time constant an order of magnitude faster than previous attempts.

Typically, to enlarge the operating domain of an object detector, more labeled training data is required. We describe a method called wormhole learning, which allows to extend the operating domain without additional data, but only with temporary access to an auxiliary sensor with certain invariance properties. We describe the instantiation of this principle with a regular visible-light RGB camera as the main sensor, and an infrared sensor as the temporary sensor. We start with a pre-trained RGB detector; then we train the infrared detector based on the RGB-inferred labels; finally we re-train the RGB detector based on the infrared-inferred labels. After these two transfer-learning steps, the RGB detector has enlarged its operating domain by inheriting part of the invariance to illumination of the infrared sensor; in particular, the RGB detector is now able to see much better at night. We analyze the wormhole learning phenomenon by bounding the possible gain in accuracy using mutual information properties of the two sensors and considered operating domain.

In this paper, we focus on the problem of directing the gaze of a vision system mounted to the body of a high-degree-of-freedom (DOF) legged robot for active perception deployments. In particular, we consider the case where the vision system is rigidly attached to the robot's body (i.e., without any additional DOF between the vision system and robot body) and show how the supernumerary DOFs of the robot can be leveraged to allow independent locomotion and gaze control. Specifically, we augment a workspace central pattern generator (CPG) with omnidirectional capabilities by coupling it with a body pose control mechanism. We leverage the smoothing nature of the CPG framework to allow online adaptation of relevant locomotion parameters, and obtain a stable mid-level controller that translates desired gaze orientation and body velocity directly into joint angles. We validate our approach on an 18-DOF hexapod robot, in a series of indoor and outdoor trials, where the robot inspects an environmental feature or follows a pre-planned path relative to a visually-tracked landmark, demonstrating simultaneous locomotion and directed vision.

An adaptive sensing method is presented to control the pinging interval of a downward-looking sonar on an Autonomous Underwater Vehicle. The goal is to conserve energy via adjusting the pinging rate automatically without reducing the localization accuracy when using terrain-aided navigation (TAN). In this paper, the TAN is implemented using a particle filter and a bias velocity estimator developed based on a Kalman filter. The adaptation on the sonar pinging interval is determined based on the depth variation of local seafloor topography which is quantified using a modified Teager Kaiser energy operator. As a result, more measurements are collected on high relief regions, and less measurements are obtained on relatively flat and smooth regions. We evaluated the adaptive sensing method in a simulated environment and applied it to a field data set. The results show that the adaptive sensing method produces an improved navigational accuracy compared to the missions with fixed sonar pinging rates. In the offline field missions, the energy consumed by the altimeter is reduced to about 30% in the adaptive sensing missions compared to continuously sensing missions where the altimeter is pinging consistently without switching off.

Humanoid robots are expected to interact in human environments, where physical interactions are unavoidable. Therefore, whole-body control methods that include multi-contact interactions are required. The new emerging technologies in touch sensing are fundamental to acquire online and rich information about these physical interactions with the environment. These technologies lead to the design of novel control systems that can profit from the tactile sensor information in an efficient form, thus producing reactive and compliant robots capable of interacting with their environment. In this paper, we present a novel control framework to integrate the multi-modal tactile information of a robot skin with different control strategies, producing dynamic behaviours suitable for Human-Robot Interactions (HRI). The control framework was experimentally evaluated on a full-size humanoid robot covered with more than 1260 skin cells distributed in the whole robot body. The results show that multi-modal tactile information can be fused hierarchically with multiple control strategies, producing active compliance in a position-controlled stiff humanoid robot.

Advanced Internet of Things (IoT) techniques have made human-environment interaction much easier. Existing solutions usually enable such interactions without knowing the identities of action performers. However, identifying users who are interacting with environments is a key to enable personalized service. To provide such add-on service, we propose WTW (who takes what), a system that identifies which user takes what object. Unlike traditional vision-based approaches, which are typically vulnerable to blockage, our WTW combines the feature information of three types of data, i.e., images, skeletons and IMU data, to enable reliable user-object matching and identification. By correlating the moving trajectory of a user monitored by inertial sensors with the movement of an object recorded in the video, our WTW reliably identifies a user and matches him/her with the object on action. Our prototype evaluation shows that WTW achieves a recognition rate of over 90% even in a crowd. The system is reliable even when users locate close by and take objects roughly at the same time.

Autonomous vehicle manufacturers recognize that LiDAR provides accurate 3D views and precise distance measures under highly uncertain driving conditions. Its practical implementation, however, remains costly. This paper investigates the optimal LiDAR configuration problem to achieve utility maximization. We use the perception area and non-detectable subspace to construct the design procedure as solving a min-max optimization problem and propose a bio-inspired measure - volume to surface area ratio (VSR) - as an easy-to-evaluate cost function representing the notion of the size of the non-detectable subspaces of a given configuration. We then adopt a cuboid-based approach to show that the proposed VSR-based measure is a well-suited proxy for object detection rate. It is found that the Artificial Bee Colony evolutionary algorithm yields a tractable cost function computation. Our experiments highlight the effectiveness of our proposed VSR measure in terms of cost-effectiveness configuration as well as providing insightful analyses that can improve the design of AV systems.

This paper presents a new passive wheel-leg transformation mechanism and its embodiment in a small mobile robot. The mechanism is based on a unique geared structure, allowing the wheel to transform between two modes, i.e., wheel or leg, potentially adapting to varying ground conditions. It consists of a central gear and legs with partial gears that rotate around the central gear to open or close the legs. When fully closed, the mechanism forms a seamless circular wheel; when opened, it operates in the leg mode. The central gear actuated by the driving motor generates opening and closing motions of the legs without using an additional actuator. The number of legs, their physical size, and the gear ratio between the central gear and the partial gears on the legs are adjustable. This design is mechanically simple, customizable, and easy to fabricate. For physical demonstration and experiments, a mobile robotic platform was built and its terrainability was tested using five different sets of the transformable wheels with varying sizes and gear ratios. For each design, the performance with successful wheel-leg transformation, obstacle climbing, and locomotion capabilities was tested in different ground conditions.

One of the challenges of developing autonomous vehicles is planning in an inhabited environment under sensing uncertainty as well as limited perception and computational resources. Besides reasoning about the behaviour of traffic participants that are within the vehicles' field of view, safe autonomous driving also requires the vehicle to reason about possible traffic participants that might exist beyond its sensing horizon, and to adapt its driving behaviour accordingly. This paper describes an inference and motion planning pipeline that is able to guarantee passive safety (collisions are possible, but the autonomous vehicle will be at rest) with respect to hypothetical hidden agents that have not been observed yet. We also incorporate the vehicle's reaction time due to sensing and computational delays into the planning process; for example, we show how having a fast reaction time due to the availability of more computational resources leads to more aggressive trajectories, while a car with a larger reaction time will choose more relaxed trajectories that require less attention.

Mobile robots are becoming very popular in real-world outdoors applications, where there are many challenges in robot control and perception. One of the most critical problems is to characterise the terrain traversed by the robot. This knowledge is indispensable for optimal terrain negotiation. Currently, most approaches are performing terrain classification from vision, but there is not enough research on terrain identification from a direct interaction of the robot with the environment. In our work, we proposed new methods for classification of force/torque data from an interaction of the legged robot foot with the ground, gathered during the walking process. We provided machine learning methods for terrain classification from raw force/torque signals for which we achieved 93% accuracy on a challenging dataset with 160 minutes of recorded fixed-length steps. We also worked on a dataset where the assumption of a fixed-length step is not valid. In this case, the final result is around 80% of accuracy. The most important fact is that the data in both cases was recorded while the robot was walking, no particular movements or controlled environment were needed. Additionally, we also proposed a clustering method which allows us to learn about the class membership based on the recorded data only, without any human supervision.

Kinematic trajectories recorded from surgical robots contain information about surgical gestures and potentially encode cues about surgeon's skill levels. Automatic segmentation of these trajectories into meaningful action units could help to develop new metrics for surgical skill assessment as well as to simplify surgical automation. State-of-the-art methods for action recognition relied on manual labelling of large datasets, which is time consuming and error prone. Unsupervised methods have been developed to overcome these limitations. However, they often rely on tedious parameter tuning and perform less well than supervised approaches, especially on data with high variability such as surgical trajectories. Hence, the potential of weak supervision could be to improve unsupervised learning while avoiding manual annotation of large datasets. In this paper, we used at a minimum one expert demonstration and its ground truth annotations to generate an appropriate initialization for a GMM-based algorithm for gesture recognition. We showed on real surgical demonstrations that the latter significantly outperforms standard task-agnostic initialization methods. We also demonstrated how to improve the recognition accuracy further by redefining the actions and optimising the inputs.

Due to the limitations of therapists time and healthcare resources to cover the increasing demand for rehabilitation services, robot-assisted rehabilitation is becoming an appealing, powerful and economical solution. In our previous research, a solution that combines Learning from Demonstration (LfD) and robotic rehabilitation to save the therapists time and reduce the therapy costs was proposed. In this paper we compare two modalities, Robot-and Telerobotic-Mediated Kinesthetic Teaching (RMKT and TMKT), for implementing LfD in robotic rehabilitation. Our results show that behaviors demonstrated in both modalities are able to be imitated accurately, but demonstrations in TMKT have less repeatability.

Spatial sensing is a fundamental requirement for applications in robotics and augmented reality. In urban spaces such as malls, airports, apartments, and others, it is quite challenging for a single robot to map the whole environment. So, we employ a swarm of robots to perform the mapping. One challenge with this approach is merging sub-maps built by each robot. In this work, we use wireless access points, which are ubiquitous in most urban spaces, to provide us with coarse orientation between sub-maps, and use a custom ICP algorithm to refine this orientation to merge them. We demonstrate our approach with maps from a building on campus and evaluate it using two metrics. Our results show that, in the building we studied, we can achieve an average Absolute Trajectory error of 0.2m in comparison to a map created by a single robot and average Root Mean Square mapping error of 1.3m from ground truth landmark locations.

In densely-packed robot swarms operating in confined regions, spatial interference-which manifests itself as a competition for physical space-forces robots to spend more time navigating around each other rather than performing the primary task. This paper develops a decentralized algorithm that enables individual robots to decide whether to stay in the region and contribute to the overall mission, or vacate the region so as to reduce the negative effects that interference has on the overall efficiency of the swarm. We develop this algorithm in the context of a distributed collection task, where a team of robots collect and deposit objects from one set of locations to another in a given region. Robots do not communicate and use only binary information regarding the presence of other robots around them to make the decision to stay or retreat. We illustrate the efficacy of the algorithm with experiments on a team of real robots.

This paper focuses on the localization and mapping problem on ground vehicles using odometric and monocular visual sensors. To improve the accuracy of vision based estimation on ground vehicles, researchers have exploited the constraint of approximately planar motion, and usually implemented it as a stochastic constraint on an SE(3) pose. In this paper, we propose a simpler algorithm that directly parameterizes the ground vehicle poses on SE(2). The out-of SE(2) motion perturbations are not neglected, but incorporated into an integrated noise term of a novel SE(2)-XYZ constraint, which associates an SE(2) pose and a 3D landmark via the image feature measurement. For odometric measurement processing, we also propose an efficient preintegration algorithm on SE(2). Utilizing these constraints, a complete visual-odometric localization and mapping system is developed, in a commonly used graph optimization structure. Its superior performance in accuracy and robustness is validated by real-world experiments in industrial indoor environments.

As inertial and visual sensors are becoming ubiquitous, visual-inertial navigation systems (VINS) have prevailed in a wide range of applications from mobile augmented reality to aerial navigation to autonomous driving, in part because of the complementary sensing capabilities and the decreasing costs and size of the sensors. In this paper, we survey thoroughly the research efforts taken in this field and strive to provide a concise but complete review of the related work - which is unfortunately missing in the literature while being greatly demanded by researchers and engineers - in the hope to accelerate the VINS research and beyond in our society as a whole.