IROS 2018 论文列表

This paper strives to answer the following question: Is it possible to recognize an intersection when seen from different road segments that constitute the intersection? An intersection or a junction typically is a meeting point of three or four road segments. Its recognition from a road segment that is transverse to or 180 degrees apart from its previous sighting is an extremely challenging and yet a very relevant problem to be addressed from the point of view of both autonomous driving as well as loop detection. This paper formulates this as a problem of video recognition and proposes a novel LSTM based Siamese style deep network for video recognition. For what is indeed a challenging problem and the limited annotated dataset available we show competitive results of recognizing intersections when approached from diverse viewpoints or road segments. Specifically, we tabulate effective recognition accuracy even as the approaches to the intersection being compared are disparate both in terms of viewpoints and weather/illumination conditions. We show competitive results on both synthetic yet highly realistic data mined from the gaming platform GTA as well as on real world data made available through Mapillary.

We present a visual odometry (VO) algorithm for a multi-camera system and robust operation in challenging environments. Our algorithm consists of a pose tracker and a local mapper. The tracker estimates the current pose by minimizing photometric errors between the most recent keyframe and the current frame. The mapper initializes the depths of all sampled feature points using plane-sweeping stereo. To reduce pose drift, a sliding window optimizer is used to refine poses and structure jointly. Our formulation is flexible enough to support an arbitrary number of stereo cameras. We evaluate our algorithm thoroughly on five datasets. The datasets were captured in different conditions: daytime, night-time with near-infrared (NIR) illumination and nighttime without NIR illumination. Experimental results show that a multi-camera setup makes the VO more robust to challenging environments, especially night-time conditions, in which a single stereo configuration fails easily due to the lack of features.

Unsupervised depth estimation from a single image is a very attractive technique with several implications in robotic, autonomous navigation, augmented reality and so on. This topic represents a very challenging task and the advent of deep learning enabled to tackle this problem with excellent results. However, these architectures are extremely deep and complex. Thus, real-time performance can be achieved only by leveraging power-hungry GPUs that do not allow to infer depth maps in application fields characterized by low-power constraints. To tackle this issue, in this paper we propose a novel architecture capable to quickly infer an accurate depth map on a CPU, even of an embedded system, using a pyramid of features extracted from a single input image. Similarly to state-of-the-art, we train our network in an unsupervised manner casting depth estimation as an image reconstruction problem. Extensive experimental results on the KITTI dataset show that compared to the top performing approach our network has similar accuracy but a much lower complexity (about 6% of parameters) enabling to infer a depth map for a KITTI image in about 1.7 s on the Raspberry Pi 3 and at more than 8 Hz on a standard CPU. Moreover, by trading accuracy for efficiency, our network allows to infer maps at about 2 Hz and 40 Hz respectively, still being more accurate than most state-of-the-art slower methods. To the best of our knowledge, it is the first method enabling such performance on CPUs paving the way for effective deployment of unsupervised monocular depth estimation even on embedded systems.

For successful physical human-robot interaction, the capability of a robot to understand its environment is imperative. More importantly, the robot should extract from the human operator as much information as possible. A reliable 3D skeleton extraction is essential for a robot to predict the intentions of the operator while s/he moves toward the robot or performs a meaningful gesture. For this purpose, we have integrated a time-of-flight depth camera with a state-of-the-art 2D skeleton extraction library namely Openpose, to obtain 3D skeletal joint coordinates reliably. We have also developed a robust and rotation invariant (in the coronal plane)hand gesture detector using a convolutional neural network. At run time (after having been trained)the detector does not require any pre-processing of the hand images. A complete pipeline for skeleton extraction and hand gesture recognition is developed and employed for real-time physical human-robot interaction, demonstrating the promising capability of the designed framework. This work establishes a firm basis and will be extended for the development of intelligent human intention detection in physical human-robot interaction scenarios, to efficiently recognize a variety of static as well as dynamic gestures.

Modular self-folding robots are versatile systems that can change their own shape from two-dimensional patterns at instant commands. This reconfigurability is commonly restrained by power limitation in autonomous environments, The robotic systems with insufficient torque may lead to inaccurate movements and even transformation failures. This paper presents methodology for optimized reconfiguration planning with torque limitation in modular self-folding robots. We determine reconfiguration schemes with optimal initial pattern and robotic base that result in minimal peak torque by minimizing robotic inertia of the modular architecture. We present minimal bounding box and capacitated spanning tree heuristic algorithms to generate optimal initial patterns and propose 3 heuristic rules for robotic base selection. Our approach is demonstrated in simulation by applying the algorithms to the robotic concept of Mori, a modular origami robot. The simulation results show that the proposed algorithms yield reconfiguration schemes with low peak torque, thereby appropriate for real-time applications in modular robotic systems.

Social norms are the understandings that govern the behavior of members of a society. As such, they regulate communication, cooperation and other social interactions. Robots capable of reasoning about social norms are more likely to be recognized as an extension of our human society. However, norms stated in a form of the human language are inherently vague and abstract. This allows for applying norms in a variety of situations, but if the robots are to adhere to social norms, they must be capable of translating abstract norms to the robotic language. In this paper we use a notion of institution to realize social norms in real robotic systems. We illustrate our approach in a case study, where we translate abstract norms into concrete constraints on cooperative behaviors of humans and robots. We investigate the feasibility of our approach and quantitatively evaluate the performance of our framework in 30 real experiments with user-based evaluation with 40 participants.

As many imitation learning algorithms focus on pure trajectory generation in either Cartesian space or joint space, the problem of considering competing trajectory constraints from both spaces still presents several challenges. In particular, when perturbations are applied to the robot, the underlying controller should take into account the importance of each space for the task execution, and compute the control effort accordingly. However, no such controller formulation exists. In this paper, we provide a minimal intervention control strategy that simultaneously addresses the problems of optimal control and competing constraints between Cartesian and joint spaces. In light of the inconsistency between Cartesian and joint constraints, we exploit the robot null space from an information-theory perspective so as to reduce the corresponding conflict. An optimal solution to the aforementioned controller is derived and furthermore a connection to the classical finite horizon linear quadratic regulator (LQR) is provided. Finally, a writing task in a simulated robot verifies the effectiveness of our approach.

By briefly heating the local environment with a heat lamp and observing what happens with a thermal camera, robots could potentially infer properties of their surroundings. However, this form of active thermography introduces large signal variations compared to traditional active thermography, which has typically been used to characterize small regions of materials in carefully controlled settings. We demonstrate that a data-driven approach with modern machine learning methods can be used to classify material samples over relatively large surface areas and variable distances. We also introduce the use of z-normalization to improve material classification and reduce variation due to distance and heating intensity. Our best performing algorithm achieved an overall accuracy of 77.7% for multi-class classification among 12 materials placed at varying distances (20 cm, 30 cm, and 40 cm). The observations were made for 5 seconds with 1s of heating and 4s of cooling. We also provide a demonstration of performance with a multi-material scene.

Active learning literature has explored the selection of optimal queries by a learning agent with respect to given criteria, but prior work in classification has focused only on obtaining labels for queried samples. In contrast, proficient learners, like humans, integrate multiple forms of information during learning. This work seeks to enable an active learner to reason about multiple query types concurrently, aimed at soliciting both instance and feature information from the teacher, and to autonomously arbitrate between queries of different types. We contribute the design of rule-based and decision-theoretic arbitration strategies and evaluate all against baselines of more traditional passive and active learning. Our findings show that all arbitration strategies lead to more efficient learning, compared to the baselines. Moreover, given a dynamically changing environment and constrained questioning budget (typical in human settings), the decision-theoretic strategy statistically outperforms all other methods since it reasons about both what query to make and when to make a query, in order to most effectively utilize its questioning budget.

Event cameras are an emerging technology in computer vision, offering extremely low latency and bandwidth, as well as a high temporal resolution and dynamic range. Inherent data compression is achieved as pixel data is only produced by contrast changes at the edges of moving objects. However, current trends in state-of-the-art visual algorithms rely on deep-learning with networks designed to process colour and intensity information contained in dense arrays, but are notoriously computationally heavy. While the combination of these visual technologies could lead to fast, efficient, and accurate detection and recognition algorithms, it is uncertain whether the compressed event-camera data actually contain the required information for these techniques to discriminate between objects and a cluttered background. This paper presents a pilot study in which off-the-shelf deep-learning is applied to visual events for object detection on the iCub robotic platform, and analyses the impact of temporal integration of the event data. We also present a novel pipeline that bootstraps event-based dataset annotation from mature frame-based algorithms, in order to more quickly generate the required datasets.

System identification of High Altitude Long Endurance fixed-wing aerial vehicles is challenging as its operating flight envelope covers wide ranges of altitudes and Mach numbers. We present a new global system identification framework geared towards such fixed-wing aerial platforms where the aim is to build a global aerodynamic model without many repetitions of local system identification procedures or the use of any aerodynamic database. Instead we apply parameter identification techniques to virtually created system identification data and update the identified parameters with available flight test data. The proposed framework was evaluated using data set outside the flight envelope of the available flight test data, i.e. at different airspeeds considering both interpolation and extrapolation scenarios. The error analysis has shown that the obtained longitudinal aerodynamic model can accurately predict the pitch rate and pitch angle, mostly within a tolerance of \pm \pmb{1.5}\pm \pmb{1.5} degrees/s and \pm \pmb{2}\pm \pmb{2} degrees respectively. Such a cost and time efficient model development framework enables high fidelity simulation and precise control which ultimately leads to higher success rates in autonomous missions.

In the last few years novel color and depth (RGB-D) sensors have greatly pushed robot perception. To enable a precise pixel-wise fusion of color and depth information good calibration is needed. The calibration determines the intrinsic parameters, the extrinsic parameters, and corrects for depth errors. While classic calibration approaches involve a dedicated calibration target and a trained expert, the autonomous calibration of such camera systems for robots operating in unknown environments is still an open problem. It demands for robust methods that do not need an expert to set up or tune the algorithm. Hence, we present a robust calibration algorithm that utilizes structure from motion (SfM) reconstructions as a calibration target and incorporates plane priors in the optimization to improve the convergence behavior and improve the calibration robustness. We evaluate our method against the state of the art performing over 300 experiments on ten different datasets, and show a significant improvement of the calibration accuracy.

Robot-assisted Fenestrated Endovascular Aortic Repair (FEVAR) is currently navigated by 2D fluoroscopy which is insufficiently informative. Previously, a semi-automatic 3D shape instantiation method was developed to instantiate the 3D shape of a main, deployed, and fenestrated stent graft from a single fluoroscopy projection in real-time, which could help 3D FEVAR navigation and robotic path planning. This proposed semi-automatic method was based on the Robust Perspective-S-Point (RP5P) method, graft gap interpolation and semiautomatic multiple-class marker center determination. In this paper, an automatic 3D shape instantiation could be achieved by automatic multiple-class marker segmentation and hence automatic multiple-class marker center determination. Firstly, the markers were designed into five different shapes. Then, Equally-weighted Focal U-Net was proposed to segment the fluoroscopy projections of customized markers into five classes and hence to determine the marker centers. The proposed Equally-weighted Focal U-Net utilized U-Net as the network architecture, equally-weighted loss function for initial marker segmentation, and then equally-weighted focal loss function for improving the initial marker segmentation. This proposed network outperformed traditional Weighted U-Net on the class-imbalance segmentation in this paper with reducing one hyperparameter - the weight. An overall mean Intersection over Union (mIoU) of 0.6943 was achieved on 78 testing images, where 81.01 % markers were segmented with a center position error <; 1.6mm. Comparable accuracy of 3D shape instantiation was also achieved and stated. The data, trained models and TensorFlow codes are available on-line.

Sensor payloads suspended from parachutes are often used in atmospheric profiling applications. They drift freely and often end up landing in inaccessible regions that make their retrieval challenging or impossible. In this paper, we develop and evaluate an approach using a multirotor unmanned aerial system to autonomously retrieve the parachute while it is still in the air. The system relies only on the initial conditions of the parachute-payload system and feedback from the vehicle's onboard cameras to track and then intercept the parachute mid-air in under 40 seconds on average. We present the results from our field experiments where we demonstrate the feasibility of the system and discuss its applicability to long-term payload transportation systems.

The gap between human waiters and state-of-the-art robot systems that try to serve something to drink is often embarrassing, with the former able to manipulate glasses and trays or glasses on trays with incredible dexterity and the latter that move at incredible slowness. In this video, we want to show that robots can do it better by moving a bottle or a tankard full of beer that are simply placed on a flat steel plate connected the flange of a robot manipulator. The robot tracks the trajectory defined by a human operator that moves its hand in the 3D space, with a motion capture system that acquires in real time the position. A feed-forward controller, placed between the user and the robot and based on the combination of a smoother and proper orientation compensation, counteracts the lateral accelerations and suppress sloshing phenomena of the liquids. Eventually a camera mounted on the robot arm provides a visual feedback to the operator with monitoring purposes. The challenge for the operator was to drop the carried object. will the feed-forward control be robust enough to avoid this event, even at high speed? Watch the video and find out!

Biped robots have not achieved the efficient and harmonious locomotion of the human beings, capable of walking and running on unstructured terrains, with obstacles, holes and slopes. With this in mind, researchers started the development of biomimetic solutions to control the locomotion of biped models. This work presents a new solution of motion control of bipedal robots with adaptable stiffness, by exploring effects of joint stiffness in modulating walking behavior. Further, torque adjustment is achieved through a biomimetic controller that mimics and adjusts the natural dynamics of the robot to the environment. Specifically, the torque adjustment is made using AFOs (adaptive frequency oscillator) to generate the correct equilibrium positions that will be applied to the impedance control that computes the torque of each joint. Results show that the biped model is capable of walking in several types of terrain, including flat terrain, ramps, stairs and flat terrain with obstacles.

We propose a novel real-time algorithm to estimate the full three-dimensional individual tire forces (i.e., vertical, longitudinal as well as lateral) of a car-like rearwheel-driven four wheel wheeled mobile robots equipped with onboard navigation sensors and wheel encoders. The key enabling idea for this is to utilize the tire model (i.e., the magic formula) in a feedback manner on the framework of the constrained Kalman filtering to render the tire force estimation: 1) more accurate as compared to the typical tire force estimation techniques neglecting the tire-road interaction; and 2) more robust as compared to the results adopting the tire model, yet, only in an open-loop manner. Our proposed algorithm, while performing this full tire force onboard/real-time estimation, also provides the estimation of: 1) tire-road friction coefficient; and 2) torque inputs of the rear left and right wheels, which are connected via differential gear. Simulations with CarSim and outdoor experiments are performed to validate the proposed estimation algorithm.

To properly fuse IMU and camera information for robotics applications, the relative timestamp offset between both sensors' data streams has to be considered. However, finding the exact timestamp offset is often impossible. Thus, it is necessary to additionally consider the offset's uncertainty if we want to produce reliable results. In order to find the offset and its uncertainty, we determine orientation estimates from IMU and camera under interval uncertainty. Subsequently, these intervals are used as a common representation for our bounded-error approach that finds an interval enclosing the true offset while also modeling the uncertainty. Calibration data can be acquired in a few seconds using a simple setup of IMU, camera and camera target. Results using both simulated and real data demonstrate that we are able to determine the offset to an accuracy of 20 ms with a computation time that is suitable for future online applications. Here, our approach could be used to monitor the timestamp offset in a guaranteed way. Additionally, our method can be adapted to determine an interval for the rotation between both sensors. While this increases the computation time drastically, it also enhances the accuracy of the timestamp offset to less than 10 ms.

Aerial filming in action scenes using a drone is difficult for inexperienced flyers because manipulating a remote controller and meeting the desired image composition are two independent, while concurrent, tasks. Existing systems attempt to utilize wearable GPS-based or infrared-based sensors to track the human movement and to assist in capturing footage. However, these sensors work only in either indoor (infrared-based) or outdoor environments (GPS-based), but not both. In this paper, we introduce a novel drone filming system which integrates monocular 3D human pose estimation and localization into a drone platform to remove the constraints imposed by wearable-sensor-based solutions. Meanwhile, given the estimated position, we propose a novel drone control system, called “through-the-lens drone filming”, to allow a cameraman to conveniently control the drone by manipulating a 3D model in the preview, which closes the gap between the flight control and the viewpoint design. Our system includes two key enabling techniques: 1) subject localization based on visual-inertial fusion, and 2) through-the-lens camera planning. This is the first drone camera system which allows users to capture human actions by manipulating the camera in a virtual environment. From the drone hardware, we integrate a gimbal camera and two GPUs into the limited space of a drone and demonstrate the feasibility of running the entire system onboard with insignificant delays, which are sufficient for filming in our real-time application. Experimental results, in both simulation and real-world scenarios, demonstrate that our techniques can greatly ease camera control and capture better videos.