ICRA 2017 Papers

A decentralized bilateral teleoperation system is studied in this paper by using estimators with task abstraction. There is only a part of mobile robots in the multi-robot system can receive the human command via the master robot located in the local side. By exchanging information through a undirected and connected graph, mobile robots in the network have to estimate the human command and the global task function for the control of teleoperation. The control framework is more scalable and flexible that the size and formation of mobile robot network can be teleoperated. Stability analysis is presented and experimental results are illustrated to show the efficacy of the performance of the decentralized teleoperation system.

We consider the general problem of moving a large number of networked robots toward a goal position through a cluttered environment while preserving network communication connectivity and avoiding both inter-robot collisions and collision with obstacles. In contrast to previous approaches that either plan complete paths for each individual robot in the high-dimensional joint configuration space or control the robot group as a whole with explicit constraints on the group's boundary and inter-robot pairwise distance, we propose a novel decentralized online behavior-based algorithm that relies on the topological structure of the multi-robot communication and sensing graphs to solve this problem. We formally describe the communication graph as a simplicial complex that enables robots to iteratively identify the frontier nodes and coordinate forward motion through the sensing graph. This approach is proved to automatically deform robot teams for collision avoidance and always preserve connectivity. The effectiveness of our approach is demonstrated using numerical simulations. The algorithm is shown to scale linearly in the number of robots.

Creating 3D maps on robots and other mobile devices has become a reality in recent years. Online 3D reconstruction enables many exciting applications in robotics and AR/VR gaming. However, the reconstructions are noisy and generally incomplete. Moreover, during online reconstruction, the surface changes with every newly integrated depth image which poses a significant challenge for physics engines and path planning algorithms. This paper presents a novel, fast and robust method for obtaining and using information about planar surfaces, such as walls, floors, and ceilings as a stage in 3D reconstruction based on Signed Distance Fields (SDFs). Our algorithm recovers clean and accurate surfaces, reduces the movement of individual mesh vertices caused by noise during online reconstruction and fills in the occluded and unobserved regions. We implemented and evaluated two different strategies to generate plane candidates and two strategies for merging them. Our implementation is optimized to run in real-time on mobile devices such as the Tango tablet. In an extensive set of experiments, we validated that our approach works well in a large number of natural environments despite the presence of significant amount of occlusion, clutter and noise, which occur frequently. We further show that plane fitting enables in many cases a meaningful semantic segmentation of real-world scenes.

Standard control design for robot implicit force control is a typical example of model-based regulator synthesis. This paper proposes a method to improve closed-loop performance of standard model-based controllers for robot implicit force control in terms of closed-loop model matching, between desired and achieved closed-loop behaviour. To this end, a data-driven controller design method, based on the Virtual Reference Feedback Tuning (VRFT) approach, is introduced. Advantages in terms of robustness with respect to unknown environment stiffness are discussed and demonstrated. The effectiveness of the proposed control strategy is experimentally validated on an industrial robot equipped with a force sensor.

Daily activity recognition can help people to maintain a healthy lifestyle and robot to better interact with users. Robots could therefore use the information coming from the activities performed by users to give them some custom hints to improve lifestyle and daily routine. The pervasiveness of smart things together with advances in cloud robotics can help the robot to perceive and collect more information about the users and the environment. In particular thanks to the miniaturization and low cost of Inertial Measurement Units, in the last years, body-worn activity recognition has gained popularity. In this work, we investigated the performances with an unsupervised approach to recognize eight different gestures performed in daily living wearing a system composed of two inertial sensors placed on the hand and on the wrist. In this context our aim is to evaluate whether the system is able to recognize the gestures in more realistic applications, where is not possible to have a training set. The classification problem was analyzed using two unsupervised approaches (K-Mean and Gaussian Mixture Model), with an intra-subject and an inter-subject analysis, and two supervised approaches (Support Vector Machine and Random Forest), with a 10-fold cross validation analysis and with a Leave-One-Subject-Out analysis to compare the results. The outcomes show that even in an unsupervised context the system is able to recognize the gestures with an averaged accuracy of 0.917 in the K-Mean inter-subject approach and 0.796 in the Gaussian Mixture Model inter-subject one.

Action parsing in videos with complex scenes is an interesting but challenging task in computer vision. In this paper, we propose a novel deep model based on 3D CNN (convolutional neural network) and LSTM (long short-term memory) module with a multi-task learning manner for effective Deep Action Parsing (DAP3D-Net) in videos. Particularly in the training phase, each action clip, sliced to several short consecutive segments, is fed into 3D CNN followed by LSTM to model the whole action dynamic information, so that action localization, classification and attributes learning can be jointly optimized via our deep model. Once the DAP3D-Net is trained, for an upcoming test video, we can describe each individual action in the video simultaneously as: Where the action occurs; What the action is and How the action is performed. To well demonstrate the effectiveness of the proposed DAP3D-Net, we also contribute a new Numerous-category Aligned Synthetic Action dataset, i.e., NASA, which consists of 200,000 action clips of 300 categories and with 33 pre-defined action attributes in two hierarchical levels (i.e., low-level attributes of basic body part movements and high-level attributes related to action motion). We learn DAP3D-Net using the NASA dataset and then evaluate it on our collected Human Action Understanding (HAU) dataset and the public THUMOS dataset. Experimental results show that our approach can accurately localize, categorize and describe multiple actions in realistic videos.

Deaf-mute communities around the world experience a need in effective human-robot interaction system that would act as an interpreter in public places such as banks, hospitals, or police stations. The focus of this work is to address the challenges presented to hearing-impaired people by developing an interpreting robotic system required for effective communication in public places. To this end, we utilize a previously developed neural network-based learning architecture to recognize Cyrillic manual alphabet, which is used for fingerspelling in Kazakhstan. In order to train and test the performance of the recognition system, we collected four datasets comprising of static and motion RGB and RGB-D data of 33 manual gestures. After applying them to standard machine learning algorithms as well as to our previously developed learning-based method, we achieved an average accuracy of 93% for a complete alphabet recognition by modeling motion depth data.

The presence of Type 2 singularities in parallel robots severely affects their performances, mainly because the platform motion control is partially lost. It also leads to a size reduction of the operational workspace. Moreover, the dynamic model of the parallel mechanism degenerates and locally, the robot becomes underactuated in the singularity. It has been proven that it is possible to cross Type 2 singularities by respecting a dynamic criterion. Nevertheless, the controllers designed up to now require a pre-planned optimized trajectory including this criterion, and as a result, this strategy can only be used by qualified users. In order to avoid this drawback and to cross these types of singularities even if the trajectory is not pre-planned, this paper proposes a controller based on virtual constraints. Furthermore, the controller is integrated in a multi-control architecture in order to switch between a classical computed torque control far from the singularity and the virtual-constraint-based control law near to the singularity locus. Experimental results on a five-bar mechanism validated the automatic Type 2 singularity crossing.

In this work, we propose a framework to deal with cross-modal visuo-tactile object recognition. By cross-modal visuo-tactile object recognition, we mean that the object recognition algorithm is trained only with visual data and is able to recognize objects leveraging only tactile perception. The proposed cross-modal framework is constituted by three main elements. The first is a unified representation of visual and tactile data, which is suitable for cross-modal perception. The second is a set of features able to encode the chosen representation for classification applications. The third is a supervised learning algorithm, which takes advantage of the chosen descriptor. In order to show the results of our approach, we performed experiments with 15 objects common in domestic and industrial environments. Moreover, we compare the performance of the proposed framework with the performance of 10 humans in a simple cross-modal recognition task.

We define a system architecture for a large swarm of miniature quadcopters flying in dense formation indoors. The large number of small vehicles motivates novel design choices for state estimation and communication. For state estimation, we develop a method to reliably track many small rigid bodies with identical motion-capture marker arrangements. Our communication infrastructure uses compressed one-way data flow and supports a large number of vehicles per radio. We achieve reliable flight with accurate tracking (<; 2 cm mean position error) by implementing the majority of computation onboard, including sensor fusion, control, and some trajectory planning. We provide various examples and empirically determine latency and tracking performance for swarms with up to 49 vehicles.

Building large-scale 2.5D maps when spatial correlations are considered can be quite expensive, but there are clear advantages when fusing data. While optimal submapping strategies have been explored previously in covariance-form using Gaussian Process for large-scale mapping, this paper focuses on transferring such concepts into information form. By exploiting the conditional independence property of the Gaussian Markov Random Field (GMRF) models, we propose a submapping approach to build a nearly optimal global 2.5D map. In the proposed approach data is fused by first fitting a GMRF to one sensor dataset; then conditional independent submaps are inferred using this model and updated individually with new data arrives. Finally, the information is propagated from submap to submap to later recover the fully updated map. This is efficiently achieved by exploiting the inherent structure of the GMRF, fusion and propagation all in information form. The key contribution of this paper is the derivation of the algorithm to optimally propagate information through submaps by only updating the common parts between submaps. Our results show the proposed method reduces the computational complexity of the full mapping process while maintaining the accuracy. The performance is evaluated on synthetic data from the Canadian Digital Elevation Data.

Object tracking is an important capability for robots tasked with interacting with humans and the environment, and it enables robots to manipulate objects. In object tracking, selecting samples to learn a robust and efficient appearance model is a challenging task. Model learning determines both the strategy and frequency of model updating, which concerns many details that can affect the tracking results. In this paper, we propose an object tracking approach by formulating a new objective function that integrates the learning paradigm of self-paced learning into object tracking such that reliable samples can be automatically selected for model learning. Sample weights and model parameters can be learned by minimizing this single objective function under the framework of kernelized correlation filters. Moreover, a real-valued error-tolerant self-paced function with a constraint vector is proposed to combine prior knowledge, i.e., the characteristics of object tracking, with information learned during tracking. We demonstrate the robustness and efficiency of our object tracking approach on a recent object tracking benchmark data set: OTB 2013.

Communication with a robot using brain activity from a human collaborator could provide a direct and fast feedback loop that is easy and natural for the human, thereby enabling a wide variety of intuitive interaction tasks. This paper explores the application of EEG-measured error-related potentials (ErrPs) to closed-loop robotic control. ErrP signals are particularly useful for robotics tasks because they are naturally occurring within the brain in response to an unexpected error. We decode ErrP signals from a human operator in real time to control a Rethink Robotics Baxter robot during a binary object selection task. We also show that utilizing a secondary interactive error-related potential signal generated during this closed-loop robot task can greatly improve classification performance, suggesting new ways in which robots can acquire human feedback. The design and implementation of the complete system is described, and results are presented for realtime closed-loop and open-loop experiments as well as offline analysis of both primary and secondary ErrP signals. These experiments are performed using general population subjects that have not been trained or screened. This work thereby demonstrates the potential for EEG-based feedback methods to facilitate seamless robotic control, and moves closer towards the goal of real-time intuitive interaction.

Current control techniques for optical manipulation of multiple micro-objects employ laser tweezers to directly trap and manipulate the target microscopic objects, and thus it is not capable of coordinating and manipulating micro-objects with arbitrary types in the micro-world, including large micro-objects, laser sensitive biological cells, or optically untrappable ones. In this paper, we propose a robotic control technique for optical manipulation of multiple microscopic objects by using micro-hands with multiple fingertips. In this control technique, several micro-hands are first formed by coordinating multiple optically trapped micro-particles that serve as the laser-driven fingertips, and then utilized for grasping and coordinative manipulation of multiple micro-objects. The proposed technique offers a robotic control framework for coordination and optical manipulation of multiple microscopic objects with arbitrary types in the micro-world, including large micro-objects, lasersensitive cells, or even untrappable microscopic objects.

Relative positioning between multiple mobile users is essential for many applications, such as search and rescue in disaster areas or human social interaction. Inertial-measurement unit (IMU) is promising to determine the change of position over short periods of time, but it is very sensitive to error accumulation over long term run. By equipping the mobile users with ranging unit, e.g. ultra-wideband (UWB), it is possible to achieve accurate relative positioning by trilateration-based approaches. As compared to vision or laser-based sensors, the UWB does not need to be with in line-of-sight and provides accurate distance estimation. However, UWB does not provide any bearing information and the communication range is limited, thus UWB alone cannot determine the user location without any ambiguity. In this paper, we propose an approach to combine IMU inertial and UWB ranging measurement for relative positioning between multiple mobile users without the knowledge of the infrastructure. We incorporate the UWB and the IMU measurement into a probabilistic-based framework, which allows to cooperatively position a group of mobile users and recover from positioning failures. We have conducted extensive experiments to demonstrate the benefits of incorporating IMU inertial and UWB ranging measurements.

In this paper we address the problem of multi-robot localization with a heterogeneous team of low-cost mobile robots. The team consists of a single centralized observer with an inertial measurement unit (IMU) and monocular camera, and multiple picket robots with only IMUs and Red Green Blue (RGB) light emitting diodes (LED). This team cooperatively navigates a visually featureless environment while localizing all robots. A combination of camera imagery captured by the observer and IMU measurements from the pickets and observer are fused to estimate motion of the team. A team movement strategy, referred to as inchworm, is formulated as follows: Pickets move ahead of the observer and then act as temporary landmarks for the observer to follow. This cooperative approach employs a single Extended Kalman Filter (EKF) to localize the entire heterogeneous multi-robot team, using a formulation of the measurement Jacobian to relate the pose of the observer to the poses of the pickets with respect to the global reference frame. An initial experiment with the inchworm strategy has shown localization within 0.14 m position error and 2.18° orientation error over a path-length of 5 meters in an environment with irregular ground, partial occlusions, and a ramp. This demonstrates improvement over a camera-only localization technique that was adapted to our team dynamic which produced 0.18m position error and 3.12° orientation error over the same dataset. In addition, we demonstrate improvement in localization accuracy with an increasing number of picket robots.

Breast cancer is the most frequently life-threatening diagnosed type of cancer among women. Early and accurate diagnosis by acquiring a tissue sample using biopsy techniques is essential. However, small lesions only visible by MRI are often missed in standard methods, indicating the need for a robotic-assisted biopsy system that is MRI-compatible. Existing proof-of-concepts are difficult to employ due to large sizes and/or actuation complexities. Therefore, a compact pneumatically-actuated 5 DOF MRI-compatible robot was further developed and controlled by a computerized valve manifold. Accuracy and efficiency measurements have been performed using two different PVC breast phantoms with embedded fish oil capsules (mimicking lesions) inside a 0.25T MRI scanner. Preliminary results show that the end-effector was able to hit the targeted capsules, and that the position accuracy is in the range of 4.7-7.3 mm. The developed robotic system has potential to perform MRI-guided breast biopsies accurately and improve the clinical workflow.

This paper proposes an approximate-kinetic-model-based self-adaptive (AKS) control system to rapidly generate target walking speed by an underactuated compasslike bipedal walker. First, a model of the underactuated compass-like bipedal walker is built, and an open-loop system is introduced and analysed as the prototype of AKS system. Second, the control law of AKS is described in detail. The dynamic updating of trajectory is proposed and the calculation of control parameters by an approximate linearized model is analysed. Finally, simulations are conducted, and thus the capability of disturbance rejection and versatility is tested. As a conclusion, target walking speeds with 4∼6% steady-state error can be generated rapidly and steadily. Limit cycle walker can obviously improve the capability of handling disturbance and various tasks by AKS control system.

Standard hexarotors are often mistakenly considered `by definition' fail-safe multi-rotor platforms because of the two additional propellers when compared to quadrotors. However this is not true, in fact, a standard hexarotor cannot statically hover with `only' five propellers. In this paper we provide a set of new general algebraic conditions to ensure static hover for any multi-rotor platform with any number of generically oriented rotors. These are elegantly formulated as the full-rankness of the control moment input matrix, and the non-orthogonality between its null-space and the row space of the control force input matrix. Input saturations and safety margins are also taken into account with an additional condition on the null-space of control moment input matrix. A deep analysis on the hoverability properties is then carried out focusing on the propeller loss in a hexarotor platform. Leveraging our general results we explain why a standard hexarotor is not robust and how it can be made robust thanks to a particular tilt of the rotors. We finally propose a novel cascaded controller based on a preferential direction in the null-space of the control moment input matrix for the large class of statically hoverable multi-rotors, which goes far beyond standard platforms, and we apply this controller to the case of failed tilted hexarotor.