IROS 2017 Papers

We present a robust multi-robot convoying approach that relies on visual detection of the leading agent, thus enabling target following in unstructured 3-D environments. Our method is based on the idea of tracking-by-detection, which interleaves efficient model-based object detection with temporal filtering of image-based bounding box estimation. This approach has the important advantage of mitigating tracking drift (i.e. drifting away from the target object), which is a common symptom of model-free trackers and is detrimental to sustained convoying in practice. To illustrate our solution, we collected extensive footage of an underwater robot in ocean settings, and hand-annotated its location in each frame. Based on this dataset, we present an empirical comparison of multiple tracker variants, including the use of several convolutional neural networks, both with and without recurrent connections, as well as frequency-based model-free trackers. We also demonstrate the practicality of this tracking-by-detection strategy in real-world scenarios by successfully controlling a legged underwater robot in five degrees of freedom to follow another robot's independent motion.

It is important to register oceanographic data into a geo-referenced coordinate system. Knowing the location of the sampling is critical. For a marine robotic network, the location of Unmanned Surface Vessels (USVs) can be measured using a Global Positioning System (GPS), however the navigation of Autonomous Underwater Vehicles (AUVs) is more challenging. In this paper, we present a method for determining the position of underwater vehicles from a moving USV using the relative range information provided by an Ultra-Short Baseline (USBL)/acoustic modem. The navigation method uses an Extended Kalman Filter (EKF) to update the states predicted from a model-based dead-reckoning technique. Since the vehicle model is relative to the surrounding fluid, we have introduced two environmental states in the state matrix. Such a modification allows us to quantify the effects induced by the ocean current on the vehicle's speed. Beyond that, the method uses a limited number of sensors, only attitude sensors and an USBL/acoustic modem, offering an alternative for AUVs without expensive instruments such as a Doppler Velocity Log (DVL) and an Inertial Measurement Unit (IMU). Experiments are conducted to evaluate the range-based navigation method on a hybrid Slocum underwater glider with an USV. As a result from the reference trial, the estimated glider position stays within the error of 15 meters comparing to the measured position of a surface buoy where the glider is attached. In the open-water trial, trajectories estimated from the range-based navigation are compared with dead-reckoning paths and current-compensated dead-reckoning paths (reference). As a result, the distance errors are bounded with the proposed navigation method while the dead-reckoning errors grow without bound.

SONAR mapping of underwater environments leads to dense point-clouds. These maps have large memory footprints, are inherently noisy and consist of raw data with no semantic information. This paper presents an approach to underwater semantic mapping where known man-made structures that appear in multibeam SONAR data are automatically recognised. From a set of SONAR images acquired by an Autonomous Underwater Vehicle (AUV) and a catalogue of `a-priori' 3D CAD models of structures that may potentially be found in the data, our algorithm proceeds in two phases. First we recognise objects using an efficient, rotation-invariant 2D descriptor combined with a histogram matching method. Then, we determine pose using a 6 degree-of-freedom registration of the 3D object to the local scene using a fast 2D correlation, refined with an iterative closest point (ICP)-based method. Once the structures located and identified, we build a semantic representation of the world based on the initial CAD models, resulting in a lightweight yet accurate world model. We demonstrate the applicability of our method on field data acquired by an AUV in Loch Linnhe, Scotland. Our method proves to be suitable for online semantic mapping of a partially man-made underwater environment such as a typical oil field.

This paper proposes an ultra-wideband (UWB) aided localization and mapping system that leverages on inertial sensor and depth camera. Inspired by the fact that visual odometry (VO) system, regardless of its accuracy in the short term, still faces challenges with accumulated errors in the long run or under unfavourable environments, the UWB ranging measurements are fused to remove the visual drift and improve the robustness. A general framework is developed which consists of three parallel threads, two of which carry out the visualinertial odometry (VIO) and UWB localization respectively. The other mapping thread integrates visual tracking constraints into a pose graph with the proposed smooth and virtual range constraints, such that a bundle adjustment is performed to provide robust trajectory estimation. Experiments show that the proposed system is able to create dense drift-free maps in real-time even running on an ultra-low power processor in featureless environments.

This research teams an Unmanned Surface Vehicle (USV) with an Unmanned Aerial Vehicle (UAV) to augment and automate marine mass casualty incident search and rescue in emergency response phase. The demand for real-time responsiveness of those missions requires fast and comprehensive situational awareness and precise operations, which are challenging to achieve because of the large area and the flat nature of the water field. The responders, drowning victims, and rescue vehicle are far apart and all located at the sea level. The long distances mean responders cannot clearly discern the rescue vehicle and victims from the surrounding water. Furthermore, being at the same elevation makes depth perception difficult. Rescue vehicle and victims at different distances from the responder will always appear to be close together. This makes it almost impossible for the responders to accurately drive the USV to the victims in time. This paper proposes the use of a UAV to compensate for the lack of elevation of the responders and to automate search and rescue operations. The benefit of this system is two fold: 1) the UAV provides responders with an overhead view of the field, covers larger area than direct visual, and allows more accurate perception of the situation, and 2) it automates the rescue process so that the responders can focus on task-level needs instead of tediously driving the USV to the victims. Thirty autonomous navigation trials in 4 rescue scenarios prove the first known successful implementation of a small UAV visually navigating a USV.

The recognition of actions from video sequences has many applications in health monitoring, assisted living, surveillance, and smart homes. Despite advances in sensing, in particular related to 3D video, the methodologies to process the data are still subject to research. We demonstrate superior results by a system which combines recurrent neural networks with convolutional neural networks in a voting approach. The gated-recurrent-unit-based neural networks are particularly well-suited to distinguish actions based on long-term information from optical tracking data; the 3D-CNNs focus more on detailed, recent information from video data. The resulting features are merged in an SVM which then classifies the movement. In this architecture, our method improves recognition rates of state-of-the-art methods by 14% on standard data sets.

Soft robots with inherent compliance have been recently investigated intensively for locomotion or manipulations. A critical problem for soft robots is the capability to estimate their shapes to enable closed-loop control for precise motion. In this paper, we propose a new low-cost sensor that can be leveraged for shape estimation of soft robots. This sensor, recently discovered as an artificial muscle, can be conveniently fabricated from low-cost conductive sewing threads. We recently found that the resistance will increase if the fabricated sensor is elongated due to an external force [1]. Since the sensor is inherently soft, it can be embedded into soft robots to estimate the shape. We establish a physics-based model to predict the external force and the displacement if the resistance is given and experimentally validate its correctness. Moreover, to demonstrate the sensing capability, we embed the proposed sensor into soft materials and successfully measure two curvatures of a two-segment soft robot. Therefore, the proposed sensor has the potential to estimate complicated shapes of soft robots to enable closed-loop control.

Multi-robot teleoperation systems benefit from human's capabilities in adaptation to new environments and decision-making in the presence of uncertainty and complexity. In these applications, usually a human operator determines the general behaviour of the multi-robot team via selecting and controlling a leader robot while the follower robots coordinate with the team autonomously. Trust is a major factor in human's allocation of autonomy among the team of robots. A trust-based online leader selection strategy is proposed in this work for multi-robot bilateral haptic teleoperation with applications in collective position tracking and synchronization. Human-to-robot trust is computed and utilized as a dynamic criterion to select the leader. Also, robot-to-human trust is used to dynamically scale the haptic force feedback cues so that the operator receives smaller force feedback when his/her performance in leading the team is higher. This reduces the physical workload of the operator. Due to the trust-based leader switching and force feedback scaling, the stability of the closed-loop system may be lost. We guarantee this stability via passivity-based techniques. Finally, our experimental results indicate that the trust-based leader selection strategy can reduce task completion time by 35.25% and formation error by 41.64% compared to a non-leader switching strategy.

There are several challenges in down-sizing robots for transportation deployment, diversification of locomotion capabilities tuned for various terrains, and rapid and on-demand manufacturing. In this paper we propose an origami-inspired method of addressing these key issues by designing and manufacturing a foldable, deployable, and self-righting version of the origami robot Tribot. Our latest Tribot prototype can jump as high as 215 mm, five times its height, and roll consecutively on any of its edges with an average step size of 55 mm. The 4 g robot self-deploys nine times of its size when released. A compliant roll cage ensures that the robot self-rights onto two legs after jumping or being deployed and also protects the robot from impacts. A description of our prototype and its design, locomotion modes, and fabrication is followed by demonstrations of its key features.

The auto-encoder method is a type of dimensionality reduction method. A mapping from a vector to a descriptor that represents essential information can be automatically generated from a set of vectors without any supervising information. However, an image and its spatially shifted version are encoded into different descriptors by an existing ordinary auto-encoder because each descriptor includes a spatial subpattern and its position. To generate a descriptor representing a spatial subpattern in an image, we need to normalize its spatial position in the images prior to training an ordinary auto-encoder; however, such a normalization is generally difficult for images without obvious standard positions. We propose a transform invariant auto-encoder and an inference model of transform parameters. By the proposed method, we can separate an input into a transform invariant descriptor and transform parameters. The proposed method can be applied to various auto-encoders without requiring any special modules or labeled training samples. By applying it to shift transforms, we can achieve a shift invariant auto-encoder that can extract a typical spatial subpattern independent of its relative position in a window. In addition, we can achieve a model that can infer shift parameters required to restore the input from the typical subpattern. As an example of the proposed method, we demonstrate that a descriptor generated by a shift invariant auto-encoder can represent a typical spatial subpattern. In addition, we demonstrate the imitation of a human hand by a robot hand as an example of a regression based on spatial subpatterns.

Locomotion at low Reynolds numbers is a topic of growing interest, spurred by its various engineering and medical applications. This paper presents a novel prototype and a locomotion algorithm for the 3-link planar Purcell's swimmer based on Lie algebraic notions. The kinematic model, based on Cox theory of the prototype swimmer is a driftless control-affine system. Using the existing strong controllability and related results, the existence of motion primitives is initially shown. The Lie algebra of the control vector fields is then used to synthesize control profiles to generate motions along the basis of the Lie algebra associated with the structure group of the system. An open loop control system with vision-based positioning is successfully implemented which allows tracking any given continuous trajectory of the position and orientation of the swimmer's base link. Alongside, the paper also provides a theoretical interpretation of the symmetry arguments presented in the existing literature to generate the control profiles of the swimmer.

Various control strategies using trajectory planning, object recognition and learning are researched for avoiding obstacles when legged robot is walking. In this paper, designed trajectory by using Non Uniform Basis Spline (NUBS) curve and control strategy, by using proposed trajectory, to effectively overcome obstacles are presented. The trajectory designed by NUBS curve has several advantages: 1) local modification, 2) tracking velocity control for each domain, and 3) low degree trajectory with a large number of control points. The robot gets remarkable effectiveness when these advantages are used to generate the trajectory for walking and overcoming obstacles. By implementation of the proposed control strategy, quadruped robot can walk over obstacles while keeping its gait, speed and balance without collision although adjusting relatively preferable state which is more suitable position or posture of the walking robot shortly before encountering obstacles is not required. The proposed trajectory and control strategy are discussed, and performance is validated through experimental evaluations.

This paper presents a vision-based control strategy for tracking a ground target using a novel vision sensor featuring a processor for each pixel element. This enables computer vision tasks to be carried out directly on the focal plane in a highly efficient manner rather than using a separate general purpose computer. The strategy enables a small, agile quadrotor Unmanned Air Vehicle (UAV) to track the target from close range using minimal computational effort and with low power consumption. To evaluate the system we target a vehicle driven by chaotic dual-pendulum trajectories. Target proximity and the large, unpredictable accelerations of the vehicle cause challenges for the UAV in keeping it within the downward facing camera's field of view (FoV). A state observer is used to smooth out predictions of the target's location and, importantly, estimate velocity. Experimental results also demonstrate that it is possible to continue to re-acquire and follow the target during short periods of loss in target visibility. The tracking algorithm exploits the parallel nature of the visual sensor, enabling high rate image processing ahead of any communication bottleneck with the UAV controller. With the vision chip carrying out the most intense visual information processing, it is computationally trivial to compute all of the controls for tracking onboard. This work is directed toward visual agile robots that are power efficient and that ferry only useful data around the information and control pathways.

Multi-person tracking with a robotic platform is one of the cornerstones of human-robot interaction. Challenges arise from occlusions, appearance changes and a time-varying number of people. Furthermore, the final system is constrained by the hardware platform: low computational capacity and limited field-of-view. In this paper, we propose a novel method to simultaneously track a time-varying number of persons in three-dimensions and perform visual servoing. The complementary nature of the tracking and visual servoing enables the system to: (i) track several persons while compensating for large ego-movements and (ii) visually control the robot to keep a selected person of interest within the field of view. We propose a variational Bayesian formulation allowing us to effectively solve the inference problem through the use of closed-form solutions. Importantly, this leads to a computationally efficient procedure that runs at 10 FPS. The experiments on the NAO-MPVS dataset confirm the importance of using visual servoing for tracking multiple persons.

Many model-based Visual Odometry (VO) algorithms have been proposed in the past decade, often restricted to the type of camera optics, or the underlying motion manifold observed. We envision robots to be able to learn and perform these tasks, in a minimally supervised setting, as they gain more experience. To this end, we propose a fully trainable solution to visual ego-motion estimation for varied camera optics. We propose a visual ego-motion learning architecture that maps observed optical flow vectors to an ego-motion density estimate via a Mixture Density Network (MDN). By modeling the architecture as a Conditional Variational Autoencoder (C-VAE), our model is able to provide introspective reasoning and prediction for ego-motion induced scene-flow. Additionally, our proposed model is especially amenable to bootstrapped ego-motion learning in robots where the supervision in ego-motion estimation for a particular camera sensor can be obtained from standard navigation-based sensor fusion strategies (GPS/INS and wheel-odometry fusion). Through experiments, we show the utility of our proposed approach in enabling the concept of self-supervised learning for visual ego-motion estimation in autonomous robots.

We address the challenge of computing search paths in real-time for subsea applications where the goal is to locate an unknown number of targets on the seafloor. Our approach maximizes a formal definition of search effectiveness given finite search effort. We account for false positive measurements and variation in the performance of the search sensor due to geographic variation of the seafloor. We compare near-optimal search paths that can be computed in real-time with optimal search paths for which real-time computation is infeasible. We show how sonar data acquired for locating targets at a specific location can also be used to characterize the performance of the search sonar at that location. Our approach is illustrated with numerical experiments where search paths are planned using sonar data previously acquired from Boston Harbor.

Large robotic collectives provide advantages such as resilience to mechanical failure of single agents and increased capabilities for search and coverage based applications. However, a lack of rapid and free-form manufacturing processes remains a barrier to high-volume fabrication of mechanically heterogeneous robotic swarms. Self-folding laser-machined structures have the potential to enable heterogeneous robotic swarms. As an initial step to realizing a functional robotic collective, we focus on the design and characterization of the locomotion of an individual laminate manufactured robot. We look to a vibration-based locomotion technique that uses flexible structures or bristles to enhance the effects of vibration, allowing for fast locomotion (i.e. a “bristle-bot”). However, previous bristle-bot implementations have not allowed for controllable steering behaviors with high locomotion speeds. We describe the extension of existing two dimensional bristle-bot models to a three dimensional model that explores parameters that govern linear and angular velocity. We implement an autonomous laminate-manufactured bristle-bot inspired robot capable of linear velocities of up to 23 cm/s and turning rates of 2 rad/s. Moving towards automated manufacturing, we also demonstrate a self-folding bristle-bot structure that uses a linear compression laminate to achieve a uniform leg fold angle.

We present a multi-query recovery policy for a hybrid system with goal limit cycle. The sample trajectories and the hybrid limit cycle of the dynamical system are stabilized using locally valid Time Varying LQR controller policies which probabilistically cover a bounded region of state space. The original LQR Tree algorithm builds such trees for non-linear static and non-hybrid systems like a pendulum or a cart-pole. We leverage the idea of LQR trees to plan with a continuous control set, unlike methods that rely on discretization like dynamic programming to plan for hybrid dynamical systems where it is hard to capture the exact event of discrete transition. We test the algorithm on a compass gait model by stabilizing a dynamic walking hybrid limit cycle with point foot contact from random initial conditions. We show results from the simulation where the system comes back to a stable behavior with initial position or velocity perturbation and noise.

In this paper, we propose to take advantage of concentric tube robots (CTR) properties to design a robotic system for optical biopsies with fiber-based imaging modalities. Follow-the-leader (FTL) deployment, with CTR body following its tip, is being integrated as a design constraint in order to minimize the device invasiveness. A design procedure is proposed and developed in the context of olfactory cell inspection. Stability issues of CTR are considered with theoretical validation of FTL deployment. Finally, an experimental assessment in lab conditions is being conducted that includes the device deployment and the reproduction of scanning task for imaging. The whole surface to be inspected can be accessed, and 20 μm resolution of the CTR tip is observed.