ICRA 2021 Papers

In colonoscopy procedures, it is important to rebuild and visualize the colonic surface to minimize the missing regions and reinspect for abnormalities. Due to the fast camera motion and deformation of the colon in standard forward-viewing colonoscopies, traditional simultaneous localization and mapping (SLAM) systems work poorly for 3D reconstruction of colon surfaces and are prone to severe drift. Thus in this paper, a preoperative colon model segmented from CT scans is used together with the colonoscopic images to achieve the 3D colon reconstruction. The proposed framework includes dense depth estimation from monocular colonoscopic images using a deep neural network (DNN), visual odometry (VO) based camera motion estimation and an embedded deformation (ED) graph based non-rigid registration algorithm for deforming 3D scans to the segmented colon model. A realistic simulator is used to generate different simulation datasets with ground truth. Simulation results demonstrate the good performance of the proposed 3D colonic surface reconstruction method in terms of accuracy and robustness. In-vivo experiments are also conducted and the results show the practicality of the proposed framework for providing useful shape and texture information in colonoscopy applications.

To date, untethered microrobots have been receiving tremendous attention for playing implacable roles of maneuverable tools in fields such as microfabrication and biomanipulation. Typical actuation of such untethered tiny robots is the magnetic field-based approaches, including gradient and rotational methods. Compared to the gradient type method, the rotational approach requires much less magnetic field strength to generate efficient actuation for magnetic microrobots. To actuate microrobots desirably, a precise periodic magnetic field should be provided. To generate precise periodic magnetic field with enhanced strength, this paper develops a prototype of 3D magnetic servoing system based on integrated solenoids, performance of which are enhanced by employing iron cores and extended number of coils. Each solenoid is equipped with a Hall sensor to provide real-time feedback signal for performing precise magnetic field control. To precisely regulate this setup, a decoupled asynchronous repetitive control (DARC) scheme is established to generate a desirable 3D periodic magnetic field with noise-level tracking error under the situation of missing execution opportunity randomly. Experimental results demonstrate the effectiveness of the proposed magnetic servoing system, which is promising for dynamic properties characterization of magnetic microrobots.

Multiple object tracking (MOT) is a critical module for enabling autonomous vehicles to achieve safe planing and navigation in cluttered environments. In tracking-by-detection systems, there are inevitably many false positives and misses among learning-based input detections. The challenge for MOT is to combine these detections into tracks, and filter them based on their uncertainties, states, and temporal consistency to achieve accurate and persistent tracks. In this paper, we propose to solve the 3D MOT problem for autonomous driving applications using a random finite set-based (RFS) Multiple Measurement Models filter (RFS-M3). In partiuclar, we propose multiple measurement models for a Poisson multi-Bernoulli mixture (PMBM) filter in support of different application scenarios. Our RFS-M3 filter can naturally model these uncertainties accurately and elegantly. We combine the learning-based detections with our RFS-M3 tracker through incorporating the detection confidence score into the PMBM prediction and update step. The superior experimental results of our RFS-M3 tracker on Waymo, Argoverse and nuSceness datasets illustrate that our RFS-M3 tracker outperforms state-of-the-art deep learning-based and traditional filter-based approaches. To the best of our knowledge, this represents a first successful attempt for employing an RFS-based approach in conjunction with 3D learning-based amodal detections for 3D MOT applications with comprehensive validation using challenging datasets made available by industry leaders.

Accurate motion capture of aerial robots in 3D is a key enabler for autonomous operation in indoor environments such as warehouses or factories, as well as driving forward research in these areas. The most commonly used solutions at present are optical motion capture (e.g. VICON) and Ultrawide-band (UWB), but these are costly and cumbersome to deploy, due to their requirement of multiple cameras/anchors spaced around the tracking area. They also require the drone to be modified to carry an active or passive marker. In this work, we present an inexpensive system that can be rapidly installed, based on single-chip millimeter wave (mmWave) radar. Importantly, the drone does not need to be modified or equipped with any markers, as we exploit the Doppler signals from the rotating propellers. Furthermore, 3D tracking is possible from a single point, greatly simplifying deployment. We develop a novel deep neural network and demonstrate decimeter level 3D tracking at 10Hz, achieving better performance than classical baselines. Our hope is that this low-cost system will act to catalyse inexpensive drone research and increased autonomy.

The need to guarantee safety of collaborative robots limits their performance, in particular, their speed and hence cycle time. The standard ISO/TS 15066 defines the Power and Force Limiting operation mode and prescribes force thresholds that a moving robot is allowed to exert on human body parts during impact, along with a simple formula to obtain maximum allowed speed of the robot in the whole workspace. In this work, we measure the forces exerted by two collaborative manipulators (UR10e and KUKA LBR iiwa) moving downward against an impact measuring device. First, we empirically show that the impact forces can vary by more than 100 percent within the robot workspace. The forces are negatively correlated with the distance from the robot base and the height in the workspace. Second, we present a data-driven model, 3D Collision-Force-Map, predicting impact forces from distance, height, and velocity and demonstrate that it can be trained on a limited number of data points. Third, we analyze the force evolution upon impact and find that clamping never occurs for the UR10e. We show that formulas relating robot mass, velocity, and impact forces from ISO/TS 15066 are insufficient—leading both to significant underestimation and overestimation and thus to unnecessarily long cycle times or even dangerous applications. We propose an empirical method that can be deployed to quickly determine the optimal speed and position where a task can be safely performed with maximum efficiency.

Segmentation of moving objects in dynamic scenes is a key process in scene understanding for navigation tasks. Classical cameras suffer from motion blur in such scenarios rendering them effete. On the contrary, event cameras, because of their high temporal resolution and lack of motion blur, are tailor-made for this problem. We present an approach for monocular multi-motion segmentation, which combines bottom-up feature tracking and top-down motion compensation into a unified pipeline, which is the first of its kind to our knowledge. Using the events within a time-interval, our method segments the scene into multiple motions by splitting and merging. We further speed up our method by using the concept of motion propagation and cluster keyslices.The approach was successfully evaluated on both challenging real-world and synthetic scenarios from the EV-IMO, EED, and MOD datasets and outperformed the state-of-the-art detection rate by 12%, achieving a new state-of-the-art average detection rate of 81.06%, 94.2% and 82.35% on the aforementioned datasets. To enable further research and systematic evaluation of multi-motion segmentation, we present and open-source a new dataset/benchmark called MOD++, which includes challenging sequences and extensive data stratification in-terms of camera and object motion, velocity magnitudes, direction, and rotational speeds.