MobiCom 2021 论文列表

Over the last decade, facial landmark tracking and 3D reconstruction have gained considerable attention due to their numerous applications such as human-computer interactions, facial expression analysis, and emotion recognition, etc. Traditional approaches require users to be confined to a particular location and face a camera under constrained recording conditions (e.g., without occlusions and under good lighting conditions). This highly restricted setting prevents them from being deployed in many application scenarios involving human motions. In this paper, we propose the first single-earpiece lightweight biosensing system, BioFace-3D, that can unobtrusively, continuously, and reliably sense the entire facial movements, track 2D facial landmarks, and further render 3D facial animations. Our single-earpiece biosensing system takes advantage of the cross-modal transfer learning model to transfer the knowledge embodied in a high-grade visual facial landmark detection model to the low-grade biosignal domain. After training, our BioFace-3D can directly perform continuous 3D facial reconstruction from the biosignals, without any visual input. Without requiring a camera positioned in front of the user, this paradigm shift from visual sensing to biosensing would introduce new opportunities in many emerging mobile and IoT applications. Extensive experiments involving 16 participants under various settings demonstrate that BioFace-3D can accurately track 53 major facial landmarks with only 1.85 mm average error and 3.38% normalized mean error, which is comparable with most state-of-the-art camera-based solutions. The rendered 3D facial animations, which are in consistency with the real human facial movements, also validate the system's capability in continuous 3D facial reconstruction.

The SIM/eSIM card stores critical information for a mobile user to access the 4G/5G network. In this work, we uncover three vulnerabilities of the current SIM practice. We show that the PIN-based access control may expose the in-SIM data to an adversary through both hardware and software. Once exposed, such in-SIM information can be used to reconstruct various keys used for device authentication, data encryption, etc. They thus enable a number of attacks, including traffic eavesdropping, man-in-the-middle attack, impersonation, etc. The fundamental problem is that, the current SIM design does not offer proper authentication and fine-grained access control to hundreds of in-SIM files for various in-card applets and off-card units. We next propose a new solution that offers both authentication and fine-grained access control. Our implementation and evaluation have confirmed the viability of our proposal.

Augmented reality/virtual reality (AR/VR) has extended beyond 3D immersive gaming to a broader array of applications, such as shopping, tourism, education. And recently there has been a large shift from handheld-controller dominated interactions to headset-dominated interactions via voice interfaces. In this work, we show a serious privacy risk of using voice interfaces while the user is wearing the face-mounted AR/VR devices. Specifically, we design an eavesdropping attack, Face-Mic, which leverages speech-associated subtle facial dynamics captured by zero-permission motion sensors in AR/VR headsets to infer highly sensitive information from live human speech, including speaker gender, identity, and speech content. Face-Mic is grounded on a key insight that AR/VR headsets are closely mounted on the user's face, allowing a potentially malicious app on the headset to capture underlying facial dynamics as the wearer speaks, including movements of facial muscles and bone-borne vibrations, which encode private biometrics and speech characteristics. To mitigate the impacts of body movements, we develop a signal source separation technique to identify and separate the speech-associated facial dynamics from other types of body movements. We further extract representative features with respect to the two types of facial dynamics. We successfully demonstrate the privacy leakage through AR/VR headsets by deriving the user's gender/identity and extracting speech information via the development of a deep learning-based framework. Extensive experiments using four mainstream VR headsets validate the generalizability, effectiveness, and high accuracy of Face-Mic.

Similar to satellite-based localization systems, messages sent by aircraft with the ADS-B protocol can be used to estimate the location of a mobile receiver. However, for a robust localization using a least-squares approach, ADS-B messages have to be collected over a long time. We propose a localization method based on matching the received signal with known ADS-B messages from distributed receivers. Our proposed method only requires three seconds of recording. Compared to satellite-based localization methods, this approach also works indoors as the signals sent by aircraft are much stronger.

Light bulbs have been recently explored to design Light Fidelity (LiFi) communication to battery-free tags, thus complementing Radiofrequency (RF) backscatter in the uplink. In this paper, we show that LiFi and RF backscatter are complementary and have unexplored interactions. We introduce PassiveLiFi, a battery-free system that uses LiFi to transmit RF backscatter at a meagre power budget. We address several challenges on the system design in the LiFi transmitter, the tag and the RF receiver. We design the first LiFi transmitter that implements a chirp spread spectrum (CSS) using the visible light spectrum. We use a small bank of solar cells for communication and harvesting and reconfigure them based on the amount of harvested energy and desired data rate. We further alleviate the low responsiveness of solar cells with a new low-power receiver design in the tag. Experimental results with an RF carrier of 17 dBm show that we can generate RF backscatter with a range of 80.3 meters/μW consumed in the tag, which is almost double with respect to prior work.

A long-standing challenge in radios for wearables is to design ultra-low power, yet high performance receivers with good sensitivity and spectral efficiency while being compatible with WiFi. The vanilla envelope detector used in standard UHF RFID is the most popular receivers on backscatter tags since they are passive but suffer from poor sensitivity and cannot decode complex modulations, which makes them a poor choice for directly decoding data from WiFi packets.

On-device deep learning (DL) inference has attracted vast interest. Mobile CPUs are the most common hardware for on-device inference and many inference frameworks have been developed for them. Yet, due to the hardware complexity, DL inference on mobile CPUs suffers from two common issues: the poor performance scalability on the asymmetric multiprocessor, and energy inefficiency.

Infrastructure-assisted autonomous driving is an emerging paradigm that aims to make affordable autonomous vehicles a reality. A key technology for realizing this vision is real-time point cloud registration which allows a vehicle to fuse the 3D point clouds generated by its own LiDAR and those on roadside infrastructures such as smart lampposts, which can deliver increased sensing range, more robust object detection, and centimeter-level navigation. Unfortunately, the existing methods for point cloud registration assume two clouds to share a similar perspective and large overlap, which result in significant delay and inaccuracy in real-world infrastructure-assisted driving settings. This paper proposes VI-Eye - the first system that can align vehicle-infrastructure point clouds at centimeter accuracy in real-time. Our key idea is to exploit traffic domain knowledge by detecting a set of key semantic objects including road, lane lines, curbs, and traffic signs. Based on the inherent regular geometries of such semantic objects, VI-Eye extracts a small number of saliency points and leverage them to achieve real-time registration of two point clouds. By allowing vehicles and infrastructures to extract the semantic information in parallel, VI-Eye leads to a highly scalable architecture for infrastructure-assisted autonomous driving. To evaluate the performance of VI-Eye, we collect two new multiview LiDAR point cloud datasets on an indoor autonomous driving testbed and a campus smart lamppost testbed, respectively. They contain total 915 point cloud pairs and cover three roads of 1.12km. Experiment results show that VI-Eye achieves centimeter-level accuracy within around 0.2s, and delivers a 5X improvement in accuracy and 2X speedup over state-of-the-art baselines.

Overcoming the conventional trade-off between throughput and bit error rate (BER) performance, versus computational complexity is a long-term challenge for uplink Multiple-Input Multiple-Output (MIMO) detection in base station design for the cellular 5G New Radio roadmap, as well as in next generation wireless local area networks. In this work, we present ParaMax, a MIMO detector architecture that for the first time brings to bear physics-inspired parallel tempering algorithmic techniques [28, 50, 67] on this class of problems. ParaMax can achieve near optimal maximum-likelihood (ML) throughput performance in the Large MIMO regime, Massive MIMO systems where the base station has additional RF chains, to approach the number of base station antennas, in order to support even more parallel spatial streams. ParaMax is able to achieve a near ML-BER performance up to 160 × 160 and 80 × 80 Large MIMO for low-order modulations such as BPSK and QPSK, respectively, only requiring less than tens of processing elements. With respect to Massive MIMO systems, in 12 × 24 MIMO with 16-QAM at SNR 16 dB, ParaMax achieves 330 Mbits/s near-optimal system throughput with 4--8 processing elements per subcarrier, which is approximately 1.4× throughput than linear detector-based Massive MIMO systems.

Robust speech enhancement is considered as the holy grail of audio processing and a key requirement for human-human and human-machine interaction. Solving this task with single-channel, audio-only methods remains an open challenge, especially for practical scenarios involving a mixture of competing speakers and background noise. In this paper, we propose UltraSE, which uses ultrasound sensing as a complementary modality to separate the desired speaker's voice from interferences and noise. UltraSE uses a commodity mobile device (e.g., smartphone) to emit ultrasound and capture the reflections from the speaker's articulatory gestures. It introduces a multi-modal, multi-domain deep learning framework to fuse the ultrasonic Doppler features and the audible speech spectrogram. Furthermore, it employs an adversarially trained discriminator, based on a cross-modal similarity measurement network, to learn the correlation between the two heterogeneous feature modalities. Our experiments verify that UltraSE simultaneously improves speech intelligibility and quality, and outperforms state-of-the-art solutions by a large margin.

In this paper, we report on our 6-year experience developing Eagle Tester (eTester for short) - a mobile battery drain testing and diagnostic tool. We show how eTester evolved from an "academic" prototype to a fully automated tool usable by the mobile industry.

In industrialized wine production, brewing and aging are two key steps. These two processes require the liquors to be bottled for a long time, sometimes more than ten years. The liquor is vulnerable and highly susceptible to microbial contamination during storage, causing undetectable deterioration. During the production process, wineries control the indoor temperature and carbon dioxide concentration to slow down other microorganisms' reproduction speed. These methods, however, do not prevent pathogenic microorganism growth. Currently, microbial culture methods are not suitable for real-time liquor quality monitoring in wineries. Therefore, we have designed a lightweight monitoring system called Lili, which uses light signals to monitor real-time liquor quality changes. Lili detects the changes in surface tension and absorption spectrum caused by microbial metabolites and growth during deterioration. Lili employs eight LEDs and one photodiode to achieve fine-grained surface tension and absorption spectrum measurements. By analyzing these changes, Lili realizes real-time quality monitoring. In this paper, the characteristic offset degree measurement and the absorption spectrum dimension expansion are two critical technologies. In addition, we implemented countermeasures against ambient light noise and sloshing interference. Lili's surface tension and absorption spectrum measurement errors are only 0.89 mN/m and 2.4%, respectively, making it useful to identify the contamination duration, microorganism content and microorganism composition. These two data points can be used to determine potential issues with liquor quality when the liquor becomes health-threatening or even just contaminated, with an accuracy of 97.5%.

As mobile devices continuously generate streams of images and videos, a new class of mobile deep vision applications are rapidly emerging, which usually involve running deep neural networks on these multimedia data in real-time. To support such applications, having mobile devices offload the computation, especially the neural network inference, to edge clouds has proved effective. Existing solutions often assume there exists a dedicated and powerful server, to which the entire inference can be offloaded. In reality, however, we may not be able to find such a server but need to make do with less powerful ones. To address these more practical situations, we propose to partition the video frame and offload the partial inference tasks to multiple servers for parallel processing. This paper presents the design of Elf, a framework to accelerate the mobile deep vision applications with any server provisioning through the parallel offloading. Elf employs a recurrent region proposal prediction algorithm, a region proposal centric frame partitioning, and a resource-aware multi-offloading scheme. We implement and evaluate Elf upon Linux and Android platforms using four commercial mobile devices and three deep vision applications with ten state-of-the-art models. The comprehensive experiments show that Elf can speed up the applications by 4.85× with saving bandwidth usage by 52.6%, while with <1% application accuracy sacrifice.

This paper presents the design, implementation, and evaluation of Shrimp, an underwater visible light communication (VLC) system. To address the unique issues in underwater environment such as water flow and scattered sunlight interference, we exploit the circularly polarized light (CPL) and double links for underwater VLC transmission. A coding scheme tailored for underwater communication based on double CPL design is developed. We prototype Shrimp on commercial-off-the-shelf (COTS) LEDs with fabricated printed circuit boards (PCBs). Extensive experiments conducted in an indoor water pool, a lake, and the sea demonstrate that Shrimp can combat against environmental interference and achieve robust communication in underwater environments. The communication distance can be up to 3 m in sea/lake water using a 3 W commodity LED, outperforming the VLC schemes designed for in-air communication.

In-air gesture control extends a touch screen and enables contact-less interaction, thus has become a popular research direction in the past few years. Prior work has implemented this functionality based on cameras, acoustic signals, and Wi-Fi via existing hardware on commercial devices. However, these methods have low user acceptance. Solutions based on cameras and acoustic signals raise privacy concerns, while WiFi-based solutions are vulnerable to background noise. As a result, these methods are not commercialized and recent flagship smartphones have implemented in-air gesture recognition by adding extra hardware on-board, such as mmWave radar and depth camera. The question is, can we support in-air gesture control on legacy devices without any hardware modifications?

Recognition of human-object interactions is practically important in various human-centric sensing scenarios such as smart supermarket, factory, and home. This paper proposes an RF-Camera system by fusing RFID and Computer Vision (CV) techniques, which is the first work to recognize the human gestural interactions with physical objects in multi-subject and multi-object scenarios. In RF-Camera, we first propose a dimension reduction method to transform the subject's 3D hand trajectory captured by depth camera to a 2D image, using which the subject's gesture can be recognized. We also propose a method to extract the facial image of target subject from an image that may contain irrelevant subjects, thereby further recognizing his/her identity. Finally, we model the physical movements of the held object's tag and further predict the tag phase data, by comparing which with real phase data of each tag human-object matching can be discovered. When implementing RF-Camera, three technical challenges need to be addressed. (i) To remove noisy data corresponding to irrelevant actions from raw sensing data, we propose a state transition diagram to determine the boundary of effective data. (ii) To predict phase data of the held target tag with unknown hand-tag offset, we quantify target tag trajectory by adding a variable hand-tag vector to captured hand trajectory. (iii) To ensure high reading rates of target tags in tag-dense scenarios, we propose a CV-assisted RFID scheduling method, in which analytics on CV data can help schedule RFID readings. We conduct extensive experiments to evaluate the performance of RF-Camera. Experimental results demonstrate that RF-Camera can recognize the gestural actions, human identity and human-object matching with an average accuracy higher than 90% in most cases.

Arterial blood pressure (ABP) monitoring using wearables has emerged as a promising approach to empower users with self-monitoring for effective diagnosis and control of hypertension. However, existing schemes mainly monitor ABP at discrete time intervals, involve some form of user effort, have insufficient accuracy, and require collecting sufficient training data for model development. To tackle these problems, we propose Crisp-BP, a novel ABP monitoring system leveraging the PPG sensor available in commercial wrist-worn devices (e.g., smartwatches or fitness trackers). It enables continuous, accurate, user-independent ABP monitoring and requires no behavior changes during collecting PPG data. The basic idea is to illuminate a skin/tissue, measure the light absorption, and characterize ABP-related blood volume change in the artery. To obtain accurate measurements and relieve the pain of training data collection, we use an arterial pulse extraction method that removes interference caused by capillary pulses. Moreover, we design a contact pressure estimation method to combat the deficiency of PPG waveform being sensitive to the contact pressure between the sensor and the skin. In addition, we leverage the great power of Bidirectional Long Short Term Memory and design a hybrid neural network model to enable user-independent ABP monitoring, so that users do not have to provide training data for model development. Furthermore, we propose a transfer learning method that first extracts general knowledge from online PPG data, then use it to improve the learning of a new model on our target problem. Extensive experiments with 35 participants demonstrate that Crisp-BP obtains the average estimation error of 0.86 mmHg and 1.67 mmHg and the standard deviation error of 6.55 mmHg and 7.31 mmHg for diastolic pressure and systolic pressure, respectively. These errors are within the acceptable range regulated by the FDA's AAMI protocol, which allows average errors of up to 5 mmHg and a standard deviation of up to 8 mmHg. Our results demonstrate that Crisp-BP is promising for improving the diagnosis and control of hypertension as it provides continuousness, comfort, convenience, and accuracy.

WiFi backscatter communication has been proposed to enable battery-free sensors to transmit data using WiFi networks. The main advantage of WiFi backscatter technologies over RFID is that data from their tags can be read using existing WiFi infrastructures instead of specialized readers. This can potentially reduce the complexity and cost of deploying battery-free sensors. Despite extensive work in this area, none of the existing systems are in widespread use today. We hypothesize that this is because WiFi-based backscatter tags do not scale well and their range and capabilities are limited when compared with RFID. To test this hypothesis we conduct several real-world experiments.

This paper reports our five-year lessons of developing and using MobileInsight, an open-source community tool to enable software-defined full-stack, runtime mobile network analytics inside our phones. We present how MobileInsight evolves from a simple monitor to a community toolset with cross-layer analytics, energy-efficient real-time user-plane analytics, and extensible user-friendly analytics at the control and user planes. These features are enabled by various novel techniques, including cross-layer state machine tracking, missing data inference, and domain-specific cross-layer sampling. Their powerfulness is exemplified with a 5-year longitudinal study of operational mobile network latency using a 6.4TB dataset with 6.1 billion over-the-air messages. We further share lessons and insights of using MobileInsight by the community, as well as our visions of MobileInsight's past, present, and future.