MobiCom 2024 Papers

The advent of 5G promises high bandwidth with the introduction of mmWave technology recently, paving the way for throughput-sensitive applications. However, our measurements in commercial 5G networks show that frequent handovers in 5G, due to physical limitations of mmWave cells, introduce significant under-utilization of the available bandwidth. By analyzing 5G link-layer and TCP traces, we uncover that improper interactions between these two layers causes multiple inefficiencies during handovers. To mitigate these, we propose M2HO, a novel device-centric solution that can predict and recognize different stages of a handover and perform state-dependent mitigation to markedly improve throughput. M2HO is transparent to the firmware, base stations, servers, and applications. We implement M2HO and our extensive evaluations validate that it yields significant improvements in TCP throughput with frequent handovers.

Mobile task automation is an attractive technique that aims to enable voice-based hands-free user interaction with smartphones. However, existing approaches suffer from poor scalability due to the limited language understanding ability and the non-trivial manual efforts required from developers or endusers. The recent advance of large language models (LLMs) in language understanding and reasoning inspires us to rethink the problem from a model-centric perspective, where task preparation, comprehension, and execution are handled by a unified language model. In this work, we introduce AutoDroid, a mobile task automation system capable of handling arbitrary tasks on any Android application without manual efforts. The key insight is to combine the commonsense knowledge of LLMs and domain-specific knowledge of apps through automated dynamic analysis. The main components include a functionality-aware UI representation method that bridges the UI with the LLM, exploration-based memory injection techniques that augment the app-specific domain knowledge of LLM, and a multi-granularity query optimization module that reduces the cost of model inference. We integrate AutoDroid with off-the-shelf LLMs including online GPT-4/GPT-3.5 and on-device Vicuna, and evaluate its performance on a new benchmark for memory-augmented Android task automation with 158 common tasks. The results demonstrated that AutoDroid is able to precisely generate actions with an accuracy of 90.9%, and complete tasks with a success rate of 71.3%, outperforming the GPT-4-powered baselines by 36.4% and 39.7%.

The advent of large language models (LLMs) has opened up new opportunities in the field of mobile task automation. Their superior language understanding and reasoning capabilities allow users to automate complex and repetitive tasks. However, due to the inherent unreliability and high operational cost of LLMs, their practical applicability is quite limited. To address these issues, this paper introduces MobileGPT1, an innovative LLM-based mobile task automator equipped with a human-like app memory. MobileGPT emulates the cognitive process of humans interacting with a mobile app---explore, select, derive, and recall. This approach allows for a more precise and efficient learning of a task's procedure by breaking it down into smaller, modular sub-tasks that can be re-used, re-arranged, and adapted for various objectives. We implement MobileGPT using online LLMs services (GPT-3.5 and GPT-4) and evaluate its performance on a dataset of 185 tasks across 18 mobile apps. The results indicate that MobileGPT can automate and learn new tasks with 82.7% accuracy, and is able to adapt them to different contexts with near perfect (98.75%) accuracy while reducing both latency and cost by 62.5% and 68.8%, respectively, compared to the GPT-4 powered baseline.

Global Navigation Satellite System (GNSS) is extensively utilized for outdoor positioning and navigation. However, achieving high-precision indoor positioning is challenging due to the significant attenuation of GNSS signals indoors. To address this issue, we propose an innovative indoor GNSS positioning system called GPMS, which uses passive metasurface technology to redirect GNSS signals from outdoors into indoor spaces. These passive metasurfaces are strategically optimized for indoor coverage by steering and scattering the GNSS signals across a wide range of incident angles. We further develop a novel localization algorithm that can determine which metasurface the signal goes through and localize the user using the set of metasurfaces as anchor points. A distinct advantage of our localization algorithm is that it can be implemented on existing mobile devices without any hardware modifications. We implement the prototype of GPMS, and deploy six metasurfaces in two indoor environments, a 10×50 m2 office floor and a 15×20 m2 lecture room, to evaluate system performance. In terms of coverage, our GPMS increases the C/N0 from 9.1 dB-Hz to 23.2 dB-Hz and increases the number of visible satellites from 3.6 to 21.5 in the office floor. In terms of indoor positioning accuracy, our proposed system decreases the absolute positioning error from 30.6 m to 3.2 m in the office floor, and from 11.2 m to 2.7 m in the lecture room, demonstrating the feasibility and benefits of metasurface-assisted GNSS for indoor positioning.

3D object detection with omnidirectional views enables safety-critical applications such as mobile robot navigation. Such applications increasingly operate on resource-constrained edge devices, facilitating reliable processing without privacy concerns or network delays. To enable cost-effective deployment, cameras have been widely adopted as a low-cost alternative to LiDAR sensors. However, the compute-intensive workload to achieve high performance of camera-based solutions remains challenging due to the computational limitations of edge devices. In this paper, we present Panopticus, a carefully designed system for omnidirectional and camera-based 3D detection on edge devices. Panopticus employs an adaptive multi-branch detection scheme that accounts for spatial complexities. To optimize the accuracy within latency limits, Panopticus dynamically adjusts the model's architecture and operations based on available edge resources and spatial characteristics. We implemented Panopticus on three edge devices and conducted experiments across real-world environments based on the public self-driving dataset and our mobile 360° camera dataset. Experiment results showed that Panopticus improves accuracy by 62% on average given the strict latency objective of 33ms. Also, Panopticus achieves a 2.1× latency reduction on average compared to baselines.

mmWave networks offer wide bandwidth for high-speed wireless communication but suffer from limited range and susceptibility to blockage. Existing coverage provisioning solutions not only incur high costs but also require significant expert knowledge and manual efforts. In this paper, we present AutoMS, an automated service framework to optimize mmWave coverage by strategically designing and placing low-cost passive metasurfaces. Our approach consists of three key components: (1) joint optimization of metasurface phase configurations and placement as well as access point beamforming codebooks. (2) a fast 3D ray-tracing simulator for accelerated large-scale metasurface channel modeling. (3) a metasurface design amenable to ultra-low-cost hot stamping fabrication, featuring high reflectivity, near 2π phase control, and wideband support. Simulation and testbed experiments show that AutoMS can increase the median received signal strength by 11 dB in target rooms and over 20 dB at previous blind spots, and improve the median throughput by over 3× in real-world scenarios.

This paper introduces a novel multi-directional speaker, named MuDiS, which utilizes a parametric array to generate highly focused sound beams in multiple directions. The system capitalizes on air nonlinearity to reproduce sound from ultrasounds, successfully overcoming challenges inherent in traditional parametric arrays, such as transducer size and wavefront shape. It supports three important features simultaneously: independent beams, wide-angle digital steering, and unintended leakage suppression. To address these challenges, we designed a specialized cell structure that connects ultrasonic transducers, redirecting an approximately omnidirectional wavefront with optimal interspacing. An optimization-based algorithm is developed to minimize unintended leakages, and a nonlinear distortion reduction scheme is proposed to enhance sound quality. The paper showcases a prototype demonstrating the system's capabilities as a multidirectional speaker with a wide sound projection angle. Experimental results validate the effectiveness of our approach. The proposed multi-beam projection system rivals the performance of commercially available single-beam projection directional speakers, and improved steering angle and sound fidelity compared to multi-beamforming performance using traditional parametric arrays.

IEEE 802.11 (WiFi) only has two modes of trust---complete trust or complete untrust. The lack of nuance leaves no room for sensors that a user does not fully trust but wants to connect to their network, such as a WiFi sensor. Solutions exist, but they require advanced knowledge of network administration. We solve this problem by introducing a new way of modulating data in the latency of the network, called Latency Shift Keying. We use specific characteristics of the WiFi protocol to carefully control the latency of just one device on the network. We build a transmitter, receiver, and modulation scheme that is designed to encode data in the latency of a network. We develop an application, Wicket, that solves the WiFi trust issue using Latency Shift Keying to create a new security association between an untrusted WiFi sensor and a wired device on the trusted network. We evaluate its performance and show that it works in many network conditions and environments.

To provide full-body 3D videos of performers showcasing diverse clothes with dynamic movements for e-commerce platforms, we develop an end-to-end, low-cost, and high-fidelity production and deployment pipeline. We first set up a low-cost capture studio with only 24 RGB cameras and embrace fast neural surface reconstruction to produce high-quality meshes without depth information. We then quickly group all the frames with local motion priors, select a keyframe for each group, and accurately register any other frame to the keyframe under the guidance of semantic labels, thereby avoiding transmitting all the frames to mobile devices and loading them into memory. For real-time rendering, we propose an on-device sparse computation method for efficient deformation from keyframes to the other frames. Evaluation over 2 self-captured performances and 8 public performances reveals that the pipeline achieves the reconstruction time of 28 minutes per frame, the average PSNR of 30.4, the average bandwidth requirement of 4.2MB/s, and the on-device frame rate of 60 fps, demonstrating superiority over existing baselines.

This paper shares the challenges facing today's augmented reality (AR) smartphone applications, particularly in the realm of localization and tracking failure. Our research identifies limitations in current vision-based landmarks such as QR codes and AprilTags, commonly used to aid in localization, and the drawbacks of LiDAR integration in variable lighting conditions, compromising AR's accuracy and functionality. We also examine the constraints of Inertial Measurement Units (IMU) on movement speed, highlighting its impact on the dynamic performance of AR applications. Based on our extensive 316 experimental cases for 113 hours, including 34 case studies with 17 subjects in 2 sites, this paper presents the field with a nuanced analysis of the failure modes inherent in smartphone-based AR localization. We further explore a prototype solution which fuses ultra-wideband (UWB)-based sensing with the vision-based systems to alleviate these failure modes. Our approach addresses the immediate challenges of AR localization and opens avenues for future research and development in creating more spatially aware and interactive digital worlds. All of our demonstration videos, code, and datasets are available here1.

Pesticide overuse poses significant risks to human health and environmental integrity. Addressing the limitations of existing approaches, which struggle with the diversity of pesticide compounds, portability issues, and environmental sensitivity, this paper introduces Hornbill. A wireless and battery-free electrochemical bio-tag that integrates the advantages of NFC technology with electrochemical biosensors for portable, precise, and touchless multi-pesticide detection. The basic idea of Hornbill is comparing the distinct electrochemical responses between a pair of biological receptors and different pesticides to construct a unique set of feature fingerprints to make multi-pesticide sensing feasible. To incorporate this idea within small NFC tags, we reengineer the electrochemical sensor, spanning the antenna to the voltage regulator. Additionally, to improve the system's sensitivity and environmental robustness, we carefully design the electrodes by combining microelectrode technology and materials science. Experiments with 9 different pesticides show that Hornbill achieves a mean accuracy of 93% in different concentration environments and its sensitivity and robustness surpass that of commercial electrochemical sensors.

Transformers have recently revolutionized the machine learning (ML) landscape, gradually making their way into everyday tasks and equipping our computers with "sparks of intelligence". However, their runtime requirements have prevented them from being broadly deployed on mobile. As personal devices become increasingly powerful at the consumer edge and prompt privacy becomes an ever more pressing issue, we explore the current state of mobile execution of Large Language Models (LLMs). To achieve this, we have created our own automation infrastructure, MELT, which supports the headless execution and benchmarking of LLMs on device, supporting different models, devices and frameworks, including Android, iOS and Nvidia Jetson devices. We evaluate popular instruction fine-tuned LLMs and leverage different frameworks to measure their end-to-end and granular performance, tracing their memory and energy requirements along the way.

Autonomous mobile systems increasingly rely on deep neural networks for perception and decision-making. While effective, these systems are vulnerable to adversarial machine learning attacks where small perturbations in the input could significantly impact the outcome of the system. Common countermeasures include leveraging adversarial training and/or data or network transformation. Although widely used, the main drawback of these countermeasures is that they require full and invasive access to the classifiers, which are typically proprietary. Additionally, the cost of training or retraining is often prohibitively expensive for large models. To tackle this, purification models have recently been proposed. The aim is to incorporate a "purification" layer before classification, thereby eliminating the necessity to modify the classifier. Despite their effectiveness, state-of-the-art purification methods are compute-intensive, rendering them unsuitable for mobile systems where resources are constrained and large latency is not desired.

Despite the prevalence of Crowdsourced Live Video Streaming (CLVS), video viewers still suffer from low QoE particularly under rush hours, as the existing Content Delivery Network (CDN) is not scalable enough to handle the massive concurrent streaming. The rapid emergence of Web 3.0 provides new incentives for revisiting and applying the classical P2P networking in CLVS. However, the highly dynamic joining or leaving behavior of CLVS viewers frequently interrupts the real-time streaming and leads to low QoE, which demands to retrofit P2P. In this work, we bridge the gap by proposing a reliable P2P-assisted CLVS system named Venus, where viewers can share their streaming content smoothly, without video freeze regardless of viewers leaving. To realize Venus, different from the single-flow sharing in previous P2P video streaming, we design a novel multiflow framework with lightweight redundancy encoding, so as to handle the inherently high viewer dynamics. Correspondingly, we introduce a multiflow scheduler to enable QoE adaption concertedly over heterogeneous multiple flows. Real-world evaluation confirms the benefits of decentralized CLVS streaming, with Venus outperforming the state-of-the-art CDN solution by almost totally eliminating the video stall while enhancing the video quality by 10.2%.

On-vehicle 3D sensing technologies, such as LiDARs and stereo cameras, enable a novel capability, 3D traffic reconstruction. This produces a volumetric video consisting of a sequence of 3D frames capturing the time evolution of road traffic. 3D traffic reconstruction can help trained investigators reconstruct the scene of an accident. In this paper, we describe the design and implementation of RECAP, a system that continuously and opportunistically produces 3D traffic reconstructions from multiple vehicles. RECAP builds upon prior work on point cloud registration, but adapts it to settings with minimal point cloud overlap (both in the spatial and temporal sense) and develops techniques to minimize error and computation time in multi-way registration. On-road experiments and trace-driven simulations show that RECAP can, within minutes, generate highly accurate reconstructions that have 2× or more lower errors than competing approaches.

Alzheimer's Disease (AD) and related dementia are a growing global health challenge due to the aging population. In this paper, we present ADMarker, the first end-to-end system that integrates multi-modal sensors and new federated learning algorithms for detecting multidimensional AD digital biomarkers in natural living environments. ADMarker features a novel three-stage multi-modal federated learning architecture that can accurately detect digital biomarkers in a privacy-preserving manner. Our approach collectively addresses several major real-world challenges, such as limited data labels, data heterogeneity, and limited computing resources. We built a compact multi-modality hardware system and deployed it in a four-week clinical trial involving 91 elderly participants. The results indicate that ADMarker can accurately detect a comprehensive set of digital biomarkers with up to 93.8% accuracy and identify early AD with an average of 88.9% accuracy. ADMarker offers a new platform that can allow AD clinicians to characterize and track the complex correlation between multidimensional interpretable digital biomarkers, demographic factors of patients, and AD diagnosis in a longitudinal manner.

Increasing bandwidth demands of mobile video streaming pose a challenge in optimizing the Quality of Experience (QoE) for better user engagement. Multipath transmission promises to extend network capacity by utilizing multiple wireless links simultaneously. Previous studies mainly tune the packet scheduler in multipath transmission, expecting higher QoE by accelerating transmission. However, since Adaptive BitRate (ABR) algorithms overlook the impact of multipath scheduling on throughput prediction, multipath adaptive streaming can even experience lower QoE than single-path. This paper proposes Chorus, a cross-layer framework that coordinates multipath scheduling with adaptive streaming to optimize QoE jointly. Chorus establishes two-way feedback control loops between the server and the client. Furthermore, Chorus introduces Coarse-grained Decisions, which assist appropriate bitrate selection by considering the scheduling decision in throughput prediction, and Finegrained Corrections, which meet the predicted throughput by QoE-oriented multipath scheduling. Extensive emulation and real-world mobile Internet evaluations show that Chorus outperforms the state-of-the-art MPQUIC scheduler, improving average QoE by 23.5% and 65.7%, respectively.

Emerging wireless IoT applications increasingly venture beyond over-the-air communication, such as deep-tissue networking for medical sensors, air-water communication for oceanography, and soil sensing for agriculture. These applications face the fundamental challenge of significant reflection and power loss at medium interfaces. We present RF-Mediator, a programmable metasurface placed near a medium interface to mask the presence of a physical boundary. Our hardware design comprises a single layer of varactor-based surface elements with specific metallic patterns and wiring. With the biasing voltage tuned element-wise, the surface dynamically mediates between the adjacent media to minimize unwanted reflection and boost transmission through the medium interface. A multi-stage control algorithm efficiently determines the surface configuration to handle all dynamic adaptation needs for medium impedance matching and beamforming jointly. We implement a lightweight and flexible metasurface prototype and experiment with diverse cross-medium setups. Extensive evaluation shows that RF-Mediator provides a median power gain of 8 dB for air-tissue links and up to 30 dB for cross-medium backscatter links.

Due to the popularity of deep neural networks (DNNs) and considerations over network overhead, data privacy, and inference latency, there is a growing interest in deploying DNNs to edge devices in recent years. However, the limited memory becomes a major bottleneck for on-device DNN deployment, making it crucial to reduce the memory footprint of DNN. The mainstream model customization solutions require intensive deployment efforts and may lead to severe accuracy degradation, and existing deep learning (DL) frameworks don't take memory as a priority. Besides, recent works to enhance the memory management scheme cannot be directly applied because of several challenges, including the unbalanced memory footprint across layers, the inevitable overhead of memory management, and the memory budget dynamicity. To tackle these challenges, we introduce FlexNN, an efficient and adaptive memory management framework for DNN inference on memory-constrained devices. FlexNN uses a slicing-loading-computing joint planning approach, to achieve optimal memory utilization and minimal memory management overhead. We implemented FlexNN atop NCNN, and conducted comprehensive evaluations with common model architectures on various devices. The results have shown that our approach is able to adapt to different memory constraints with optimal latency-memory trade-offs. For example, FlexNN can reduce the memory consumption by 93.81% with only a 3.64% increase in latency, as compared with the original NCNN on smartphones.