TulipTender: A Mobile 3D Parkinson's Disease Assessment System Using a Single Camera

Abstract

Parkinson's disease (PD) assessment currently relies on subjective, episodic expert observations that often miss critical symptom fluctuations. Existing mobile methods are either impractical for home use or fail to capture the full spectrum of motor deficits. In this paper, we present TulipTender, a system that runs entirely on a consumer laptop using only its built-in camera, without any cloud offloading. TulipTender introduces a multi-scale 3D human modeling framework that jointly recovers full-body motion and fine hand articulations in metric space from a single video. This enables both millimeter-scale finger-tapping and meter-scale gait analysis. Furthermore, it suppresses visual artifacts, such as specular reflections and floor penetration, in uncontrolled home environments via a physics-aware purification framework. The system then extracts interpretable kinematic variables mirroring Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS) descriptors. These variables are processed by a lightweight hierarchical cross-modal transformer (HCT) for accurate, explainable score prediction. Evaluated on an RTX 4090 with 70 participants across diverse settings, including 43 inpatients, 20 outpatients, 2 nursing-home residents, and 5 home-based participants, TulipTender achieves F1-scores of 82% and 84% for finger tapping and gait, respectively. The full pipeline also runs locally on a MacBook Pro, supporting the feasibility of privacy-preserving assessment on consumer hardware.