Vinci2: Providing Proactive Assistance in Continuous Egocentric Videos

Abstract

When should an intelligent assistant speak up without beingasked? Continuous egocentric video offers rich, evolving context that en-ables a new form of assistance: one that is proactive rather than merelyreactive. Yet existing approaches either wait passively for user queries ortreat every detected event as requiring a response, without consideringthe user’s history, current activity, or whether assistance would actuallybe welcome. We reframe proactive assistance as a context-dependent de-cision problem: the agent must not only perceive what is happening,but reason over accumulated temporal context to determine when andwhether to intervene. To this end, we present Vinci2, a proactive ego-centric assistance system that advances the on-device assistant Vincifrom reactive response toward proactivity. On the evaluation side, wepresent EgoServe, the first large-scale benchmark for proactive assistancein continuous egocentric video. EgoServe comprises over 3,000 serviceinstances organized along 4 temporal memory horizons, ranging fromimmediate safety alerts to long-term habit coaching, across 10 servicecategories. On the modeling side, we propose EgoMemo, a training-free,memory-augmented agent that maintains three complementary mem-ory representations: multi-scale temporal summaries, a semantic knowl-edge graph, and visual embedding archives. At each timestep, EgoMemoperforms retrieval-augmented reasoning to determine whether assistanceis warranted and, if so, produces contextually grounded responses. Ex-periments demonstrate that EgoMemo establishes strong baselines onEgoServe while remaining competitive on existing egocentric benchmarks.Our benchmark and code are publicly available at Vinci2.