paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

ECCV 2026

Multi-dimensional Preference Alignment by Conditioning Reward Itself

Jiho Jang, Jin-Young Kim, Kyungjune Baek, Nojun Kwak

Reinforcement Learning from Human Feedback has emergedas a standard for aligning di!usion models. However, we identify a fun-damental limitation of the standard DPO formulation when optimizingmulti-dimensional preferences: by relying on the Bradley-Terry model,it aggregates heterogeneous evaluation axes (e.g., aesthetic quality andsemantic alignment) into a single scalar reward. This aggregation createsa reward conflict where the model is forced to unlearn desirable featuresof a specific dimension if they appear in a globally non-preferred sam-ple. To address this issue, we propose Multi Reward Conditional DPO(MCDPO) which resolves reward conflicts by introducing a disentangledBradley-Terry objective. MCDPO explicitly injects a preference outcomevector as a condition during training, which allows the model to learnthe correct optimization direction for each reward axis independentlywithin a single network. Extensive experiments on Stable Di!usion 1.5and SDXL demonstrate that MCDPO achieves superior performance onbenchmarks even with 18% and 3% training data respectively. Notably,our conditional framework enables dynamic and multiple-axis control atinference time using Classifier Free Guidance to amplify specific rewarddimensions without additional training or external reward models.

ECCV 2026

MAVFusion: Efficient Infrared and Visible Video Fusion via Motion-Aware Sparse Interaction

Xilai Li, Weijun Jiang, Xiaosong Li, Yang Liu, Hongbin Wang, Tao Ye, Huafeng Li, Haishu Tan

Infrared and visible video fusion combines the object saliencyfrom infrared images with the texture details from visible images toproduce semantically rich fusion results. However, most existing meth-ods are designed for static image fusion and cannot effectively han-dle frame-to-frame motion in videos. Current video fusion methods im-prove temporal consistency by introducing interactions across frames,but they often require high computational cost. To mitigate these chal-lenges, we propose MAVFusion, an end-to-end video fusion frameworkfeaturing a motion-aware sparse interaction mechanism that enhancesefficiency while maintaining superior fusion quality. Specifically, we lever-age optical flow to identify dynamic regions in multi-modal sequences,adaptively allocating computationally intensive cross-modal attentionto these sparse areas to capture salient transitions and facilitate inter-modal information exchange. For static background regions, a lightweightweak interaction module is employed to maintain structural and appear-ance integrity. By decoupling the processing of dynamic and static re-gions, MAVFusion simultaneously preserves temporal consistency andfine-grained details while significantly accelerating inference. Extensiveexperiments demonstrate that MAVFusion achieves state-of-the-art per-formance on multiple infrared and visible video benchmarks, achieving aspeed of 14.16 FPS at 640 × 480 resolution. The source code is availableat https://github.com/ixilai/MAVFusion.

ECCV 2026

LMGenDrive: Bridging Multimodal Understanding and Generative World Modeling for End-to-End Driving

Hao Shao, Letian Wang, Yang Zhou, Yuxuan Hu, Zhuofan Zong, Steven Waslander, Wei Zhan, Hongsheng Li

Recent years have witnessed remarkable progress in autonomousdriving, yet generalization to long-tail and open-world scenarios remainsthe primary bottleneck for large-scale deployment. To address this, oneline of research explores LLMs and VLMs for their vision-language un-derstanding and reasoning capabilities, equipping AVs with the abilityto interpret rare and safety-critical situations when generating drivingactions. In parallel, another line investigates generative world models tocapture the spatio-temporal evolution of driving scenes, enabling agentsto imagine and evaluate possible futures before acting. Inspired by hu-man intelligence, which seamlessly unites understanding and imaginationas a hallmark of AGI, this work explores a unified model that bringsthese two capabilities together for autonomous driving. We present LM-GenDrive, the first framework that unifies LLM-based multimodal un-derstanding with generative world models for end-to-end closed-loop au-tonomous driving. Given multi-view camera inputs and natural-languageinstructions, our model generates both realistic future driving videosand corresponding control signals. By coupling an LLM with genera-tive video capabilities, LMGenDrive gains complementary benefits: fu-ture video prediction enhances spatio-temporal scene modeling, while theLLM provides strong semantic priors and instruction grounding learnedfrom large-scale pretraining. A progressive three-stage training strat-egy—ranging from vision pretraining to multi-step long-horizon driv-ing—is proposed to further improve stability and performance. The re-sulting model can also operate in two complementary modes: low-latencyonline planning and autoregressive offline video generation. Experimentsshow that LMGenDrive significantly outperforms state-of-the-art meth-ods on challenging closed-loop driving benchmarks, improving instruc-tion following, spatio-temporal understanding, and robustness to rarescenarios. Our work not only sets a new state-of-the-art in autonomousdriving, but also demonstrates that unifying multimodal understandingand generation provides a promising direction for building more gener-alizable and robust embodied decision-making systems.

ECCV 2026

LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing

Xinyu Wang, Chongbo Zhao, Fangneng Zhan, Yue Ma

Streaming video editing has made rapid progress, yet practi-cal deployment is still limited by two core issues: maintaining stable back-grounds and non-edited regions over time, and achieving the low latencyrequired for real-time interactive scenarios. Meanwhile, recent streamingvideo generation methods are mostly developed for synthesis and cannotbe directly applied to editing due to the strict preservation requirementand region-specific control. In this work, we present a novel streamingvideo editing framework that performs causal, frame-by-frame editingwith strong content preservation and real-time responsiveness. Our keydesign is a three-stage distillation pipeline that progressively transfersediting capability from a powerful bidirectional foundation model to anefficient unidirectional streaming editor, enabling stable long-horizon ed-its without sacrificing visual fidelity. To further support real-time deploy-ment, we introduce an AR-oriented mask cache that reuses region-relatedcomputation across frames, substantially reducing redundant processingand accelerating inference. Finally, we establish a dedicated benchmarkfor streaming video editing. Extensive evaluations demonstrate that ourmethod achieves state-of-the-art visual quality among streaming base-lines while drastically boosting inference speed to 12.66 FPS, making itsuitable for interactive and augmented reality applications.

ECCV 2026

Learning Semantic-Robust Change Detection via Semantic-Invariant Self-Distillation

Jiuhe Qu, Yingping Liang, Ying Fu

Change detection aims to identify semantic changes betweenremote sensing images. However, features from models are easily dis-turbed by non-semantic variations, such as illumination, shadows, andatmospheric changes, leading to false alarms and limited generaliza-tion in real-world scenarios. In this paper, we propose SCDistill, aframework for learning semantic-robust change detection via semantic-invariant self-distillation. First, to strengthen semantic consistency, weintroduce a semantic-invariant self-distillation strategy that learns se-mantic robustness from perturbed yet semantically consistent data, em-powering the change detector to extract disturbance-resistant featuresand achieve more reliable and accurate semantic change identification.Second, to expand paired data with non-semantic variations, we designa diffusion-based perturbation simulation pipeline that synthesizes com-plex environmental changes, enabling the model to explicitly learn todistinguish semantic changes from appearance-level fluctuations and re-duce false alarms caused by non-semantic disturbances. These compo-nents promote robustness from data and representation perspectives,leading to synergistic performance gains. Extensive experiments demon-strate that SCDistill achieves state-of-the-art performance on multiplesemantic change detection benchmarks and exhibits strong generaliza-tion to binary change detection and change captioning tasks. Code isaccessible at https://github.com/elecreak/SCDistill.

ECCV 2026

Jumping the Landing Phase: Noise Variance Matching Enables Accurate Few-Step Inversion

Yang Luo, Zhineng Chen, Ya Gao, Xieping Gao, Yu-Gang Jiang

Among inversion approaches in diffusion models, DDIM in-version is widely used for reconstructing latent trajectories, offering adeterministic and training-free alternative to iterative methods. How-ever, we observe that it often suffers from reconstruction errors due to afundamental asymmetry in the inversion process: the noise is predictedfrom zt−1 rather than zt , leading to a consistent underestimation of noisemagnitude. This bias causes a partial denoising effect, where the latentstate is insufficiently perturbed, degrading reconstruction quality. Iter-ative methods such as ReNoise mitigate this issue by repeatedly refin-ing the predicted noise. We find that during the initial iterations, thepredicted variance rapidly increases toward its correct scale—an earlystage we term the landing phase, which accounts for most of ReNoise’scomputational cost. Based on this insight, we propose Noise VarianceMatching (NVM), a simple and efficient strategy that explicitly alignsthe predicted noise variance with that of a forward reference sample,bypassing the landing phase. NVM preserves the deterministic and skip-step inference property of DDIM while maintaining high reconstructionfidelity under few-step settings. Experiments across various datasets anddiffusion backbones demonstrate that NVM achieves competitive or su-perior reconstruction quality with far fewer UNet calls, offering a prac-tical and scalable alternative to iterative refinement-based methods.

ECCV 2026

Integrated Forward–Inverse Network for Reconstruction for Lensless Image Reconstruction

Donggeon Bae, Jaewoo Jung, Yong Guk Kang, Kyung Chul Lee, Taeyoung Kim, Jongho Kim, Sangjun Byun, Joonsik Park, Seung-Ah Lee

Lensless imaging enables compact and versatile computa-tional cameras by replacing bulky optics with thin coded elements. How-ever, reconstruction from the resulting measurements is challenging: large-footprint point-spread functions (PSFs) produce highly multiplexed ob-servations, making inversion severely ill-conditioned and sensitive to cal-ibration errors and model mismatch. While deep learning approaches,including hybrid models that incorporate physics priors, have shownpromise, explicitly maintaining data fidelity throughout the network hier-archy remains difficult. Here, we propose the Integrated Forward–InverseNetwork (IFIN), a physics-guided architecture that interleaves differen-tiable forward projections with learnable inverse updates at every scale,enabling complementary cues to be exploited jointly in the measurementand image domains. This bidirectional coupling supports progressive,physics-consistent refinement and permits system-constrained PSF ker-nel adaptation under model uncertainty. On challenging lensless bench-marks, including a newly introduced dataset, IFIN achieves state-of-the-art reconstruction quality. We further observe competitive performanceon Gaussian deblurring and simulated inline holography reconstruction,suggesting that the same interleaving principle can extend beyond lens-less cameras.

ECCV 2026

Importance-Aware Low-Rank Distillation of Diffusion Transformers

Denis Zavadski, Sebastian Heid, Damjan Kalšan, Stefan Roth, Carsten Rother

Diffusion Transformers (DiTs) have emerged as a dominantarchitecture for high-quality text-to-image generation, yet their scaleposes challenges for efficient deployment. While truncated singular valuedecomposition (SVD) is a principled tool for parameter reduction, evi-dence from large language models (LLMs) suggests that naive low-rankapproximation can cause catastrophic failure. In contrast, we find thattruncated SVD in DiTs produces smooth degradation even under sub-stantial global compression, with redundancy distributed across projec-tion matrices throughout the whole network rather than concentratedin a few transformer blocks. Building on these insights, we introduceSVDtrunc, a two-step block-level compression scheme, first allocatingranks across blocks and compressing the least important ones via trun-cated SVD under a global parameter budget, and then fine-tuning allblocks with modular knowledge distillation and a rectified-flow objec-tive. We apply SVDtrunc to FLUX.dev across compression levels rangingfrom 40–90 % of the original parameter count. Across three benchmarks,GenEval, HPSv2, and DPG, we outperform all competing approaches.Notably, and in contrast to prior work, we retain near-full performanceat 68 % and remain competitive even at 57 % of the original parameterbudget. Furthermore, we show that SVDtrunc complements step distil-lation and achieves strong results even without fine-tuning, positioningit as a practical continuation of efficiency improvements beyond diffusionstep reduction for large-scale generative models.

ECCV 2026

i-Design: Step-by-Step Graphic Layout Design with Progressive Aesthetic Policy Optimization

Sohan Patnaik, Rishabh Jain, Balaji Krishnamurthy, Mausoom Sarkar

Automated layout generation powers a wide range of designs, such as posters, banners, and user interfaces. However, current single-shot generators overlook the iterative nature of human design and often produce aesthetically subpar compositions. We present i-Design, a unified framework that models layout generation as a progressive process, placing k elements at a time, conditioned on previously rendered partial canvases. Our approach combines Progressive Imitation Learning (PIL) on partial canvases, followed by Progressive Aesthetic Policy Optimization (PAPO), a reinforcement learning algorithm that aligns generation with aesthetic preferences. In PAPO, a frozen vision judge evaluates rendered layout pairs, constructs a directed comparison graph, and computes a global aesthetic consensus to derive trajectory-level rewards reflective of visual appeal rather than spatial accuracy. During training, multiple rollouts are rendered through a non-differentiable Skia engine and scored in real time by the judge to enable adaptive aesthetic learning. Experiments on Crello and WebUI benchmarks demonstrate that i-Design surpasses SoTA methods, achieving a 13% improvement in win rate and a 7% gain in mIoU, producing layouts with superior balance, alignment, and visual harmony. Our findings demonstrate that iterative generation and progressive aesthetic optimization offer a powerful new paradigm for human-aligned graphic layout generation.

ECCV 2026

HyLaR: Hybrid Latent Reasoning with Decoupled Policy Optimization

Tao Cheng, Shi-Zhe Chen, Hao Zhang, Yixin Qin, Jinwen Luo, Zheng Wei

Chain-of-Thought (CoT) reasoning significantly elevates thecomplex problem-solving capabilities of multimodal large language mod-els (MLLMs). However, adapting CoT to vision typically discretizes sig-nals to fit LLM inputs, causing early semantic collapse and discard-ing fine-grained details. While external tools can mitigate this, theyintroduce a rigid bottleneck, confining reasoning to predefined oper-ations. Although recent latent reasoning paradigms internalize visualstates to overcome these limitations, optimizing the resulting hybriddiscrete-continuous action space remains challenging. In this work, wepropose HyLaR (Hybrid Latent Reasoning), a framework that seamlesslyinterleaves discrete text generation with continuous visual latent rep-resentations. Specifically, following an initial cold-start supervised fine-tuning (SFT), we introduce DePO (Decoupled Policy Optimization) toenable effective reinforcement learning within this hybrid space. DePOdecomposes the policy gradient objective, applying independent trust-region constraints to the textual and latent components, alongside anexact closed-form von Mises-Fisher (vMF) KL regularizer. Extensive ex-periments demonstrate that HyLaR outperforms standard MLLMs andstate-of-the-art latent reasoning approaches across fine-grained percep-tion and general multimodal understanding benchmarks. Code is avail-able at https://github.com/EthenCheng/HyLaR.

ECCV 2026

GryphOne: Symbol-Aware Masked Diffusion for Structural Refinement in Offline Handwritten Mathematical Expression Recognition

Takaya Kawakatsu, Ryo Ishiyama

Handwritten mathematical expression recognition (HMER)requires reasoning over diverse symbols and structures, yet autoregressivemodels struggle with exposure bias and syntax inconsistency. We presentGryphOne, a discrete diffusion framework which reformulates HMER asiterative symbolic refinement instead of sequential generation. GryphOneprogressively refines symbols and relations, removing autoregression andimproving consistency. Symbol-aware tokenization and random-maskingmutual learning further enhance robustness to handwriting diversity. Onthe MathWriting benchmark, GryphOne achieves 5.51% CER and 59.9%EM (ExpRate), outperforming all reimplemented models in the matchedsetting as well as the commercial HMER system. Held-out evaluation onCROHME 2014–2023 further shows strong cross-dataset generalization.

ECCV 2026

Gaussian Volumetric Representation for Efficient Shear–Warp Visualization

Mayuri Mathur, Ojaswa Sharma

Medical image visualization requires volumetric rendering al-gorithms that preserve anatomical fidelity while maintaining high ren-dering speeds. To address the high computational cost of large volumet-ric datasets, we propose a Gaussian-based volumetric representation forefficient visualization of dense medical volumes without compromisingstructural and radiometric details. We optimize the proposed represen-tation using Monte Carlo volumetric estimation, which enables trainingon a highly sparse subset of voxels while maintaining consistency withthe dense volumetric objective. In addition, we introduce a curriculumlearning strategy that progressively incorporates structured slice-basedsampling during training. Sparse voxel samples provide an early globalcoverage of the volume, while slice samples capture spatially correlatedregions that aid geometric structure and texture continuity. This combi-nation enables the Gaussian representation to learn anatomical detailsof various structures and corresponding textures from sparse supervisionwhile significantly reducing the computational cost associated with densevoxel processing. The learned representation supports slice-based ren-dering methods such as shear–warp volume rendering, enabling efficientvisualization of multimodal medical datasets including MRI and Cryosec-tion volumes while preserving anatomical structures. Using sparse super-vision, our method achieves up to 43.86 FPS rendering with a compres-sion ratio of 11.31:1.

ECCV 2026

FSDC-DETR: A Frequency-Spatial Domain Collaborative DETR for Small-Object Detection

Aiwen Liu, Chengguang Zhu, Gang Wang, Dandan Zhu, Haodong Lin, Yan Wang, Huiyu Zhou, Zhengyi Pan

Small object detection (SOD) remains a challenging task inreal-world applications. Despite recent advances, existing detectors re-main limited by rigid processing that entangle spatial aggregation withimplicit frequency aliasing and truncation, leading to inadequate preser-vation of high-frequency components for SOD. To tackle these limita-tions, we propose a Frequency-Spatial Domain Collaborative DetectionTransformer (FSDC-DETR), a novel collaborative framework that ex-plicitly models complementary spatial and frequency representations.Specifically, we first introduce Dual-Branch Frequency-Spatial AdaptiveFusion (DBFSAF) to enhance frequency diversity and adaptively cap-ture frequency-spatial domain discriminative representations. Buildingon these representations, a frequency-spatial interaction scheme is fur-ther explored within the hybrid encoder to enable progressive featurepropagation to the decoder. In particular, structure-aware frequency-spatial aggregation is achieved through Shunt Frequency-Spatial Fea-ture Fusion (SFS-FF), establishing bidirectional interaction and pro-gressive cross-scale propagation between frequency and spatial repre-sentations for coherent discriminative modeling. Meanwhile, informativehigh-frequency responses are preserved during scale transitions throughFrequency-Spatial Dynamic Downsampling (FSD-Down), thereby mini-mizing frequency degradation throughout multi-scale fusion for the pre-cise SOD. Experimental results demonstrate that FSDC-DETR achievesstate-of-the-art performance, improving AP by 6.4 on VisDrone-DET2019and 6.6 on AITODv2, with gains of 6.8 and 6.9 AP for small objects. Thecode is available at https://github.com/nevereverinsomnia/FSDC-DETR.

ECCV 2026

From One-to-One to Many-to-Many: Dynamic Cross-Layer Injection for Deep Vision-Language Fusion

Cheng Chen, Yuyu Guo, Pengpeng Zeng, Jingkuan Song, Peng Di, Hang Yu, Lianli Gao

Vision-Language Models (VLMs) create a severe visual fea-ture bottleneck by using a crude, asymmetric connection that links onlythe output of the vision encoder to the input of the large language model(LLM). This static architecture fundamentally limits the ability of LLMsto achieve comprehensive alignment with hierarchical visual knowledge,compromising their capacity to accurately integrate local details withglobal semantics into coherent reasoning. 5 To resolve this, we introduceCross-Layer Injection (CLI), a novel framework that forges a dy-namic “many-to-many” bridge between the two modalities. CLI con-sists of two synergistic, parameter-efficient components: an AdaptiveMulti-Projection (AMP) module that harmonizes features from di-verse vision layers, and an Adaptive Gating Fusion (AGF) mech-anism that empowers the LLM to selectively inject the most relevantvisual information based on its real-time decoding context. We validatethe effectiveness and versatility of CLI by integrating it into LLaVA-OneVision and LLaVA-1.5. Extensive experiments on 28 diverse bench-marks demonstrate significant performance improvements, establishingCLI as a scalable paradigm that unlocks deeper multimodal understand-ing by granting LLMs on-demand access to the full visual hierarchy. Codeis available at https://github.com/codefuse-ai/CLI.

ECCV 2026

Extending a Large View Synthesis Model for Multi-view Panoptic Segmentation

Kwonyoung Ryu, In-Jae Lee, Jonghyun Jin, Hyunjee Lee, Jongmin Lee, Jaesik Park

Large view synthesis models synthesize novel views throughcross-view attention without explicit 3D representations, and recent stud-ies have shown that they learn accurate spatial correspondence fromRGB supervision alone. We observe that this correspondence generalizesbeyond appearance. When non-photorealistic signals such as binary en-coded panoptic labels are passed through the model, they are propagatedto novel views with consistent spatial structure. These results indicatethat the correspondence learned for RGB view synthesis can also propa-gate view-independent per-pixel labels. From this observation, we studyhow large view synthesis models can be extended beyond appearancerendering to 3D scene understanding. We propose a panoptic segmen-tation pipeline that reuses a frozen view synthesis model to propagatepanoptic labels from input views to novel views, without 3D reconstruc-tion or any segmentation-specific training of the view synthesis model.Given panoptic labels on the input views, we encode them into binarychannel representations and pass them through the same model to rendertarget-view segmentation. On ScanNet, our method achieves segmenta-tion quality on par with Gaussian based approaches requiring explicit3D reconstruction, while outperforming them in novel view synthesis bymore than 7 dB. The label propagation also transfers across datasets,surpassing these approaches on Replica without any fine-tuning.

ECCV 2026

Explicit Semantic–Spatial Alignment for Open-Vocabulary Object Detection

Pengyang Su, Weihong Ren, Shuhuan Han, Qian Dong, Haoran Xu, Xi Ai, Zijian Wang, Zhiyong Wang, Honghai LIU

Open-Vocabulary object Detection (OVD) typically relies onVision–Language Models (VLMs) to associate visual regions with arbi-trary textual concepts. To compensate for the limited localization capa-bility of VLMs, existing approaches often incorporate large pre-trainedvisual features that inherently encode object-centric spatial cues. How-ever, these features are commonly introduced through direct fusion orshallow adaptation, which fails to explicitly align semantic representa-tions with object-level spatial information, resulting in imprecise local-ization. In this paper, we propose a novel framework for explicit seman-tic–spatial alignment in open-vocabulary object detection. Rather thandirectly merging pre-trained object-centric visual features, our methodprogressively aligns semantic representations with object-level spatialcues through a staged alignment process. Specifically, we introduce alightweight spatial adapter that spatially recalibrates auxiliary object-centric visual features to suppress background responses and emphasizesalient object regions. Building upon this, we design a frequency-awarefusion mechanism that decomposes the adapted object-centric visual fea-tures into spectral components and adaptively injects object-sensitivespatial cues into semantic features at multiple levels. This targeted fu-sion strategy enables effective semantic alignment while preserving theoriginal semantic embedding space. Extensive experiments on the OV-COCO and OV-LVIS benchmarks demonstrate the effectiveness of ourapproach against state-of-the-art methods.

ECCV 2026

EvDiff: High Quality Video with an Event Camera

Weilun Li, Lei Sun, Ruixi Gao, Qi Jiang, Yuqin Ma, kaiwei wang, Ming-Hsuan Yang, Luc Van Gool, Danda Paudel

As neuromorphic sensors, event cameras asynchronously recordchanges in brightness as streams of sparse events with the advantages ofhigh temporal resolution and high dynamic range. Reconstructing inten-sity images from events is a highly ill-posed task due to the inherent am-biguity of absolute brightness. Early methods generally follow an end-to-end regression paradigm, directly mapping events to intensity frames ina deterministic manner. While effective to some extent, these approachesoften yield perceptually inferior results and struggle to scale up in modelcapacity and training data. In this work, we propose EvDiff, an event-based diffusion model that follows a surrogate training framework to pro-duce high-quality videos. To reduce the high computational cost of high-frame-rate video generation, we design an event-based diffusion modelthat performs only a single forward diffusion step, equipped with a tem-porally consistent EvEncoder. Furthermore, our novel Surrogate TrainingFramework eliminates the dependence on paired event–image datasets,allowing the model to leverage large-scale image datasets for higher ca-pacity. The proposed EvDiff is capable of generating high-quality col-orful videos solely from monochromatic event streams. Experiments onreal-world datasets demonstrate that our method strikes a sweet spotbetween fidelity and realism, outperforming existing approaches on bothpixel-level and perceptual metrics. The code will be released publicly.

ECCV 2026

Estimating Velocity and Spin of Spherical Objects from Rolling-Shutter Image(s)

Wenjie Xue, Jun Yang, Jingmin Wang, Limin Shang

Rolling-shutter cameras introduce characteristic distortionswhen imaging fast moving objects, and these effects are typically treatedas artifacts to be corrected. In this work, we instead leverage rolling-shutter distortions as a valuable source of temporal information to es-timate the 3D translational and angular velocities of rapidly movingspherical objects from a single rolling-shutter frame. We design a robustand easily detectable spherical pattern and propose a correspondence-free formulation that recovers motion by enforcing geometric consis-tency in a back-projection framework. By exploiting the geometry ofthe sphere, translational and rotational motions are decoupled and esti-mated through a two-stage optimization process, enabling reliable veloc-ity recovery even for textureless objects. Extensive experiments on bothsynthetic and real datasets demonstrate accurate and robust estimationof motion parameters under challenging high-speed conditions.

ECCV 2026

EgoVITA: Learning to Plan and Verify for Egocentric Video Reasoning

Yogesh Kulkarni, Pooyan Fazli

Egocentric video understanding requires procedural reasoning under partial observability and continuously shifting viewpoints. Current multimodal large language models (MLLMs) struggle with this setting, often generating plausible but visually inconsistent or weakly grounded responses. We introduce EgoVITA, a framework that decomposes egocentric video reasoning into a structured plan-then-verify process. The model first generates an egocentric plan: a causal sequence of anticipated actions from a first-person perspective. This plan is then evaluated by an exocentric verification stage that uses third-person reasoning over the same video to verify its spatiotemporal and logical consistency, without exocentric video input. This decomposition enables cross-perspective feedback without requiring paired ego-exo supervision. To train this reasoning process, we adopt Group Relative Policy Optimization (GRPO) with two dense reward signals: one that grounds anticipated actions in subsequent visual observations and another that reinforces consistent third-person verification. EgoVITA achieves state-of-the-art performance on egocentric reasoning benchmarks, outperforming Qwen2.5-VL-7B by +7.7 on EgoBlind and +4.4 on EgoOrient, while maintaining strong generalization on exocentric video tasks with only 52k training samples.

ECCV 2026

EgoTraj: Real-World Egocentric Human Trajectory

Ahmad Yehia, Abduallah Mohamed, Tianyi Wang, Kun Qian, Jiseop Byeon, Junfeng Jiao, Christian Claudel

Accurately forecasting human trajectories from an egocen-tric perspective plays a central role in applications such as humanoidrobotics, wearable sensing systems, and assistive navigation. However,progress in this direction remains limited due to the scarcity of egocen-tric trajectory datasets collected in real-world environments. Address-ing this need, we introduce EgoTraj, an egocentric multimodal opendataset recorded using Meta Quest Pro (MQPro). EgoTraj contains 75sequences of human navigation collected from multiple MQPro wearersin real-world urban environments. Each recording provides synchronizedRGB video along with ground-truth data, including 6-degree-of-freedomhead poses, per-frame 3D gaze vectors, and scene annotations. To thebest of our knowledge, EgoTraj differs from typical egocentric trajectorydatasets by capturing long-horizon, self-chosen pedestrian navigation onconsumer AR headset across urban routes with broad participant diver-sity. We benchmark several state-of-the-art trajectory prediction modelsand ablate the contributions of gaze, scene, and motion cues. The resultshighlight the utility of EgoTraj for AR-based perception, navigation, andassistive systems. The EgoTraj dataset, code, and EgoViz Dashboard arepublicly available at https://github.com/yehiahmad/EgoTraj.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.