ECCV 2026 论文列表

Recently, Multimodal Large Language Models (MLLMs) have demonstrated significant potential in complex visual tasks through the integration of Chain-of-Thought (CoT) reasoning. However, in Video Question Answering, extended thinking processes do not consistently yield performance gains and may even lead to degradation due to “visual anchor drifting”, where models increasingly rely on self-generated text, sidelining visual inputs and causing hallucinations. While existing mitigations typically introduce specific mechanisms for the model to reattend to visual inputs during inference, these approaches often incur prohibitive training costs and suffer from poor generalizability across different architectures. To address this, we propose FrameRepeat, an automated enhancement framework which features a lightweight repeat scoring module that enables Video-LLMs to autonomously identify which frames should be reinforced. We introduce a novel training strategy, Add-One-In (AOI), that uses MLLM output probabilities to generate supervision signals representing repeat gain. This can be used to train a frame scoring network, which guides the frame repetition behavior. Experimental results across multiple models and datasets demonstrate that FrameRepeat is both effective and generalizable in strengthening important visual cues during the reasoning process.

Text-to-image diffusion models are increasingly aligned with human preferences through preference optimization (PO) with learned reward models. We observe that the CLIP-based reward models used in this pipeline encode demographic biases, and that PO amplifies them: after alignment, the demographic distribution of generated images narrows significantly compared to the base model. We show that this degradation is progressive—race entropy drops steadily throughout training— and that existing diversity-preservation methods such as DRIFT and DiverseGRPO do not resolve it, because visual diversity and demographic fairness are distinct properties. To address this, we propose DRPO, which identifies demographic directions in the reward feature space via linear probing and removes them through orthogonal projection before reward computation. DRPO requires no demographic annotations during training and is compatible with any PO method. Experiments on Stable Diffusion and SDXL with six baselines demonstrate that DRPO maintains generation quality while substantially restoring the demographic balance lost during alignment.

Unified Multimodal Models (UMMs) excel in text-to-image generation and editing but often entangle multimodal generative reasoning with high-fidelity visual synthesis. We introduce Query-Kontext, a novel approach that bridges a Vision-Language Model (VLM) and a diffusion model via multimodal “kontext” tokens. This design cleanly decouples the complex reasoning delegated to the VLM, including instruction understanding, grounding, and identity preservation, from the high-quality visual rendering executed by the diffusion model. We propose a three-stage progressive training strategy: (1) connecting the VLM to a lightweight diffusion head to activate generative reasoning; (2) scaling to a large, pre-trained diffusion model to enhance visual realism; and (3) incorporating a low-level image encoder for fine-grained instruction tuning. Supported by a comprehensive multi-task dataset, extensive experiments demonstrate that Query-Kontext matches or outperforms state-of-theart task-specific and unified methods across diverse reference-to-image scenarios.

Autoregressive text-to-image generation has recently achieved remarkable progress, offering high-fidelity synthesis via a unified generative framework. However, fine-grained semantic control remains challenging due to the attribute entanglement and the misalignment between textual and fine-grained visual representations. In this paper, we introduce Attribute Token Arithmetic (ATA), a method that enables disentangled and continuous attribute control in visual autoregressive modelling. Inspired by the vector arithmetic property observed in word embeddings, ATA identifies semantic directions corresponding to visual attributes (e.g., aging, fatness, emotion) directly within the pretrained autoregressive latent space. These directions are learned from a single reference image, without model retraining or large-scale supervision. During generation, attributes can be continuously adjusted and compositionally combined through simple arithmetic operations with other attribute tokens. Extensive experiments demonstrate that ATA achieves identitypreserving, fine-grained, and multi-attribute adjustment, outperforming existing autoregressive editing baselines in controllability, generality, and computational efficiency. Our code will be released here.

Feature tracking plays a fundamental role in understanding scene motion and supports various downstream tasks. Event cameras, with their high temporal resolution and asynchronous sensing, enable low-latency and motion-robust perception, making them well-suited for feature tracking under fast and non-linear motion. However, existing event-based feature tracking methods rely on fixed heuristic rules based on hand-tuning for event accumulation. Such strategies fail to adapt to diverse motion dynamics, leading to degraded performance under abrupt motion changes or low-motion scenarios. In this paper, we model event accumulation as a sequential decision-making problem and introduce reinforcement learning (RL) framework to adaptively control the accumulation process for online event-based feature tracking. Our approach trains a RL agent that decides whether to continue accumulating events or to perform tracking inference based on motion cues. The proposed adaptive temporal agent enables dynamic adaptation to varying motion patterns without relying on hand-crafted rules. Furthermore, we introduce a Dynamic Event-based Tracking (DEFT) dataset with dynamic motion distributions to evaluate the robustness of the feature tracking. Extensive experiments demonstrate that integrating our plug-and-play framework to existing feature tracking methods consistently outperforms heuristic-based approaches, improving robustness under dynamic motion while offering a better balance between tracking accuracy and efficiency. Our project codes and datasets are available at https://github.com/kmax2001/GoSTOP.

Reasoning about objects in the physical world requires understanding both the spatial structure of a scene and the properties of objects within it. To perform reasoning tasks, a model must identify candidate objects from the scene and then reason over their properties, such as material, physical attributes, affordance, and function to satisfy the task constraints. We formalize this reasoning process as Perceptual Taxonomy, a hierarchical framework that organizes visual understanding into structured scene–object–property representations. Existing benchmarks, however, address only parts of this framework: some focus on spatial understanding alone, while others evaluate individual property families—material, affordance, or physical attributes—each under a separate schema. They fail to cover a challenging yet practically important capability: cross-property reasoning, where a model must simultaneously integrate reasoning across all property families to answer questions about physical environments. To evaluate this capability, we introduce PercepTax, a benchmark for perceptual taxonomy reasoning. We annotate 3,173 object classes with 54 fine-grained attributes across four property families: material, physical attributes, affordance, and function. The benchmark contains 5,802 images from both synthetic and real domains and 28,033 questions spanning object description, singleproperty recognition, 3D spatial reasoning, and cross-property reasoning. Experiments show that state-of-the-art vision language models (VLMs) perform strongly on object description and single-property tasks, but accuracy drops substantially on cross-property reasoning that requires integrating multiple attributes simultaneously. Oracle experiments with ground-truth objects’ properties with 3D positions and taxonomy-guided in-context learning both improve performance substantially yet remain well below human levels, confirming that both visual perception of individual properties and their cross-property integration are bottlenecks for VLMs.

We propose a self-supervised framework designed to enhance the capability of scene text detectors in identifying and recognizing text in scenarios where instances are shown at significant distances, typically small, blurred, and frequently missed by conventional models. Our approach leverages the high-fidelity performance of existing text spotting models on large, clear text as a foundational supervisor. By temporally back-tracking these high-confidence detections through video sequences, we automatically synthesize pseudo-labels for preceding frames where the distant text is still visually degraded or undersized. These pseudolabels enable training a student model specialized for early text detection, without requiring any manual annotation. The success of this approach depends on accurate pseudo-label generation, for which we develop a dedicated scene text tracker capable of maintaining consistent text identities across challenging video sequences. In addition, we propose SceneText50, a diverse multilingual outdoor dataset to facilitate training and evaluation. Experiments show that our framework significantly improves early detection accuracy and robustness across varied scenes and languages. Code and data are at https://github.com/trid2912/BackTrackingText.

Single-image dehazing aims to recover a clear scene from a hazy image and is generally formulated as an image-to-image translation task; however, it faces two limitations. Its performance depends heavily on the haze-formation priors embedded in the model. Prevailing methods adopt the Atmospheric Scattering Model (ASM), whose assumptions of single scattering and homogeneous media are often violated, leading to residual haze and color drift. Moreover, large-scale real hazy/clear pairs are impractical to collect, and existing synthesis approaches fail to reproduce the full complexity of natural haze. To address these issues, we present RTE-FM-Dehazer, a novel dehazing approach, together with a scalable data pipeline. Unlike the ASM, the Radiative Transfer Equation (RTE) jointly accounts for both scattering and absorption, naturally accommodating the non-homogeneous, multiple-scattering media that characterize real hazy scenes. Motivated by the structural similarity between the RTE diffusion-absorption term and the ODE in flow matching, we introduce a diffusion-absorption regularizer derived from a reduced RTE, to steer the flow matching trajectory at each step. Next, leveraging modern vision–language models, we build an automated pipeline and release P-HAZE, a dataset of 50000 realistic hazy/clear pairs. Extensive evaluations demonstrate that RTE-FM-Dehazer, trained solely on P-HAZE, effectively eliminates artifacts like residual haze and color drift, exhibits strong cross-domain generalization, and achieves leading results on five real-world dehazing benchmarks. Code and data are available at https://github.com/vincentweikey/RTE-FM-Dehazer.

Interactive video generation systems for camera-controlled world exploration roll out growing sequences of latent video frames, entangling state transition with high-frequency observation synthesis. We propose Walking in the Implicit, a scene-centric paradigm that changes the rollout variable from frame latents to a fixed-length, renderable implicit state, termed Neural Implicit Scene (NIS). This factorizes interactive generation into stochastic transition of a compact scene state and deterministic pose-conditioned rendering given the sampled state. We instantiate this paradigm as NeuWorld: a transformer VAE learns locally anchored NIS from sparse posed frames, and a diffusion transformer evolves NIS conditioned on future camera trajectories and geometryaware retrieved history. By reusing the VAE encoder as a unified conditioner, NeuWorld maps camera, reference-image, and history cues into the same NIS modality, avoiding external heterogeneous encoders. Trained from scratch on public posed-view data without pretrained video backbones or auxiliary 3D reconstructors, NeuWorld achieves strong longhorizon consistency with favorable inference efficiency.

We introduce LATTE (Linearized Attention with Tunable Taylor Series Expansion), a training-free attention linearization method that selectively applies linearization to a subset of self-attention blocks in a pretrained vision transformer. LATTE approximates the softmax function via a second-order Taylor expansion centered on tunable anchor points, thereby enabling a hybrid architecture that seamlessly integrates standard self-attention with linear attention. This method preserves model performance without requiring any fine-tuning or retraining but only minimal calibration on the target dataset. LATTE is highly scalable, linearizing more than half of the attention layers retains more than 95% of the original model’s performance and reduces attention GFLOPs by up to 15× (e.g., from 16.384 to 1.082 at T=1000 for the CLIP attention block). We evaluate LATTE across diverse benchmarks, including image classification, object detection, and vision–language multimodal tasks, demonstrating consistent superiority over current state-of-the-art training-free linearization methods. Code will be released here.

Autoregressive (AR) streaming models have emerged as a powerful paradigm for long video generation. However, the linearly growing Key-Value (KV) cache poses a significant bottleneck, leading to memory overload and degraded inference throughput. A common compression method is to drop redundant KV tokens, which often breaks long-range dependencies, resulting in temporal flickering and identity loss. In this paper, we propose Instance-Specific Parametric Absorption (ISPA), a novel framework that shifts the KV cache compression from discarding to distilling. The core idea is to transit a subset of layers from FullAttention (F-Layers) to memory-efficient Local-Attention (L-Layers) by "absorbing" historical context into the model’s weights. Specifically, during a brief warmup phase, ISPA monitors the output discrepancy between global and local attention. At the transition point, we solve a closed-form least-squares problem to compute an instance-specific weight modulation that compensates for the missing history. Experiments across architectures (1.3B to 14B) demonstrate that ISPA can remove up to 50% of the KV cache with near-lossless visual quality. We hope this perspective encourages future work to explore parametric memory consolidation beyond external token-level cache management for streaming generative models.

Reliable detection of sharp features in point clouds—loci where the surface-normal field is discontinuous—remains challenging under increased local density variation and noise. SFD-Net couples a compact multi-scale Local Geometric Descriptor (LGD) with an enhanced PointNet++ backbone. LGD aggregates second-moment statistics of surface normal differences at three nested neighborhood scales, yielding rigid-motionand scale-invariant cues that remain stable under joint noise and spacing variability. On the ABC benchmark, SFD-Net achieves state-of-the-art F1-score and competitive False Positive Rate (FPR) across four noise conditions under a consistent evaluation setup. LGD further improves competitive detectors in a plug-and-play manner without architectural changes, and transfer to Stanford Large-Scale 3D Indoor Spaces (S3DIS) confirms generalization to real scans. Downstream applications potentially benefit from more reliable feature localization. Our source code and pretrained models are publicly available at https: //github.com/inyoungoh-cde/SFD-Net.

Existing video personalization methods preserve visual likeness but treat video and audio separately. Without access to the visual scene, audio models cannot synchronize sounds with on-screen actions; and because classical voice-cloning models condition only on a reference recording, a text prompt cannot redirect speaking style or acoustic environment. Although prompt-conditioned audio models could offer such control, they lack access to the visual scene. We propose ID-LoRA (Identity-Driven In-Context LoRA), which jointly generates a subject’s appearance and voice in a single model, letting a text prompt, a reference image, and a short audio clip govern both modalities together. ID-LoRA adapts the LTX-2 joint audio-video diffusion backbone via parameter-efficient In-Context LoRA and, to our knowledge, is the first method to personalize visual appearance and voice within a single generative pass. Two challenges arise from this formulation. Reference and generation tokens share the same positional-encoding space, making them hard to distinguish; we address this with negative temporal positions, which place reference tokens in a disjoint region of the RoPE space while preserving their internal temporal structure. Furthermore, speaker characteristics tend to be diluted during denoising; we introduce identity guidance, a classifier-free guidance variant that amplifies speaker-specific features by contrasting predictions with and without the reference signal. In human preference studies ID-LoRA is preferred over Kling 2.6 Pro, the leading commercial unified model with voice personalization capabilities, by 73% of annotators for voice similarity and 65% for speaking style. Automatic metrics confirm these gains: on crossenvironment settings, speaker similarity improves by 24% over Kling, with the gap widening as reference and target conditions diverge. A preliminary user study further suggests that joint generation provides a useful inductive bias for physically grounded sound synthesis. ID-LoRA achieves these results with only ∼3K training pairs on a single GPU. Code, models, and data are available at https://id-lora.github.io.

Current Vision-Language-Action (VLA) paradigms in autonomous driving primarily rely on Imitation Learning (IL), which introduces inherent challenges such as distribution shift and causal confusion. Online Reinforcement Learning offers a promising pathway to address these issues through trial-and-error learning. However, applying online reinforcement learning to VLA models in autonomous driving is hindered by inefficient exploration in continuous action spaces. To overcome this limitation, we propose MindDrive, a VLA framework comprising a large language model (LLM) with two distinct sets of LoRA parameters. The one LLM serves as a Decision Expert for scenario reasoning and driving decision-making, while the other acts as an Action Expert that dynamically maps linguistic decisions into feasible trajectories. By feeding trajectory-level rewards back into the reasoning space, MindDrive enables trial-and-error learning over a finite set of discrete linguistic driving decisions, instead of operating directly in a continuous action space. This approach effectively balances optimal decision-making in complex scenarios, human-like driving behavior, and efficient exploration in online reinforcement learning. Extensive experiments validate the efficacy of our online reinforcement learning framework, which outperforms state-of-the-art IL and offline Reinforcement Learning methods on the challenging Bench2Drive benchmark. To the best of our knowledge, this is the first work to demonstrate the effective application of online reinforcement learning to VLA models in autonomous driving.

We present LlamaSeg, a visual autoregressive framework that unifies multiple image segmentation tasks via natural language instructions. By reformulating segmentation as visual generation, LlamaSeg encodes masks as visual tokens and uses a LLaMA-style Transformer for direct next-token prediction, naturally fitting segmentation into autoregressive architectures. To support large-scale training, we introduce a data annotation pipeline and construct the SA-OVRS dataset, which contains 2M segmentation masks annotated with over 5,800 open vocabulary labels or diverse textual descriptions, spanning diverse realworld scenarios. This enables our model to localize objects in images based on text prompts and to generate fine-grained masks. We further introduce the composite metric average Hausdorff Distance (d ) AHD to evaluate mask contour fidelity for generative models better. Experiments show that LlamaSeg consistently outperforms existing generative approaches on multiple segmentation benchmarks and delivers finer, more accurate segmentation masks. Code and dataset are available at https://github.com/GML-FMGroup/llamaseg.

Large vision-language models have endowed GUI agents with strong general capabilities for interface understanding and interaction. However, due to insufficient exposure to domain-specific software operation data during training, these agents exhibit significant domain bias— they lack familiarity with the specific operation workflows (planning) and UI element layouts (grounding) of particular applications, limiting their real-world task performance. In this paper, we present GUIDE (GUI Unbiasing via Instructional-Video Driven Expertise), a trainingfree, plug-and-play framework that resolves GUI agent domain bias by autonomously acquiring domain-specific expertise from web tutorial videos through a retrieval-augmented automated annotation pipeline. Guide introduces two key innovations. First, a subtitle-driven Video-RAG pipeline unlocks video semantics through subtitle analysis, performing progressive three-stage retrieval—domain classification, topic extraction, and relevance matching—to identify task-relevant tutorial videos. Second, a fully automated VLM pairwise video annotation pipeline feeds consecutive keyframes enhanced with UI element detection into VLMs, inferring the required planning and grounding knowledge that are injected into the agent’s corresponding modules to address both manifestations of domain bias. Extensive experiments on OSWorld demonstrate Guide’s generality as a plug-and-play component for both multi-agent systems and single-model agents. It yields +4.47–+7.48 percentage-point improvements and reduces execution steps—without modifying any model parameters or architecture—validating Guide as an architecture-agnostic enhancement to bridge GUI agent domain bias. Code, dataset, and additional results are available on our project page: https://sharryXR. github.io/GUIDE/.

Visual token pruning is a crucial strategy for accelerating VLMs by compressing redundant image patches, yet existing methods often fail to preserve critical cues under dense instructions and fine-grained queries. In this paper, we investigate this failure and identify two underlying bottlenecks: the widespread dispersion of textual noise that corrupts dense cross-modal scoring, and the feature fragmentation inherent to standard token selection. To address these issues, we propose Entropy-Aware Dense Pruning (EADP), a framework that reformulates pruning as a structured compression problem. EADP first leverages statistical entropy to quantify and filter out textual noise, yielding a robust, fine-grained instruction relevance score. Subsequently, instead of naive Top-K selection, EADP casts token selection as a submodular maximization problem with a spatial prior, explicitly ensuring a holistic and non-redundant visual representation. Extensive experiments demonstrate that EADP improves the accuracy-efficiency trade-off of VLMs, robustly preserving fine-grained visual cues under strict token budgets while achieving SoTA performance on challenging multimodal benchmarks.

Image Quality Assessment (IQA) is a long-standing problem in computer vision. Previous methods typically focus on predicting numerical scores without explanation or provide low-level descriptions lacking precise scores. Recent reasoning-based vision language models (VLMs) have shown strong potential for IQA, enabling joint generation of quality descriptions and scores. However, we notice that existing VLM-based IQA methods tend to exhibit unreliable reasoning due to their limited capability of integrating visual and textual cues. In this work, we introduce Zoom-IQA, a VLM-based IQA model to explicitly emulate key cognitive behaviors: uncertainty awareness, region reasoning, and iterative refinement. Specifically, we present a two-stage training pipeline: 1) supervised fine-tuning (SFT) on our Grounded-RationaleIQA (GR-IQA) dataset to teach the model to ground its assessments in key regions; and 2) reinforcement learning (RL) for dynamic policy exploration, primarily stabilized by our KL-Coverage regularizer to prevent reasoning and scoring diversity collapse, and supported by a Progressive Re-sampling Strategy to mitigate annotation bias. Extensive experiments show that Zoom-IQA achieves improved robustness, explainability, and generalization. The application to downstream tasks, such as image restoration, further demonstrates the effectiveness of Zoom-IQA.

Editing images with pre-trained text-to-image flow models typically requires carefully balancing target alignment with the desired prompt and source consistency with the original image. Existing approaches either rely on inversion-based pipelines or heuristic sourceto-target trajectory constructions, which often depend on architecturespecific designs or are sensitive to hyperparameters. In this paper, we propose h-flow, a training-free and theoretically grounded flow-based editing framework. Inspired by Doob’s h-Transform, we reformulate image editing as conditional generation under multiple terminal events corresponding to source consistency and target alignment. We first extend the classical h-Transform from SDE-based models to the deterministic RF framework by constructing an equivalent SDE with identical marginals. Within this formulation, we design dedicated h-functions for source consistency and target alignment, yielding closed-form reconstruction guidance and velocity-based semantic editing signals. We further introduce a velocity orthogonal decomposition to decouple reconstruction and editing directions, enabling a controllable trade-off between the two objectives. Extensive experiments demonstrate that h-flow achieves effective, robust, and flexible editing across diverse scenarios.