ECCV 2026 Papers

Existing methods for instruction-guided image editing are usually confined to text-based CoT reasoning for semantic understanding and reasoning, and lack of visual awareness necessary for fine-grained synthesis. In this work, we propose CORE-V, a novel Visual CoT framework to explicitly perform visually-interactive reasoning process for controllable image editing. Specifically, CORE-V refines global semantics into detailed entities and generates visual imagination images based on attribute description in the instructions. These visual imagination images, together with spatial conditions of input images (i.e., segmentation masks, depth maps and edge maps) are applied within the reasoning process. To achieve visual reasoning training, we build the CORE-Edit-800K dataset that contains both reference images and spatial controls aligned with reasoning pipelines. Furthermore, we introduce CORE-Bench to assess directional editing similarity. Comprehensive experiments on EmuEdit, Reason-Edit and CORE-Bench demonstrate superior performance of CORE-V for visual-semantic alignment and structural integrity.

Large-scale video diffusion models (VDMs) deliver strong generation performance, but full fine-tuning for downstream tasks incurs prohibitive computational costs. Existing parameter-efficient finetuning (PEFT) methods have two critical flaws on billion-scale models: they still require substantial trainable parameters, and reward-based training suffers from noise-induced optimization instability in conditionguided tasks. We propose MagicPrompt, a lightweight framework that achieves extreme parameter efficiency and stable reward optimization. It first adopts Attention-Embedded Prompt Tuning, which steers generation via lightweight soft prompts with orders of magnitude fewer parameters while preserving pre-trained knowledge. It further introduces Dual-Space Reward Feedback Optimization, which uses self-supervised latent objectives to improve condition-guided reward training. Experiments show MagicPrompt reaches competitive performance with less than 1% trainable parameters and notably reduces training costs.

Video diffusion models have achieved remarkable visual fi-delity, yet often struggle to adhere to basic physical constraints, suchas support, motion consistency, and object permanence, leading to im-plausible dynamics even in short clips. We revisit this challenge throughthe lens of the Information Bottleneck principle, arguing that physicallygrounded video generation should prioritize time-predictive representa-tions over high-entropy appearance variations. Building on this perspec-tive, we propose Latent Dynamics Optimization (LDO), a post-trainingframework that leverages a predictive latent world model (V-JEPA 2)to guide a pretrained video diffusion model towards dynamics-consistentgeneration without altering its sampling process. LDO operates on twolevels: (1) aligning Gram-matrix geometry of intermediate diffusion fea-tures with the latent world model’s casually predicted dynamics, and(2) encouraging temporally consistent rollouts with Group-Relative Pol-icy Optimization (GRPO) using the world model as a dynamics critic.Experiments on VideoPhy and PisaBench show that LDO substantiallyimproves physical commonsense, object permanence, and trajectory fi-delity while preserving visual quality, suggesting that predictive latentsupervision offers a practical route to make video generators not onlyphotorealistic but also physically legible.

Differentiable vector graphics have enabled powerful gradientbased optimization of vector primitives directly from raster images. However, existing frameworks formulate this as a flat optimization problem, forcing hundreds to thousands of randomly initialized curves to blindly compete for pixel-level error reduction. This disordered optimization leads to topology collapse, where macroscopic structures are distorted by internal high-frequency noise, resulting in a redundant and uneditable “polygon soup” that limits practical editability. To address this limitation, we propose Vector Scaffolding, a novel hierarchical optimization framework that shifts from flat pixel-matching to structured topological construction tailored for vector graphics. By identifying a key cause of topology collapse as the mathematical imbalance between area and boundary gradients, we introduce Interior Gradient Aggregation to stabilize the learning dynamics of multi-scale curve mixtures. Upon this stabilized landscape, we employ Progressive Stratification and Rapid Inflation Scheduling to progressively densify vector primitives with extremely high learning rates (×50). Experiments demonstrate that our approach accelerates optimization by 2.5× while simultaneously improving PSNR by up to 1.4dB over the previous state of the art.

While Multimodal Large Language Models (MLLMs) demonstrate superior generalization in fundamental video tasks, restricted context windows limit their long video understanding. To accommodate this constraint, models typically resort to keyframe selection. However, uniform sampling or static query-guided selection often overlooks critical temporal context, failing to adapt to the varying query temporal granularities. In this paper, we propose ReMem, a temporal granularityadaptive keyframe selection framework for training-free LongVideoQA. ReMem introduces a dual-level memory-augmented adaptation. At the query level, Memory-Driven Question Parsing leverages LLM longterm memory to decode question temporal granularity and extract semantic entities. At the video level, Synergistic Dual-Semantic Frame Alignment exploits intrinsic structural memory to align frames with query semantics, guiding Structure-Aware Dynamic Frame Routing to cluster events and optimally distribute sampling budgets. By explicitly preserving temporal information with memory mechanisms, ReMem suppresses redundancy and empowers MLLMs to perform robust multi-granular video reasoning. Evaluations across four popular LongVideoQA benchmarks using three MLLMs demonstrate highly efficient, state-of-the-art zero-shot performance; notably, LLaVA-Video with ReMem reaches 54.5% (+12.3%) on LVBench and 67.1% (+8.2%) on LongVideoBench. Code is available at ReMem.

Recent open-weight text-to-image (T2I) diffusion models still struggle with multi-instance prompts, often omitting or merging instances and mixing semantics among similar objects. We trace these failures to early denoising steps, before instance boundaries are reliably stabilized. Existing training-free guidance is largely driven by cross-attention or other token-conditioned semantic signals. Such guidance can separate concepts at the token level, but largely assumes that distinct instance regions have already emerged. In early denoising steps, it cannot reliably carve out these regions, so count failures and semantic mixing persist. By contrast, self-attention exposes class-agnostic instance layouts during early denoising. To exploit this asymmetry, we propose ISAC (Instance-to-Semantic Attention Control), a training-free, model-agnostic objective that first stabilizes self-attention layouts and then binds cross-attention semantics within them, without fine-tuning or external vision models. Across T2I-CompBench, HRS-Bench, and our newly curated IntraCompBench, ISAC consistently outperforms prior training-free methods. Furthermore, ISAC enhances layout-to-image controllers by refining coarse, overlapping bounding boxes into dense instance masks. Code and IntraCompBench are available at https://shjo-april.github.io/ISAC.

Gaussian splatting (GS) has garnered significant attention in VR/AR and digital content creation due to its explicit parameterization and efficient rendering capabilities. However, existing GS-based methods for deformable objects face two key limitations: (i) illumination is erroneously baked into textures, causing physically inconsistent responses under dynamic deformations and lighting changes; (ii) snapshot-based reconstruction restricts post-reconstruction material editing. To address these challenges, we propose Deformable and Relightable GS (DR-GS), a unified Gaussian framework that integrates physically-based inverse rendering, relighting, and deformation-aware manipulation. Through explicitly disentangling geometry, illumination, and material representations, DR-GS overcomes the limitations of static snapshots, resolving unrealistic appearance under varying conditions while enabling postreconstruction parameter editing. Extensive experiments show that DRGS achieves leading visual quality across static reconstruction, dynamic deformation, and relighting, reliably preserving reflections and specular highlights on glossy surfaces. It further establishes a fully decoupled geometry-illumination-material pipeline, enabling high-quality 3D asset creation and comprehensive post-editing.

Color constancy algorithms estimate and remove the effect of the illuminant from a captured image, enabling consistent color perception under varying illumination conditions. Learning-based methods achieve high accuracy but tend to be biased toward the spectral characteristics of training cameras. Traditional algorithms such as white-patch Retinex are computationally efficient and camera-independent, but underperform when scene statistics deviate from their assumptions. Therefore, in this paper, we propose a hybrid approach that combines classical assumptions with learned features. Our model processes images in a block-based manner through two complementary branches. The backbone branch extracts scene features and identify reliable regions for illuminant estimation. Guided by these saliency maps, the statistics branch computes per-block illumination priors using a soft maximum formulation that provides a differentiable approximation to the classical white-patch Retinex. Features from both branches are fused and refined through multi-head self-attention, which enables blocks to exchange information across the image, addressing the aggregation bias common to block-based methods. Experiments on five benchmarks demonstrate that our model achieves state-of-the-art performance with a compact architecture (∼1.1 MB) that requires no camera-specific calibration or metadata.

Visual feedback is critical for motor skill acquisition in sports and rehabilitation, and psychological studies show that observing nearperfect versions of one’s own performance accelerates learning more effectively than watching expert demonstrations alone. We propose to enable such personalized feedback by automatically editing a person’s motion to reflect higher skill. Existing motion editing approaches are poorly suited for this setting because they assume paired input-output data—rare and expensive to curate for skill-driven tasks—and explicit edit guidance at inference. We introduce ExpertEdit, a framework for skill-driven motion editing trained exclusively on unpaired expert video demonstrations. ExpertEdit learns an expert motion prior with a masked language modeling objective that infills masked motion spans with expert-level refinements. At inference, novice motion is masked at skill-critical moments and projected into the learned expert manifold, producing localized skill improvements without paired supervision or manual edit guidance. Across eight diverse techniques and three sports from Ego-Exo4D [15] and Karate Kyokushin [52], ExpertEdit outperforms state-of-the-art supervised motion editing methods on multiple metrics of motion realism and expert quality. Project page and benchmark available at: https://vision.cs.utexas.edu/projects/expert_edit/

This paper presents DualCamCtrl, a novel architecture forcamera-controlled video generation that internalizes geometric reason-ing into the diffusion process. Existing methods rely solely on ray-basedcamera conditioning, which entangles appearance and geometry model-ing within a single representation, or resort to external 3D cache andmulti-stage pipelines that are hard to jointly optimize. DualCamCtrladdresses this by generating depth as a co-evolving modality alongsideRGB through a dual-branch framework, enabling geometry and appear-ance to inform each other throughout denoising. To govern this cross-modal interaction, we propose the SemantIc Guided Mutual Alignment(SIGMA) mechanism, which schedules RGB–depth fusion in a stage-aware manner based on our empirical finding that depth and cameraposes exert asymmetric influence across the denoising trajectory. To-gether with a two-stage training strategy, these designs enable DualCam-Ctrl to disentangle and jointly evolve appearance and geometry, gener-ating videos that faithfully adhere to specified camera trajectories. Ex-tensive experiments demonstrate that DualCamCtrl achieves more con-sistent camera-controlled video generation with over 40% reductionon camera rotation errors compared with prior methods. Project Page:https://soyouthinkyoucantell.github.io/dualcamctrl-page.

Scene Coordinate Regression (SCR) methods are increasingly adopted for visual localization. In these approaches, the scene is implicitly encoded within a neural network that regresses a 3D world coordinate for each image pixel. Because the scene is represented only through the network parameters and not stored explicitly as images or maps, such methods are often assumed to be privacy-preserving. In this work, we show that this assumption is incorrect in practice. Specifically, we introduce a query-based attack that reconstructs the 3D geometry of the training environment from an SCR model under different levels of model access. To do so, we repeatedly query the model with batches of proxy images unrelated to the target scene to obtain dense pixel-wise 3D coordinates. Reliable points are identified through their stability under small input perturbations and can be further refined in a white-box setting. These stable points are accumulated across independent query batches to recover the scene geometry. From the recovered 3D representation, we also invert the network features to synthesize images from arbitrary viewpoints, revealing additional appearance information. Experiments on indoor and outdoor datasets demonstrate that substantial portions of training environments can be reconstructed with high geometric fidelity. Beyond geometry, we also recover an approximate color appearance, which exposes recognizable layout and potentially sensitive scene elements. This directly contradicts claims in the literature that SCR representations are privacy-preserving by design, and reveals a real risk when such systems are deployed in private or security-critical spaces. The project page is available here.

Extracting routable road networks from satellite imagery requires accurate topology recovery beyond pixel-level segmentation. Recent methods decompose the task into keypoint detection and connectivity prediction, but reliably inferring road connectivity under structural ambiguity remains challenging. Overpasses introduce non-planar crossings that can create false shortcuts, and occlusions break visual continuity, requiring long-range structural reasoning. We formulate road extraction as fully unobserved connectivity inference and construct a global but sparse Euclidean graph from detected keypoints to better leverage the long-term context. To improve link prediction, we transform this graph into its line graph and perform reasoning using a Graph Transformer. We propose an end-to-end pipeline that integrates vision-based segmentation, sparse graph construction, and structured inference. Our method explicitly models overpasses and uses topology-preserving vertex extraction to mitigate routing-critical errors. This approach achieves state-of-the-art performance on City-scale, SpaceNet, and Global-scale benchmarks in topology metrics including TOPO-F1 and APLS. In addition to quantitative gains, our method improves reconstruction of multi-level road structures crucial for real-world routing reliability. The code is available at: https://github.com/wzzzzzzy/LineGraph2Road.

3D Gaussian Splatting (3DGS) achieves impressive rendering quality but degrades drastically under sparse views. Through optimization manifold analysis, we reveal that sparse supervision produces anisotropic loss surfaces where rendering quality collapses sharply along three geometric fragility axes: position shifts, rotation/scale ill-conditioning, and opacity instability. Guided by this analysis, we propose Stable 3DGS (StableGS), which steers optimization toward isotropic stability via geometric probing. Unlike the generic, uniform smoothing of methods like SAM, which cannot fit 3DGS, StableGS uses domain-specific geometric probes to target the splatting operator’s unique instabilities. Our approach comprises: (1) Attribute-Space Probing that varies each Gaussian’s attributes based on Hessian-informed sensitivity analysis to measure rendering fragility, then minimizes loss at probe points; (2) Viewpoint-Space Probing that samples geometrically critical poses and enforces consistency between base and probed configurations. StableGS significantly advances state-of-the-art (SOTA) performance across multiple benchmarks, achieving substantial improvements such as +0.64dB PSNR on DTU [14] with 74% anisotropy reduction in optimization manifold. Code is available at https://github.com/zyl123456aB/StableGS.

Action anticipation, i.e., predicting future actions from past video observations, is fundamental to intelligent systems that assist humans. Despite progress in model architectures, current evaluation practices exhibit critical limitations: methods evaluate exclusively in closedset settings where the training and test action vocabularies are identical, thereby preventing an understanding of generalization to novel action classes encountered in real-world deployment. We therefore introduce the first open-vocabulary evaluation framework for action anticipation, where models are trained on one egocentric dataset and tested on entirely different egocentric datasets with novel action vocabularies. Since our thorough evaluation shows that adapting existing approaches to this task is insufficient, we propose a novel approach that employs horizonspecific learnable queries and a lightweight text encoder adaptation for open-vocabulary long-term action anticipation. It substantially outperforms other approaches that we have adapted to this task.

Leveraging Multimodal Large Language Models (MLLMs) has become pivotal for advancing Universal Multimodal Embeddings (UME) in addressing diverse cross-modal tasks. Recent studies demonstrate that incorporating generative Chain-of-Thought (CoT) reasoning can substantially enhance task-specific representations compared to discriminative methods. However, the generated reasoning CoTs of existing generative embedding methods are limited to the textual analysis of queries and are irrelevant to the retrieval of the targets. To address these limitations, we propose a reasoning-driven UME framework that integrates Embedder-Guided Reinforcement Learning (EG-RL) to optimize the Reasoner to produce evidential Traceability CoT (T-CoT). Our key contributions are threefold: (1) We design an EG-RL framework where the Embedder provides explicit supervision to the Reasoner, ensuring the generated CoT traces are aligned with embedding tasks. (2) We introduce T-CoT, which extracts critical multimodal cues to focus on retrieval-relevant elements and provides multimodal inputs for the Embedder. (3) With limited computational resources, our framework outperforms the pioneering embedding model on both MMEB-V2 and UVRB benchmarks. The integration of multimodal evidence in structured reasoning, paired with retrieval-oriented alignment, effectively strengthens cross-modal semantic consistency and boosts the model’s fine-grained matching capability as well as its generalization across complex scenarios. Our work demonstrates that targeted reasoning optimization can significantly improve multimodal embedding quality, providing a practical and efficient solution for reasoning-driven UME development. Project page.

3D reconstruction from a lengthy video stream input poses a dilemma for feed-forward reconstruction models (FFRMs), that a whole-stream inference context cannot be retained under limited GPU memory. Recent studies seek to resolve this problem via a trade-off between the integrity of inference context and GPU memory usage, which either suffer from a rapid memory inflation or degraded context integrity due to artificially capping memory usage. Driven by our key observation that the initial saliency of a token reliably dictates its long-term importance across the stream, we propose RegVGGT, a training-free token regulation method which aggressively regulates the tokens of incoming frames. By admitting at most 1% of tokens per frame to update the context memory, our method dramatically suppresses memory inflation as the stream progresses. Equipped with a FlashAttention-compatible token saliency estimation scheme, RegVGGT is capable of processing thousands of frames on a consumer-grade GPU with negligible compromise to reconstruction quality. Extensive experiments demonstrate that RegVGGT achieves state-of-the-art performance on long-horizon benchmarks across diverse FFRM prediction tasks, surpassing prior FFRMbased stream reconstruction baselines by a large margin.

Detection transformer models, including DETR and its extensions, learn to output a set of object-level embeddings that can be simultaneously decoded into 2D bounding boxes and class distributions. In this paper, we investigate what pre-trained 2D detection transformers understand about the 3D properties of objects. Specifically, we investigate the extent to which properties including the depth of objects from the camera and the 3D location of objects relative to the camera can be recovered from object-level embeddings using linear and non-linear probes. Across a range of detection transformer models, our results show a surprisingly strong and previously unknown ability of 2D DETR models to represent useful information about the 3D properties of objects, despite the complete lack of 3D supervision during model pre-training.

Video anomaly detection is critical for public safety and security, yet remains highly challenging despite extensive research due to large variations in appearance, viewpoint, and scene dynamics. Among existing approaches, human pose-based methods have emerged as a major line of research, showing strong performance since many anomalies in public datasets involve humans and pose representations are robust to appearance changes while providing compact motion descriptions. However, these methods often overlook bounding-box trajectories, although such information is inherently available in pose-based pipelines. In this paper, we explicitly leverage these trajectories as a primary anomaly cue. We present TrajVAD, a framework that models multi-class boundingbox trajectories using normalizing flows to learn normal kinematic patterns. Its trajectory-only variant, TrajVAD-T, eliminates pose estimation, reaches 87.7 AP on ShanghaiTech, and achieves the best results on MSAD among compared methods. TrajVAD-P adds a reliability-gated pose branch and improves performance to 88.6 AUROC and 90.9 AP on ShanghaiTech, establishing bounding-box trajectories as an effective modality for video anomaly detection.

Video temporal grounding (VTG) aims to localize events in untrimmed videos given natural language prompts. Current VTG methods directly inherit the paradigm from 2D object detection, applying rigid IoU-based one-to-one matching between queries and ground truth spans. However, this paradigm fundamentally conflicts with the temporal fluidity of video events: unlike spatial objects, a sub-event (moment) can be semantically aligned with the prompt despite only covering part of the full event span. This matching scheme incorrectly suppresses such moment predictions as negatives, preventing models from learning rich temporal representations. To resolve this, we propose Moment-Event DETR (ME-DETR), a novel framework that embraces the natural part-whole structure of events through dynamic query specialization. During training, after standard Hungarian matching assigns primary event queries to ground truths, we identify other high-confidence predictions within each matched span and designate them as auxiliary moment queries. Through our synergistic supervision strategy, these moment queries are liberated from unreasonable suppression to explore semantically rich sub-events (moments). Extensive experiments demonstrate that ME-DETR establishes new state-of-the-art results, achieving +2.43% mAP improvement on QVHighlights test set without requiring post-processing NMS. Code: https://github.com/zouyuda220/ME-DETR.