ECCV 2026 Papers

Recent world-model-based Vision-Language-Action (VLA)architectures have improved robotic manipulation through predictive vi-sual foresight. However, dense future prediction introduces visual redun-dancy and accumulates errors, causing long-horizon plan drift. Mean-while, recent sparse methods typically represent visual foresight usinghigh-level semantic subtasks or implicit latent states. These representa-tions often lack explicit kinematic grounding, weakening the alignmentbetween planning and low-level execution. To address this, we proposeStructVLA, which reformulates a generative world model into an explicitstructured planner for reliable control. Instead of dense rollouts or seman-tic goals, StructVLA predicts sparse, physically meaningful structuredframes. Derived from intrinsic kinematic cues (e.g., gripper transitionsand kinematic turning points), these frames capture spatiotemporal mile-stones closely aligned with task progress. We implement this approachthrough a two-stage training paradigm with a unified discrete token vo-cabulary: the world model is first trained to predict structured frames andsubsequently optimized to map the structured foresight into low-level ac-tions. This approach provides clear physical guidance and bridges visualplanning and motion control. In our experiments, StructVLA achievesstrong average success rates of 75.0% on SimplerEnv-WidowX and 94.8%on LIBERO. Real-world deployments further demonstrate reliable taskcompletion and robust generalization across both basic pick-and-placeand complex long-horizon tasks.

Embodied Instruction Following requires agents to executenatural-language tasks via holistic perception and sound planning. Cur-rent LLM-driven agents suffer from two primary bottlenecks: SpatialMyopia, where limited egocentric views hinder global topological under-standing, and Granularity Imbalance, where planning is polarized be-tween inefficient atomic actions that lack interpretability and ambiguoussubgoals that fail to align with low-level control. To address these is-sues, we propose STEP, a framework that empowers agents with SpatialThinking and Egocentric Pointing. For perception, STEP integrates Hy-brid Map-Egocentric Perception, fusing multi-view streams into Bird’s-Eye-View maps to maintain both global context and local visual cues.For planning, STEP introduces Point-Level Planning, which derives spa-tial waypoints through explicit reasoning chains, achieving highly inter-pretable, intermediate-granularity execution. We also design STEP-CoT,a scalable data engine, to generate a 400k Chain-of-Thought datasetthat aligns diverse modalities—views, maps, actions, and instructions—through reasoning chains. Results on ALFRED and AI2-THOR-Nav,along with real-world deployments, demonstrate STEP’s superior perfor-mance and generalization. Ultimately, STEP achieves a unified paradigm:Look at the View, Think with the Map, and Point to the Goal.

Diffusion models produce high-quality images, but their de-ployment remains expensive because generation requires many denoisingsteps through large transformer or U-Net backbones. Existing methodsimprove efficiency through post-training quantization or timestep reduc-tion, but they optimize these two axes separately. Quantization methodsusually assume a fixed precision, timestep reduction methods assume afixed schedule, and both often rely on proxy costs such as BitOps insteadof real hardware measurements. We introduce DiffPro, a post-trainingframework that jointly optimizes per-layer quantization precision and thedenoising schedule under a unified budget defined by measured memoryand latency on real hardware. DiffPro combines three key ideas: (i) acomposite sensitivity metric that combines PCA dimensionality withdiagonal Hessian curvature for bit allocation, (ii) Dynamic ActivationQuantization (DAQ), which adapts activation scales across timesteps tohandle temporal activation drift, and (iii) a drift-guided timestep selectorthat removes redundant steps while preserving the late refinement phase.On DiT-XL/2 (ImageNet 256 × 256), DiffPro achieves 50% step reduction,7.9× compression, and 2.7× speedup at FID 5.89 (∆FID 3.62), improvingover uniform thinning by 2.5 FID at matched precision. On SDXL U-Net(COCO 2014 1024 × 1024), DiffPro achieves 5.6× compression with FID26.50 (∆FID 2.0) and 45% energy reduction, all without retraining.

Multi-view visual reasoning is essential for intelligent systems that must understand complex environments from sparse and discrete viewpoints, yet existing research has largely focused on single-image or temporally dense video settings. In real-world scenarios, reasoning across views requires integrating partial observations without explicit guidance, while collecting large-scale multi-view data with accurate geometric and semantic annotations remains challenging. To address this gap, we leverage physically grounded simulation to construct diverse, high-fidelity 3D scenes with precise per-view metadata, enabling scalable data generation that remains transferable to real-world settings. Based on this engine, we introduce VIEW2SPACE, a multi-dimensional benchmark for sparse multi-view reasoning, together with a scalable, disjoint training split supporting millions of grounded question–answer pairs. Using this benchmark, a comprehensive evaluation of state-of-theart vision–language and spatial models reveals that multi-view reasoning remains largely unsolved, with most models performing only marginally above random guessing. We further investigate whether training can bridge this gap. Our proposed Grounded Chain-of-Thought with Visual Evidence substantially improves performance under moderate difficulty, and generalizes to real-world data, outperforming existing approaches in cross-dataset evaluation. We further conduct difficulty-aware scaling analyses across model size, data scale, reasoning depth, and visibility constraints, indicating that while geometric perception can benefit from scaling under sufficient visibility, deep compositional reasoning across sparse views remains a fundamental challenge. Code and benchmark are available online3.

While 3D Gaussian Splatting (3DGS) provides an efficientand explicit representation for novel view synthesis, enforcing stylisticcoherence across viewpoints remains challenging. Existing 3D stylizationmethods typically apply 2D feature-matching losses independently perrendered view, which leads to unstable style allocation, many-to-one fea-ture reuse, and limited cross-view consistency. We propose a capacity-controlled framework for multi-view stylization of 3DGS, grounded inoptimal transport. Specifically, we reformulate local style matching as asemi-balanced optimal transport problem. By introducing explicit column-capacity constraints with tunable strength, our formulation mitigatesmany-to-one matching and enables controllable allocation of style fea-tures. This transport-based objective provides a principled mechanismfor balancing feature coverage and stylistic diversity while maintainingstable correspondences across viewpoints. To further enhance cross-viewcoherence, we incorporate a novel cross-view matching guidance to con-strain correspondences between scene content and style patterns. In ad-dition, we introduce several geometric regularizations to enhance thevanilla 3DGS, thereby enabling optimized Gaussian primitives to rep-resent finer-grained textures during stylization. Extensive experimentsdemonstrate that our approach significantly improves multi-view stylis-tic consistency and produces stable, expressive 3D stylizations while pre-serving the core semantic structure of the scene.

Recent progress in multimodal reasoning has enabled agentsthat interpret imagery, connect it with language, and execute struc-tured analytical tasks. Extending these capabilities to remote sensingremains challenging, as models must reason over spatial scale, geographicstructures, and multispectral indices while maintaining coherent multi-step logic. To address this gap, we introduce OpenEarthAgent, a unifiedframework for tool-augmented geospatial reasoning trained on satelliteimagery, natural-language queries, and structured reasoning traces. Be-yond serving as a benchmark, OpenEarthAgent establishes a cohesiveagentic architecture built around a unified executable tool registry andtrajectory-based policy learning. The framework standardizes heteroge-neous visual, spectral, GIS, and georeferenced raster operations undera consistent callable schema, enabling modular orchestration and deter-ministic execution. Training is performed via supervised fine-tuning onstructured reasoning trajectories with deterministic replay validation toensure executability and spatial correctness. The accompanying corpuscomprises 14,538 training and 1,169 evaluation instances with over 107Kreasoning steps, spanning urban, environmental, disaster, and infrastruc-ture domains and incorporating GIS operations alongside index analysessuch as NDVI, NBR, and NDBI. Grounded in explicit reasoning traces,the learned agent demonstrates structured reasoning, stable spatial un-derstanding, and interpretable tool-driven behavior across diverse EOscenarios. We report consistent improvements over a strong baseline andcompetitive performance against recent open and closed-source models.https://github.com/mbzuai-oryx/OpenEarthAgent.

Composed visual retrieval (CVR) aims to locate a targetimage or video that reflects a user’s modification of a reference visualinput. Prior works typically design separate models for images and videos,resulting in fragmented paradigms, limited reasoning diversity, and pooradaptability to diverse user intents. We introduce OCTOPUS, a novelmulti-agentic assistant for tool-integrated progressive self-improvementand user-friendly synergistic compositional retrieval. OCTOPUS featuresthree cooperative agents – a Perceiver, a Creator, and a Retriever – thatcollectively emulate a human-like process of perception, imagination, andreflection. The Perceiver interprets composed queries, formulates seman-tic instructions, and refines them through self-reflection. The Creatoremploys visual imagination and object-level reasoning tools to generatediverse textual and visual proxies that capture missing or ambiguoussemantics. The Retriever performs bidirectional retrieval and applies anevaluation mechanism to ensure alignment between results and user intent.Comprehensive experiments on five benchmarks demonstrate that OCTO-PUS consistently outperforms both training-free and supervised baselines.The code is available at https://github.com/zbzzbzzbz/OCTOPUS.

Foundation models (FMs) have driven significant progressin computational pathology. These models can easily exceed a billionparameters and produce high-dimensional embeddings, thus limiting theirapplicability for research or clinical use when computing resources aretight. Here we introduce Pathryoshka, a novel multi-teacher distillationframework inspired by agglomerative models and Matryoshka representa-tion learning to reduce pathology FM sizes while allowing for adaptableembedding dimensions. We evaluate our framework with a distilled modelon ten public pathology benchmarks with varying downstream tasks.Compared to its much larger teachers, Pathryoshka reduces the modelsize by 86-92% at on-par performance. It outperforms state-of-the-artsingle-teacher distillation models of comparable size by a median marginof 7.0 and other pathology multi-teacher distillation by 5.3 percentagepoints in accuracy. By enabling efficient deployment without sacrificingaccuracy or representational richness, Pathryoshka democratizes accessto state-of-the-art pathology FMs for the broader research and clinicalcommunity.

3D affordance grounding aims to understand how diverseobjects can be manipulated, making it a cornerstone of embodied inter-action. However, prior works struggle to generalize to out-of-distribution,open-world scenarios, leaving a critical gap between limited dataset per-formance and real-world application needs. Inspired by the saying: “WhatI can not create, I do not understand”, we find generative modelscan generate semantically valid HOI images, which indicates inherent en-coding of affordance concepts. Building on this insight, we propose DAG,the first innovative diffusion-based 3D affordance grounding frameworkthat extracts general affordance knowledge from text-to-image diffusionmodels for 3D affordance prediction. Specifically, we extract the affor-dance priors from a diffusion model to encode HOI priors, and designan affordance block with a multi-source affordance decoder for dense3D affordance prediction. Extensive experiments show that DAG con-sistently outperforms state-of-the-art methods and exhibits strong open-world generalization, even in the challenging one-shot setting. The codeof our method is released on https://github.com/hq-King/DAG.

We introduce iMED, the first synchronized multi-endoscopedataset for robot-assisted minimally invasive surgery (RAMIS), designedto address a fundamental limitation in surgical vision benchmarking: theabsence of independent held-out viewpoints. Existing surgical datasetscapture single-trajectory sequences from a single endoscope, making itimpossible to distinguish geometric generalization from photometric in-terpolation along a narrow forward-facing path. iMED provides 340 se-quences (≈170K synchronized timepoints with 4 views per timepoint)recorded simultaneously from two independent stereo endoscopes acrossex vivo, postmortem, and live surgical settings, spanning 14 specimenswith diverse anatomical regions and motion regimes. The dataset in-cludes calibrated camera intrinsics, ArUco-based frame-wise pose esti-mates with uncertainty quantification, instrument segmentation masks,and rich clinical metadata. Using a train-on-one-endoscope, test-on-anotherevaluation protocol, we benchmark 23 state-of-the-art methods acrossrigid and deformable novel view synthesis, pose estimation, feature match-ing, and monocular depth estimation. Our experiments show that meth-ods relying primarily on photometric supervision degrade substantiallyunder this held-out-endoscope setting, while methods with explicit ge-ometric regularization are more robust across views in our benchmark.Notably, large-scale foundation models transfer surprisingly well to sur-gical imagery, often outperforming domain-specific models, suggestingpitfalls in current fine-tuning protocols. iMED establishes a new evalua-tion protocol for geometric generalization in surgical vision. Dataloaderand dataset links can be found at github.com/surgical-vision/imed.

Ever since the advent of foundation models and the pre-training–finetuning paradigm, there have been numerous efforts to mergemultiple task-specific experts into a single multi-task model. Prior worklargely focuses on finding a single merged model, but it often under-performs individual experts due to parameter interference. To resolvethis, dynamic model merging employs routing to activate task-relevantparameters per input. However, existing routers typically require eitheradditional training with abundant labeled datasets or assume the accessto task IDs of each input at inference time. In this work, we aim toclose the gap to expert performance without additional training or task-ID-access assumption. To this end, we formulate routing as training-freetask classification for each test input. Using singular value decomposition(SVD)-based low-rank manifold approximations for each task, SiM scorestasks by the projection residual of the test input feature onto each taskmanifold and routes accordingly. The task manifolds are pre-computableoffline from a pretrained backbone using a small per-task support set(e.g., 32 examples per task) prior to merging process, requiring no routertraining and no data during the merging process. Moreover, SiM inte-grates seamlessly with subspace-/mask-based merging that representstask-expert via lightweight compressed task vectors, avoiding the needto store full expert parameters. Experiments across computer vision andnatural language processing benchmarks under task-unknown inferencedemonstrate that SiM substantially improves merged-model performanceand consistently narrows the gap to individual task experts. Our code isavailable at https://github.com/BAIKLAB/SiM

Video-to-audio (V2A) generation aims to synthesize realis-tic audio that is both semantically consistent with and temporally syn-chronized to a silent video. Despite recent progress, many methods stillrely on multi-stage training, resulting in high computational costs andlong runtimes, or transform visual input into text to leverage pretrainedtext-to-audio models, sacrificing fine-grained temporal cues. To over-come these limitations, we propose Flowley, an end-to-end, single-stagetraining architecture that produces soundtracks by combining visual fea-tures with textual prompts. Crucially, we introduce Progressive Soft-masked Cross-Attention, which embeds audio-visual synchronization di-rectly within its attention mechanism, adding zero additional computa-tional cost compared to standard attention layers. We further observethat existing V2A benchmarks lack sound-oriented descriptive captions,which can potentially degrade the quality of the synthesized audio. Toremedy this, we propose SoundCap, a plug-and-play pipeline for creatingdetailed, sound-aware captions that guide the model. Remarkably, with-out integrating any pretrained audio-visual alignment modules, Flow-ley achieves state-of-the-art performance on VGGSound across multiplemetrics. Moreover, by incorporating SoundCap, we further exceed theperformance of the strongest existing close-sourced methods in terms ofaudio quality in the zero-shot setting.

Recent studies indicate that test-time scaling (TTS) or re-flection mechanisms can substantially improve performance in multi-modal generative tasks. At the core of TTS lies verification, which as-sesses the semantic alignment between textual instructions and generatedimages. Such reflection inherently couples generation and understanding,its integration into a single unified model for visual comprehension andsynthesis remains largely unexplored. Existing methods typically dependon external Visual Large Language Models (VLLMs) for verification, re-sulting in additional computational overhead and fragmented pipelines.We introduce UniReflect, the first unified framework for visual un-derstanding and generation that performs in-model verificationvia self-reflection tuning. The proposed model is explicitly trained to as-sess the correspondence between the prompt and the generated image,produce a structured analysis, and output a special token that guides thedecoding of a refined image, which enables seamless TTS within a singleunified model, eliminating the need for external verifiers or extra edit-ing prompts. Extensive experiments demonstrate that UniReflect notonly advances image generation fidelity but also provides reliable visualverification, setting a new benchmark for unified multi-modal models.

Autoregressive video diffusion models have demonstrated re-markable progress, yet they remain bottlenecked by intractable linearKV-cache growth, temporal repetition, and compounding errors duringlong-video generation. To address these challenges, we present Pack-Forcing, a unified framework that efficiently manages the generationhistory through a novel three-partition KV-cache strategy. Specifically,we categorize the historical context into three distinct types: (1) Sinktokens, which preserve early anchor frames at full resolution to maintainglobal semantics; (2) Mid tokens, which achieve a massive spatiotempo-ral compression (∼32× token reduction) via a dual-branch network fusingprogressive 3D convolutions with low-resolution VAE re-encoding; and(3) Recent tokens, kept at full resolution to ensure local temporal co-herence. To strictly bound the memory footprint without sacrificing qual-ity, we introduce a dynamic top-k context selection mechanism for themid tokens, coupled with a continuous Temporal RoPE Adjustment thatseamlessly re-aligns position gaps caused by dropped tokens with negli-gible overhead. Empowered by this principled hierarchical context com-pression, PackForcing can generate coherent 2-minute, 832×480 videosat 14–16 FPS on a single H200 GPU. It achieves a bounded KV cacheenables a remarkable 24× temporal extrapolation (5 s → 120 s), oper-ating effectively trained on merely 5-second clips. Extensive results onVBench demonstrate state-of-the-art temporal consistency (26.07) anddynamic degree (56.25), proving that short-video supervision is sufficientfor high-quality, long-video synthesis.

Evolutions in the world, such as water pouring or ice melt-ing, happen regardless of being observed. Video world models generate“worlds” via 2D frame observations. Can these generated “worlds” evolveregardless of observation? To probe this question, we design a benchmarkto evaluate whether video world models can decouple state evolutionfrom observation. Our benchmark, StEvo-Bench, applies observationcontrol to evolving processes via instructions of occluder insertion, turn-ing off the light, or specifying camera “lookaway” trajectories. By evalu-ating video models with and without camera control for a diverse set ofnaturally-occurring evolutions, we expose their limitations in decouplingstate evolution from observation. StEvo-Bench proposes an evaluationprotocol to automatically detect and disentangle failure modes of videoworld models across key aspects of natural state evolution. Analysis ofStEvo-Bench results provide new insight into potential data and ar-chitecture bias of present-day video world models. Benchmark website:https://glab-caltech.github.io/STEVOBench/.

Defocus blur degrades fine image structures and limits vi-sual perception, which can adversely affect downstream vision tasks. Al-though recent deep learning deblurring methods have achieved strongperformance, their effectiveness depends on training data and often de-grades across cameras and lenses due to limited optical diversity and re-alism in existing datasets. In this paper, we propose a pipeline for synthe-sizing realistic defocus deblurring datasets for diverse compound lenses.It integrates efficient wave-optics PSF computation via Debye CZT prop-agation, depth-aware defocus rendering with occlusion handling, and blursynthesis in the radiometrically linear space with camera ISP simulation.This unified pipeline enables the scalable generation of photorealisticdefocus datasets with diverse lens characteristics. Using our pipeline,we generate CLDefocus, a large-scale synthetic dataset containing lens-diverse defocus image pairs. We further analyze the limitations of real-captured defocus datasets and show that such imperfections can biasfull-reference evaluation. Extensive experiments demonstrate that mod-els trained on CLDefocus achieve improved cross-device generalizationcompared to models trained on existing real and synthetic datasets. Codeand dataset are available at: https://github.com/lykelee/CLDefocus.

Recent advances in neural rendering have established 3DGaussian Splatting (3DGS) as a highly efficient representation for novelview synthesis, enabling fast training and real-time rendering with strongfidelity. However, when supervision is limited to sparse input views,3DGS tends to overfit to the observed images and generalize poorlyto unseen viewpoints. We address this challenge from the perspectiveof flat minima (FM) optimization, which seeks solutions that remainstable under small parameter perturbations. Viewing Gaussian parame-ters as trainable weights, we adapt FM principles to the geometric anddynamic nature of 3DGS with a lightweight training framework. Ourmethod regularizes optimization with controlled Gaussian perturbationsthat account for each Gaussian’s anisotropy and the training progress,preserving fine details while improving robustness to sparse-view overfit-ting. To further stabilize this flat minima optimization process, we intro-duce periodic reinitialization, which temporarily returns non-positionalparameters to their initial states for a short window. Together, thesetechniques integrate seamlessly into existing 3DGS pipelines without ar-chitectural changes. Experiments on LLFF and Mip-NeRF360 datasetsdemonstrate improved quantitative metrics and perceptual quality un-der sparse-view supervision, producing reconstructions that are sharper,more stable, and better generalized to novel viewpoints.

Despite the practical relevance of sparse decision-based black-box threats, they have received limited attention in semantic segmenta-tion. To bridge this gap, we adapt the most representative decision-basedblack-box sparse attacks from the classification domain to serve as base-lines, establishing a rigorous benchmark for this underexplored setting.In this context, we demonstrate that one of the existing methods suf-fers from severe query inefficiency due to its image-centric pixel accu-mulation, which rapidly exhausts query budgets across the vast imagespace. To overcome this, we propose SegPAR, a novel decision-basedframework that shifts to a class-centric exploration paradigm. Further-more, to eliminate the misleading feedback generated by standard de-cision rewards during pixel accumulation, we introduce a novel discrep-ancy reward. Extensive experiments show that SegPAR significantly out-performs black-box baselines in sparsity efficiency and MIoU reduction,while remaining competitive with white-box sparse attacks. Code is avail-able at https://github.com/KAU-QuantumAILab/SegPAR.

We consider the problem of reconstructing a H × W × 31 hy-perspectral image from a H × W grayscale snapshot measurement thatis captured using only a single diffractive lens and a filterless panchro-matic photosensor. This problem is severely ill-posed, but we presenta model that produces high-quality results in simulation and experi-ment. We make efficient use of limited training data by creating a con-ditional denoising diffusion model that operates on small patches in ashift-invariant manner. During inference, we synchronize per-patch hy-perspectral predictions using guidance by physical consistency with thesystem’s optical point spread function. Our experiments reveal that thepatch size can be as small as the point spread function, with local opticalcues being the main source of information about complete spectra. Also,by drawing multiple samples, our model provides per-pixel uncertaintyestimates that strongly correlate with reconstruction error.