ECCV 2026 论文列表

Foundation models (FMs) have driven significant progressin computational pathology. These models can easily exceed a billionparameters and produce high-dimensional embeddings, thus limiting theirapplicability for research or clinical use when computing resources aretight. Here we introduce Pathryoshka, a novel multi-teacher distillationframework inspired by agglomerative models and Matryoshka representa-tion learning to reduce pathology FM sizes while allowing for adaptableembedding dimensions. We evaluate our framework with a distilled modelon ten public pathology benchmarks with varying downstream tasks.Compared to its much larger teachers, Pathryoshka reduces the modelsize by 86-92% at on-par performance. It outperforms state-of-the-artsingle-teacher distillation models of comparable size by a median marginof 7.0 and other pathology multi-teacher distillation by 5.3 percentagepoints in accuracy. By enabling efficient deployment without sacrificingaccuracy or representational richness, Pathryoshka democratizes accessto state-of-the-art pathology FMs for the broader research and clinicalcommunity.

3D affordance grounding aims to understand how diverseobjects can be manipulated, making it a cornerstone of embodied inter-action. However, prior works struggle to generalize to out-of-distribution,open-world scenarios, leaving a critical gap between limited dataset per-formance and real-world application needs. Inspired by the saying: “WhatI can not create, I do not understand”, we find generative modelscan generate semantically valid HOI images, which indicates inherent en-coding of affordance concepts. Building on this insight, we propose DAG,the first innovative diffusion-based 3D affordance grounding frameworkthat extracts general affordance knowledge from text-to-image diffusionmodels for 3D affordance prediction. Specifically, we extract the affor-dance priors from a diffusion model to encode HOI priors, and designan affordance block with a multi-source affordance decoder for dense3D affordance prediction. Extensive experiments show that DAG con-sistently outperforms state-of-the-art methods and exhibits strong open-world generalization, even in the challenging one-shot setting. The codeof our method is released on https://github.com/hq-King/DAG.

We introduce iMED, the first synchronized multi-endoscopedataset for robot-assisted minimally invasive surgery (RAMIS), designedto address a fundamental limitation in surgical vision benchmarking: theabsence of independent held-out viewpoints. Existing surgical datasetscapture single-trajectory sequences from a single endoscope, making itimpossible to distinguish geometric generalization from photometric in-terpolation along a narrow forward-facing path. iMED provides 340 se-quences (≈170K synchronized timepoints with 4 views per timepoint)recorded simultaneously from two independent stereo endoscopes acrossex vivo, postmortem, and live surgical settings, spanning 14 specimenswith diverse anatomical regions and motion regimes. The dataset in-cludes calibrated camera intrinsics, ArUco-based frame-wise pose esti-mates with uncertainty quantification, instrument segmentation masks,and rich clinical metadata. Using a train-on-one-endoscope, test-on-anotherevaluation protocol, we benchmark 23 state-of-the-art methods acrossrigid and deformable novel view synthesis, pose estimation, feature match-ing, and monocular depth estimation. Our experiments show that meth-ods relying primarily on photometric supervision degrade substantiallyunder this held-out-endoscope setting, while methods with explicit ge-ometric regularization are more robust across views in our benchmark.Notably, large-scale foundation models transfer surprisingly well to sur-gical imagery, often outperforming domain-specific models, suggestingpitfalls in current fine-tuning protocols. iMED establishes a new evalua-tion protocol for geometric generalization in surgical vision. Dataloaderand dataset links can be found at github.com/surgical-vision/imed.

Ever since the advent of foundation models and the pre-training–finetuning paradigm, there have been numerous efforts to mergemultiple task-specific experts into a single multi-task model. Prior worklargely focuses on finding a single merged model, but it often under-performs individual experts due to parameter interference. To resolvethis, dynamic model merging employs routing to activate task-relevantparameters per input. However, existing routers typically require eitheradditional training with abundant labeled datasets or assume the accessto task IDs of each input at inference time. In this work, we aim toclose the gap to expert performance without additional training or task-ID-access assumption. To this end, we formulate routing as training-freetask classification for each test input. Using singular value decomposition(SVD)-based low-rank manifold approximations for each task, SiM scorestasks by the projection residual of the test input feature onto each taskmanifold and routes accordingly. The task manifolds are pre-computableoffline from a pretrained backbone using a small per-task support set(e.g., 32 examples per task) prior to merging process, requiring no routertraining and no data during the merging process. Moreover, SiM inte-grates seamlessly with subspace-/mask-based merging that representstask-expert via lightweight compressed task vectors, avoiding the needto store full expert parameters. Experiments across computer vision andnatural language processing benchmarks under task-unknown inferencedemonstrate that SiM substantially improves merged-model performanceand consistently narrows the gap to individual task experts. Our code isavailable at https://github.com/BAIKLAB/SiM

Video-to-audio (V2A) generation aims to synthesize realis-tic audio that is both semantically consistent with and temporally syn-chronized to a silent video. Despite recent progress, many methods stillrely on multi-stage training, resulting in high computational costs andlong runtimes, or transform visual input into text to leverage pretrainedtext-to-audio models, sacrificing fine-grained temporal cues. To over-come these limitations, we propose Flowley, an end-to-end, single-stagetraining architecture that produces soundtracks by combining visual fea-tures with textual prompts. Crucially, we introduce Progressive Soft-masked Cross-Attention, which embeds audio-visual synchronization di-rectly within its attention mechanism, adding zero additional computa-tional cost compared to standard attention layers. We further observethat existing V2A benchmarks lack sound-oriented descriptive captions,which can potentially degrade the quality of the synthesized audio. Toremedy this, we propose SoundCap, a plug-and-play pipeline for creatingdetailed, sound-aware captions that guide the model. Remarkably, with-out integrating any pretrained audio-visual alignment modules, Flow-ley achieves state-of-the-art performance on VGGSound across multiplemetrics. Moreover, by incorporating SoundCap, we further exceed theperformance of the strongest existing close-sourced methods in terms ofaudio quality in the zero-shot setting.

Recent studies indicate that test-time scaling (TTS) or re-flection mechanisms can substantially improve performance in multi-modal generative tasks. At the core of TTS lies verification, which as-sesses the semantic alignment between textual instructions and generatedimages. Such reflection inherently couples generation and understanding,its integration into a single unified model for visual comprehension andsynthesis remains largely unexplored. Existing methods typically dependon external Visual Large Language Models (VLLMs) for verification, re-sulting in additional computational overhead and fragmented pipelines.We introduce UniReflect, the first unified framework for visual un-derstanding and generation that performs in-model verificationvia self-reflection tuning. The proposed model is explicitly trained to as-sess the correspondence between the prompt and the generated image,produce a structured analysis, and output a special token that guides thedecoding of a refined image, which enables seamless TTS within a singleunified model, eliminating the need for external verifiers or extra edit-ing prompts. Extensive experiments demonstrate that UniReflect notonly advances image generation fidelity but also provides reliable visualverification, setting a new benchmark for unified multi-modal models.

Autoregressive video diffusion models have demonstrated re-markable progress, yet they remain bottlenecked by intractable linearKV-cache growth, temporal repetition, and compounding errors duringlong-video generation. To address these challenges, we present Pack-Forcing, a unified framework that efficiently manages the generationhistory through a novel three-partition KV-cache strategy. Specifically,we categorize the historical context into three distinct types: (1) Sinktokens, which preserve early anchor frames at full resolution to maintainglobal semantics; (2) Mid tokens, which achieve a massive spatiotempo-ral compression (∼32× token reduction) via a dual-branch network fusingprogressive 3D convolutions with low-resolution VAE re-encoding; and(3) Recent tokens, kept at full resolution to ensure local temporal co-herence. To strictly bound the memory footprint without sacrificing qual-ity, we introduce a dynamic top-k context selection mechanism for themid tokens, coupled with a continuous Temporal RoPE Adjustment thatseamlessly re-aligns position gaps caused by dropped tokens with negli-gible overhead. Empowered by this principled hierarchical context com-pression, PackForcing can generate coherent 2-minute, 832×480 videosat 14–16 FPS on a single H200 GPU. It achieves a bounded KV cacheenables a remarkable 24× temporal extrapolation (5 s → 120 s), oper-ating effectively trained on merely 5-second clips. Extensive results onVBench demonstrate state-of-the-art temporal consistency (26.07) anddynamic degree (56.25), proving that short-video supervision is sufficientfor high-quality, long-video synthesis.

Evolutions in the world, such as water pouring or ice melt-ing, happen regardless of being observed. Video world models generate“worlds” via 2D frame observations. Can these generated “worlds” evolveregardless of observation? To probe this question, we design a benchmarkto evaluate whether video world models can decouple state evolutionfrom observation. Our benchmark, StEvo-Bench, applies observationcontrol to evolving processes via instructions of occluder insertion, turn-ing off the light, or specifying camera “lookaway” trajectories. By evalu-ating video models with and without camera control for a diverse set ofnaturally-occurring evolutions, we expose their limitations in decouplingstate evolution from observation. StEvo-Bench proposes an evaluationprotocol to automatically detect and disentangle failure modes of videoworld models across key aspects of natural state evolution. Analysis ofStEvo-Bench results provide new insight into potential data and ar-chitecture bias of present-day video world models. Benchmark website:https://glab-caltech.github.io/STEVOBench/.

Defocus blur degrades fine image structures and limits vi-sual perception, which can adversely affect downstream vision tasks. Al-though recent deep learning deblurring methods have achieved strongperformance, their effectiveness depends on training data and often de-grades across cameras and lenses due to limited optical diversity and re-alism in existing datasets. In this paper, we propose a pipeline for synthe-sizing realistic defocus deblurring datasets for diverse compound lenses.It integrates efficient wave-optics PSF computation via Debye CZT prop-agation, depth-aware defocus rendering with occlusion handling, and blursynthesis in the radiometrically linear space with camera ISP simulation.This unified pipeline enables the scalable generation of photorealisticdefocus datasets with diverse lens characteristics. Using our pipeline,we generate CLDefocus, a large-scale synthetic dataset containing lens-diverse defocus image pairs. We further analyze the limitations of real-captured defocus datasets and show that such imperfections can biasfull-reference evaluation. Extensive experiments demonstrate that mod-els trained on CLDefocus achieve improved cross-device generalizationcompared to models trained on existing real and synthetic datasets. Codeand dataset are available at: https://github.com/lykelee/CLDefocus.

Recent advances in neural rendering have established 3DGaussian Splatting (3DGS) as a highly efficient representation for novelview synthesis, enabling fast training and real-time rendering with strongfidelity. However, when supervision is limited to sparse input views,3DGS tends to overfit to the observed images and generalize poorlyto unseen viewpoints. We address this challenge from the perspectiveof flat minima (FM) optimization, which seeks solutions that remainstable under small parameter perturbations. Viewing Gaussian parame-ters as trainable weights, we adapt FM principles to the geometric anddynamic nature of 3DGS with a lightweight training framework. Ourmethod regularizes optimization with controlled Gaussian perturbationsthat account for each Gaussian’s anisotropy and the training progress,preserving fine details while improving robustness to sparse-view overfit-ting. To further stabilize this flat minima optimization process, we intro-duce periodic reinitialization, which temporarily returns non-positionalparameters to their initial states for a short window. Together, thesetechniques integrate seamlessly into existing 3DGS pipelines without ar-chitectural changes. Experiments on LLFF and Mip-NeRF360 datasetsdemonstrate improved quantitative metrics and perceptual quality un-der sparse-view supervision, producing reconstructions that are sharper,more stable, and better generalized to novel viewpoints.

Despite the practical relevance of sparse decision-based black-box threats, they have received limited attention in semantic segmenta-tion. To bridge this gap, we adapt the most representative decision-basedblack-box sparse attacks from the classification domain to serve as base-lines, establishing a rigorous benchmark for this underexplored setting.In this context, we demonstrate that one of the existing methods suf-fers from severe query inefficiency due to its image-centric pixel accu-mulation, which rapidly exhausts query budgets across the vast imagespace. To overcome this, we propose SegPAR, a novel decision-basedframework that shifts to a class-centric exploration paradigm. Further-more, to eliminate the misleading feedback generated by standard de-cision rewards during pixel accumulation, we introduce a novel discrep-ancy reward. Extensive experiments show that SegPAR significantly out-performs black-box baselines in sparsity efficiency and MIoU reduction,while remaining competitive with white-box sparse attacks. Code is avail-able at https://github.com/KAU-QuantumAILab/SegPAR.

We consider the problem of reconstructing a H × W × 31 hy-perspectral image from a H × W grayscale snapshot measurement thatis captured using only a single diffractive lens and a filterless panchro-matic photosensor. This problem is severely ill-posed, but we presenta model that produces high-quality results in simulation and experi-ment. We make efficient use of limited training data by creating a con-ditional denoising diffusion model that operates on small patches in ashift-invariant manner. During inference, we synchronize per-patch hy-perspectral predictions using guidance by physical consistency with thesystem’s optical point spread function. Our experiments reveal that thepatch size can be as small as the point spread function, with local opticalcues being the main source of information about complete spectra. Also,by drawing multiple samples, our model provides per-pixel uncertaintyestimates that strongly correlate with reconstruction error.

LangDriveCTRL is a natural-language-controllable framework for editing real-world driving videos to synthesize diverse traffic scenarios. It represents each video as an explicit 3D scene graph, decomposing the scene into a static background and dynamic object nodes. To enable fine-grained editing and realism, it introduces a feedback-driven agentic pipeline. An Orchestrator converts user instructions into executable graphs that coordinate specialized multi-modal agents and tools. An Object Grounding Agent aligns free-form text with target object nodes in the scene graph; a Behavior Editing Agent generates multiobject trajectories from language instructions; and a Behavior Reviewer Agent iteratively reviews and refines the generated trajectories. The edited scene graph is rendered and harmonized using a video diffusion tool, and then further refined by a Video Reviewer Agent to ensure photorealism and appearance alignment. LangDriveCTRL supports both object node editing (removal, insertion, and replacement) and multi-object behavior editing from natural-language instructions. Quantitatively, it achieves nearly 2× higher instruction alignment than the previous SoTA, with superior photorealism, structural preservation, and traffic realism.

While knowledge distillation (KD) is widely adopted for train-ing lightweight models by leveraging supervision from larger teachermodels, relying solely on output token distributions has proven insuf-ficient for compressing Multimodal Large Language Models (MLLMs).Since output tokens are a byproduct of the model attending to visualinputs, prior works have explored explicitly distilling attention to pro-vide a direct supervisory signal. While promising, the precise utility ofwhich attention signals to distill remains under-explored. In this work, wechallenge the conventional reliance on prompt-to-vision attention by re-vealing that downstream performance correlates strongly with response-to-vision attention similarity to the teacher, but negligibly with that ofprompt-conditioned attention. Furthermore, we observe that attentiondistributions exhibit significant variance across individual tokens, indi-cating that a uniform distillation objective is suboptimal. To this end,we introduce Token-level Response-visual Attention Guidance(TRAG), a distillation objective that 1) shifts the focus to response-to-vision signals and 2) employs token-specific objectives by adaptivelyweighting the Kullback-Leibler divergence based on attention entropy, ef-fectively guiding the student to mirror the teacher’s precise visual focus.Extensive experimental results on multiple benchmarks demonstrate thatTRAG significantly outperforms prior distillation baselines. Our code isavailable at https://github.com/jhjangjh/TRAG.

We present PointGT, a point-based 3D representation thatenables simultaneous editing of object geometry and appearance. Ex-isting reconstruction and view synthesis techniques produce volumetric3D representations that are high-quality and photorealistic, but are dif-ficult to edit. In particular, recent efforts to enable texture editing for3D Gaussian Splatting representations are not compatible with geometryedits and deformations. Our method combines a point-based represen-tation that is well-suited for geometry deformations with a learned UVmapping technique that enables high-resolution texture editing. We showthat PointGT enables fine-grained editing of both geometry and texturein point-based neural representations with high rendering quality.

Sign language translation (SLT) remains challenging due to its high spatio-temporal complexity, long sequences, and the need to model multiple articulators without relying on gloss annotations. Existing approaches are typically tailored to individual datasets or languages and struggle to scale, while overlooking the relationships between sign languages that could inform more effective cross-lingual transfer. We present SIGNET, a framework that enables motion-level knowledge transfer for cross-language sign language translation. Our key insight is that, although sign languages differ in grammar and lexicon, pretrained models capture motion-level visual patterns that can be reused across datasets and languages. SIGNET integrates multiple pretrained sign language backbones through an attention-based, hand-prior aggregation mechanism that guides a gated fusion network in dynamically selecting the most relevant experts. Comprehensive experiments on four benchmarks (How2Sign, Phoenix14T, CSL-Daily, and MeineDGS) demonstrate state-of-the-art translation performance, and SIGNET also surpasses prior methods on WLASL for sign language recognition.

Large vision–language models (LVLMs) incur substantial com-putational and memory overhead due to quadratic attention complexityand enlarged key–value (KV) caches. Training-free vision token prun-ing is promising, yet most methods rely on text-to-vision attention-scoremagnitudes, which often entangle instruction-relevant signals with po-sitional bias and visual attention sinks. To address this issue, we pro-pose DIVA, an instruction-aware, layer-wise vision token pruning methodbased on dual-probe attention discrepancy, with two key modules: DIVA-Pruner and VF-Calibrator. DIVA-Pruner selects important tokens us-ing the discrepancy between attention distributions of contextual andnon-contextual probe sets. A lightweight VF-Calibrator calibrates unim-portant tokens that bypass self-attention, mitigating representation mis-match and enabling effective reuse in subsequent LVLM blocks. On di-verse benchmarks and LVLM baselines, DIVA consistently outperformsprior methods across token budgets, even in highly constrained regimes.Notably, under a 97.2% token reduction, DIVA retains 92.6% of baselineperformance on LLaVA-NeXT-7B with a 10.8× reduction in KV cache.

Recent advances in video diffusion models have delivered re-markable generation quality, yet their generation cost remains high. Thisinefficiency is largely driven by the iterative denoising process, with com-putationally expensive per-step architectures. Step distillation enablesfew-step generation, but per-step latency remains high because a large-scale backbone is invoked at each denoising step. Several works seek toreduce per-step latency via block pruning using importance scores aver-aged across denoising steps. However, such averaged scores can obscurestage-critical blocks and bias the pruning process, removing blocks essen-tial for certain stages while leaving redundancy in others. In this paper,we propose a novel pruning paradigm, Stage-Specialized Block Pruning,that partitions the denoising process into multiple stages and assigns atailored pruned variant to each stage. Each variant consists of sharedblocks across multiple stages and stage-specific blocks for its assignedstage. With parameter sharing, the overall parameter budget is smallerthan the original model, while the per-step computational footprint is re-duced for lower latency. To derive stage-specialized variants, we introducea hierarchical selection mechanism consisting of an Inter-Stage VarianceSelector (IVS) and an Intra-Stage Quality Selector (IQS). IVS comparesthe variance of stage-wise block-importance distributions and prioritizesthe stage with higher variance for pruning, indicating greater removableredundancy. IQS mitigates noisy importance estimation by generatingmultiple low-importance pruning candidates within the selected stageand choosing the one with the highest video generation quality. IVS andIQS are applied iteratively until reaching a target latency budget. Fur-thermore, we introduce a two-phase training pipeline to recover the per-formance of these variants and facilitate sequential composition for few-step generation. Extensive experiments on the representative video diffu-sion model Wan2.1-1.3B demonstrate that our method reduces per-stepinference cost while maintaining competitive performance on VBench.

Motion generation for rigged shapes is vital for scalable 4D asset production. However, template-based methods are limited by specific topologies and fail to generalize across diverse morphologies. Conversely, per-case optimization is computationally expensive, susceptible to local optima, and highly sensitive to viewpoint-induced ambiguities. In this paper, we present SkelMo, a diffusion-based framework designed for category-agnostic skeletal animation generation from 2D video guidance. To overcome the scarcity of high-quality training data, we have curated a large-scale dynamic dataset comprising approximately 25,200 diverse 3D animations, each featuring complete textures, skeletal rigging, and a wide array of comprehensive animation sequences. To bridge the kinematic gap between 2D visual motion cues and heterogeneous 3D skeletal structures, we propose a structural-semantic injection mechanism. Our model integrates texture and semantic attributes directly into skeletal joint representations. This allows it to map perceived visual dynamics to specific joint hierarchies and their functional roles. This enables SkelMo to synthesize high-fidelity animations that maintain anatomical consistency across a vast range of unseen categories, from existing biological species to fantastical beings. Extensive experiments demonstrate that our approach significantly outperforms existing methods, setting a new state-of-the-art benchmark for robust and efficient 4D asset generation. Project Page: https://research.davytao.me/skelmo/.