ECCV 2026 论文列表

While foundation models have emerged as general-purposevisual backbones, their representations are primarily optimized for se-mantics and lack explicit modeling of physical factors, such as reflectance,hindering their efficacy in tasks requiring explicit material reasoning. Weintroduce Φeat, a novel material-grounded visual backbone that encour-ages a representation sensitive to material identity, including reflectanceand mesostructure. Instead of relying on generic data augmentations,we pretrain our model by contrasting observations of the same materialunder controlled variations in lighting and geometry. This encouragesinvariance to extrinsic factors while preserving sensitivity to intrinsicmaterial properties. We show that the resulting representation providesstrong priors for material-centric tasks, including feature-based mate-rial selection and classification. Our results demonstrate that physicallyinspired weak supervision is an effective strategy for learning represen-tations tailored to material perception.

Panoramic object detection has made steady progress, butexisting methods remain largely closed-set and therefore cannot recog-nize novel categories in open environments. Extending open-vocabularydetection to panoramas is challenging because prompt-image alignmentis weakened by two factors: semantic ambiguity in language-only promptsand severe geometric distortion caused by equirectangular projection. Wepresent OpenPanoD, a framework for open-vocabulary panoramic de-tection that aligns multimodal prompts with geometry-aware sphericalrepresentations. On the prompt side, a multimodal prompt encoder com-bines language descriptions with visual exemplars to obtain category em-beddings that are both generalizable and visually specific. On the imageside, GeoFormer maps panoramic features onto a quasi-uniform spher-ical grid and performs spherical attention to reduce projection-induceddistortion and boundary discontinuity. The resulting prompt and imageembeddings are fused in a unified detection head for BFoV/RBFoV pre-diction. Experiments on 360-Indoor and PANDORA show that Open-PanoD consistently improves novel-category performance over strongopen-vocabulary and visual-prompt baselines. These results demonstratethe importance of jointly addressing semantic ambiguity and panoramicgeometry for open-vocabulary detection in 360-degree scenes.

Unbounded 3D world generation is emerging as a founda-tional task for scene modeling in computer vision, graphics, and robotics.In this work, we present WorldFlow3D, a novel method capable of gen-erating unbounded 3D worlds. Building upon a foundational property offlow matching – namely, defining a path of transport between two datadistributions – we model 3D generation more generally as a problemof flowing through 3D data distributions, not limited to conditional de-noising. We find that our latent-free flow approach generates causal andaccurate 3D structure, and can use this as an intermediate distribution toguide the generation of more complex structure and high-quality texture– all while converging more rapidly than existing methods. We enablecontrollability over generated scenes with vectorized scene layout condi-tions for geometric structure control and visual texture control throughscene attributes. We confirm the effectiveness of WorldFlow3D on bothreal outdoor driving scenes and synthetic indoor scenes, validating cross-domain generalizability and high-quality generation on real data distri-butions. We confirm favorable scene generation fidelity over approachesin all tested settings for unbounded scene generation.

Vision-as-inverse-graphics, the concept of reconstructing im-ages into editable programs, remains challenging for Vision-LanguageModels (VLMs), which inherently lack fine-grained spatial grounding inone-shot settings. To address this, we introduce VIGA (Vision-as-Inverse-Graphics Agent), an interleaved multimodal reasoning framework wheresymbolic logic and visual perception actively cross-verify each other. VIGAoperates through a tightly coupled code-render-inspect loop: synthesizingsymbolic programs, projecting them into visual states, and inspectingdiscrepancies to guide iterative edits. Equipped with high-level semanticskills and an evolving multimodal memory, VIGA sustains evidence-basedmodifications over long horizons. This training-free, task-agnostic frame-work seamlessly supports 2D document generation, 3D reconstruction,multi-step 3D editing, and 4D physical interaction. Finally, we intro-duce BlenderBench, a challenging visual-to-code benchmark. Empirically,VIGA substantially improves accuracy compared with one-shot base-lines in BlenderGym (35.32% ), SlideBench (117.17% ) and our proposedBlenderBench (124.70% ).

Diffusion models have demonstrated significant applicationsin the field of image generation. However, their high computational andmemory costs pose challenges for deployment. Model quantization hasemerged as a promising solution to reduce storage overhead and acceler-ate inference. Nevertheless, existing quantization methods for diffusionmodels struggle to mitigate outliers in activation matrices during in-ference, leading to substantial performance degradation under low-bitquantization scenarios. To address this, we propose HQ-DM, a novelQuantization-Aware Training framework that applies Single HadamardTransformation to activation matrices. This approach effectively reducesactivation outliers while preserving model performance under quanti-zation. Compared to traditional Double Hadamard Transformation, ourproposed scheme offers distinct advantages by seamlessly supporting INTconvolution operations while preventing the amplification of weight out-liers. For conditional generation on the ImageNet 256×256 dataset usingthe LDM-4 model, our W4A4 and W4A3 quantization schemes improvethe Inception Score by 12.8% and 467.73%, respectively, over the existingstate-of-the-art method.

We introduce Vision Bridge Transformer (ViBT), alarge-scale instantiation of Brownian Bridge Models designed for con-ditional generation. Unlike traditional diffusion models that transformnoise into data, Bridge Models offer an efficient data-to-data translationparadigm by directly modeling the trajectory between inputs and out-puts. We first demonstrate the advantages of this paradigm through acontrolled apples-to-apples comparison against diffusion models, reveal-ing improved efficiency and performance in this setting. Building on this,we successfully scale the Bridge paradigm up to a 20B-parameter modelto tackle complex image and video translation tasks. To enable stabletraining at this scale, we adopt a Transformer architecture equipped witha variance-stabilized velocity matching objective. Together, these ad-vances unlock the potential of scaling Bridge Models, establishing strongperformance for instruction-based image editing and video stylization.

A unified representation for text and vision is a natural pur-suit, as it enables simpler multimodal modeling and more efficient training.However, representing images as discrete signals in the same way as textinevitably introduces severe information loss. Existing work strugglesto balance low-level details and high-level semantics in discrete repre-sentations: reconstruction-oriented representations often lack semanticinformation, whereas semantically stronger features typically suffer fromsevere loss of detail. We present ViQ, a Visual Quantized Representationsframework, which is designed to balance semantics and details in discreterepresentations while supporting inputs at native resolutions, therebyenabling it to serve as a unified and general discrete representation forarbitrary visual inputs. Our approach structures quantization learninginto two stages: text-aligned pre-training and feature discretization. Withtext-aligned pre-training, we enhance the visual encoder semantic-richsupervision from the pretrained language model and enable it to pro-cess native-resolution visual inputs. During discretization, we proposea proximal representation learning strategy to progressively compactthe feature space, along with a position-aware head-wise quantizationmechanism that enables flexible processing of arbitrary resolutions. Ex-tensive experiments on multimodal tasks demonstrate that ViQ achievescompetitive performance compared to state-of-the-art multimodal visionencoders with continuous and high-dimensional visual features, whilemaintaining high precision in low-level reconstruction. We also showthat multimodal training with visual quantized representations largelyimproves efficiency, yielding up to 20%-70% acceleration with differentbase LLMs and training recipes.

Recent advances in Chain-of-Thought reasoning have beenreported to enhance video reasoning capabilities of multimodal large lan-guage models (MLLMs). This progress naturally raises a question: Canthese models perform complex video reasoning in a manner comparable tohuman experts? However, existing video benchmarks primarily evaluatevisual perception and grounding, with questions that can be answeredbased on explicit prompts or isolated visual cues (e.g., “What is thewoman wearing?”). Such benchmarks do not fully capture the intricaciesof real-world reasoning, where humans must actively search for, integrate,and analyze multiple clues before reaching a conclusion. To address this,we introduce Video-Holmes, a benchmark inspired by the reasoningprocess of Sherlock Holmes, designed to evaluate the complex video rea-soning capabilities of MLLMs. Video-Holmes consists of 1,837 questionsderived from 270 manually annotated suspense short films, spanningseven carefully designed tasks. Each task is constructed by first identify-ing key events and causal relationships within films, and then designingquestions that require models to actively locate and connect multiplerelevant visual clues scattered across video clips. Evaluation of state-of-the-art MLLMs reveals that while these models generally excel at vi-sual perception, they encounter difficulties with integrating informationand often miss critical clues. For example, the best-performing model,Gemini-3.0-Pro, achieves an accuracy of only 49.6%, with most modelsscoring below 40%. We aim for Video-Holmes to serve as a “Holmes-test”for multimodal reasoning, motivating models to reason more like humansand emphasizing the ongoing challenges in this field.

We present the Identity-Preserving Large Human Recon-struction Model (IPRM), a feed-forward framework that reconstructsphotorealistic, clothed 3D humans from a single in-the-wild image whilepreserving 3D identity. Recent works predominantly infer 3D structurefrom 2D features, making it challenging to achieve 3D consistency andpreserve human identity in 3D space. To alleviate these challenges, IPRManchors the single-view 3D human reconstruction by constructing a human-based 3D feature space and explicitly preserves the human 3D identityfeatures during inference. Specifically, we introduce an efficient and ro-bust SMPL-based sparse voxel representation to transform 2D imageidentity features into 3D space, categorizing them into 3D visible iden-tity tokens and 3D invisible tokens to be inferred. Using these 3D tokens,an identity-aware 3D token inference module is proposed to propagateprojected 3D identity features from visible to invisible tokens, ensuringthat only unobserved regions are predicted while the observed identityremains intact. We further design an encoder–decoder architecture thatdecodes SMPL-based 3D features into either a 3D Gaussian Splatting ormesh representation, and augment it with a 3D ID Adapter for identitypreservation. Instead of conventional conditioning on image tokens (2DID Adapter), this adapter utilizes 3D identity tokens extracted from aparallel identity branch as guidance to inject token-wise identity infor-mation. Comprehensive experiments on existing benchmarks and in-the-wild data show IPRM surpasses state-of-the-art methods in reconstruc-tion performance, efficiency, 3D consistency and identity consistency.

Existing Video Detailed Captioning (VDC) methods pre-dominantly rely on costly human annotations or distillation from pow-erful proprietary models, creating a dependency on external supervision.In this paper, we propose VDC-Agent, an autonomous self-evolvingframework that empowers a single Multimodal Large Language Model(MLLM) to generate and refine high-quality captions through principle-guided self-reflection. To overcome the inference latency inherent in it-erative refinement, we further propose to internalize this reflective ca-pability into the model. Specifically, we construct VDC-Agent-19K, apreference dataset derived from the agent’s self-scored trajectories, andintroduce a Curriculum Direct Preference Optimization (DPO) strategy.This strategy leverages the quality gap between generated candidates toprogressively align the model from easy to hard samples. Extensive ex-periments demonstrate that VDC-Agent achieves state-of-the-art perfor-mance on VDC and DREAM-1K benchmarks, generating captions withsuperior detail and faithfulness. Crucially, our internalization strategyretains the inference efficiency of the base model while significantly en-hancing its generalization capabilities, as validated by both quantitativemetrics and human evaluation.

Multi-modal semantic segmentation (MSS) is essential forrobust perception in complex environments, yet its potential remainslargely untapped due to the prohibitive cost of human annotations. Whileunsupervised semantic segmentation (USS) has seen success on single RGBmodality, its naive extension to multi-modal data is hamperedby fusion degradation. This is because, in the absence of explicitsupervision, existing frameworks struggle to reconcile the heterogeneousstructural patterns captured by different sensors, failing to effectivelyexploit their complementary information. In this paper, we make thefirst attempt to address the novel problem of Unsupervised Multi-modal Semantic Segmentation (UMSS), aiming to effectively exploitcomplementary sensor information in a fully label-free setting. To thisend, we propose UniM2 (Unified Multi-Modal), a novel frameworkbuilt upon DINOv3 that transforms conventional fusion methods intoconsistent performance gains. Our key idea is to learn a unified latentspace driven by Cross-modal Correspondence Synergy (CMCS) toextract intrinsic shared semantic cues, bypassing the need for label-guidedadaptive fusion. To mitigate inherent inter-modal conflicts, we introducea Cross-modal Harmonizer (CMH) that designates RGB as a stablereference, effectively suppressing inconsistent relational supervision whileguiding the model to exploit complementary structural features. Extensiveexperimental results on NYU-Depth-v2 and MFNet show that UniM2improves mIoU by 6.4% and 9.8%, respectively, demonstrating clearadvantages over existing frameworks in UMSS task.

Remote photoplethysmography (rPPG) enables contactlessphysiological monitoring by capturing subtle skin-color variations fromfacial videos. However, most existing methods predominantly rely ontime-domain modeling, making them vulnerable to motion artifacts andillumination fluctuations, where weak physiological clues are easily over-whelmed by noise. To address these challenges, we propose FreqPhys,a frequency-guided rPPG framework that explicitly leverages physio-logical frequency priors for robust signal recovery. Specifically, Freq-Phys first applies a Physiological Bandpass Filtering module to sup-press out-of-band interference, and then performs Physiological Spec-trum Modulation together with adaptive spectral selection to empha-size pulse-related frequency components while suppress residual in-bandnoise. A Cross-domain Representation Learning module further fusesthese spectral priors with deep time-domain features to capture infor-mative spatial–temporal dependencies. Finally, a frequency-aware con-ditional diffusion process progressively reconstructs high-fidelity rPPGsignals. Extensive experiments on six benchmarks demonstrate that Fre-qPhys yields significant improvements over state-of-the-art approaches,particularly under challenging motion conditions. It highlights the im-portance of explicitly modeling physiological frequency priors. Our codeis available at the https://github.com/WeiQian98/FreqPhys.

High-fidelity semantic 3D scene representations are crucialfor numerous applications, including robotics, autonomous driving, andsimulation. Beyond this, the ability to edit such representations enablesdevelopers to adapt these applications more easily to specific target sce-narios. Current approaches provide limited support for controllable edit-ing. We introduce TASE, a method that projects pretrained 2D seman-tic features into a truncation-aware embedding space to enable flexi-ble 3D scene editing. Our method explicitly optimizes a feature spacein which progressively reducing feature channels yields increasingly ab-stract semantic representations, while retaining more channels preservesfine-grained detail. Additionally, we improve multi-view consistency ofthe features using a scale- and translation-consistency loss. The result-ing truncation-aware embedding space enables text-driven edits to 3Dscenes, providing explicit control over how strongly edits adhere to theoriginal scene content and allowing more substantial modifications thanprior methods. Moreover, we propose a finetuning stage for the edit-ing image generation model to mitigate artifacts caused by geometricchanges. Experimental results demonstrate competitive performance in3D scene editing, substantially outperforming prior methods on editsinvolving large geometric modifications.

Data scarcity is a major constraint for data-driven Text-to-3D Motion generation. Prior attempts exploit abundant 2D motion datato address it, but typically treat 3D generation as a post-hoc stage of 2Dprediction, resulting in camera-pose-dependent multi-stage pipelines andunderutilization of rich 2D motion semantics (e.g., temporal dynamics).To respond, we propose SWSL, a Semantic-aware Weakly SupervisedLearning framework that leverages 2D-motion semantics to directly opti-mize end-to-end Text-to-3D Motion models. SWSL comprises three mod-ules: Semantic Space Construction (SSC), which first constructs a sharedsemantic space aligning text, 2D-motion, and 3D-motion; Semantic-awarePseudo-label Enhancement (SPE), which then uses weak supervisionfrom 2D-motion embeddings to refine 3D pseudo-labels; and Semantic-level Feedback Optimization (SFO), which finally supplies embedding-level feedback for optimization. SWSL offers: (i) reliance reduction ontext-3D motion data by utilizing text-2D motion pairs; (ii) 2D data con-straint looseness by extracting semantics from single-view, easy-accessible2D motions; and (iii) scalability along mainstream Text-to-3D Motionadvancement. On HumanML3D, SWSL yields consistent FID reductionsunder both a domain-specific protocol and a more general TMR-basedprotocol: -0.012 and -0.017 on simple baselines, along with -0.010 on acomplicated one in a domain-specific setting, and -0.048, -0.038, and -0.056 under TMR setting - demonstrating its efficacy, robustness, andgeneralization beyond the domain-specific bias.

To adapt billion-parameter Vision Transformers for long-video understanding, recent methods freeze the backbone and train light-weight convolutional modules. While effective for parameter-efficient trai-ning, existing adapters do not reduce inference-time computation, leav-ing scalability with respect to video length largely unaddressed. Tokenselection can reduce attention cost by pruning redundant tokens, butit breaks the spatial grid structure required by convolutional adapters.This forces an expensive dense reconstruction, nullifying much of thepotential speedup. We address this by introducing native sparse 2D con-volutions, a primitive that allows these adapters, for the first time, tooperate directly and efficiently on dynamically pruned token sets. Weintegrate this primitive into SV-TAD, an adapter framework for tem-poral action detection, reducing VideoMAEv2-L computation by up to64% and achieving 2.2× faster inference, while maintaining state-of-the-art accuracy on THUMOS-14 and ActivityNet-1.3. When scaled toInternVideoNext-L, our approach surpasses the previous state of the artat roughly half its computational cost. Moreover, the sparse formulationnaturally supports auxiliary task tokens, which improves fine-grained as-sembly detection on ATTACH. Code and trained models are available athttps://github.com/pcr-upm/eccv26_tad.

We present StructSplat, a feed-forward and generalizable3D Gaussian reconstruction framework that operates directly on uncal-ibrated images without requiring camera parameters. Existing methodseither rely on per-scene optimization or assume known camera poses,and often entangle geometry and appearance within a unified backbone,limiting reconstruction fidelity and generalization. Our key idea is toadopt a structured representation that organizes geometry, seman-tic, and texture cues with explicit roles in the reconstruction process.Specifically, we introduce a pixel-aligned feature injection mechanismto enable accurate texture modeling from 2D observations, incorporatesemantic-aware priors to improve global consistency, and design a cameraalignment strategy to prevent information leakage and improve general-ization. Experiments show that our method significantly outperformsprior approaches on challenging benchmarks. On DL3DV, our methodachieves 28.045 PSNR, surpassing AnySplat (22.377) by +5.67 dB. Incross-dataset evaluation, our method achieves +1.94 dB over AnySplaton ACID and +1.72 dB on RealEstate10K.

Large Vision-Language Models (LVLMs) rely extensively onVisual Instruction Tuning (VIT) to elicit their multimodal reasoning ca-pabilities. However, we find a discrepancy: VIT often packs multiple lan-guage tasks about the same image for conversational, multi-turn training,whereas existing benchmarks evaluate LVLMs in isolated, single-turn sce-narios. The models can suffer from visual attention decay and contextualoverfitting during multi-turn training, making it hard for them to realizetheir full potential in the mismatched test phase. To close the gap, wepropose learning with Stochastic Turn Depth (StochasT), which stochas-tically groups language tasks for the same image into clusters of vary-ing sizes (turn depth) while preserving their organic order. Hence, whileStochasT draws on Dropout and stochastic depth for ResNets, it doesnot actually drop anything to maximize the utility of the training data.Furthermore, we introduce a challenging, benchmark-agnostic evaluationmechanism based on the Balanced Latin Square to measure LVLMs’ ro-bustness under varying contextual dependencies. Extensive experimentsdemonstrate that StochasT effectively grants LVLMs strong, harmonizedcapabilities for both single-turn and multi-turn use cases. Code is avail-able at: https://yuanqing-ai.github.io/StochasT.

Foundational feed-forward visual geometry models enableaccurate and efficient camera pose estimation and scene reconstruction bylearning strong scene priors from massive RGB datasets. However, theireffectiveness drops when applied to mixed sensing modalities, such asRGB-thermal (RGB-T) images. We observe that while a visual geometrygrounded transformer pretrained on RGB data generalizes well to thermal-only reconstruction, it struggles to align RGB and thermal modalitieswhen processed jointly. To address this, we propose SEAR, a simple yet ef-ficient fine-tuning strategy that adapts a pretrained geometry transformerto multimodal RGB-T inputs. Despite being trained on a relatively smallRGB-T dataset, our approach significantly outperforms state-of-the-artmethods for 3D reconstruction and camera pose estimation, achievingsignificant improvements over all metrics and delivering higher detailand consistency between modalities with negligible overhead in inferencetime compared to the original pretrained model. Notably, SEAR enablesreliable multimodal pose estimation and reconstruction even under chal-lenging conditions, such as low lighting and dense smoke. We validateour architecture through extensive ablation studies and demonstratehow the model aligns both modalities. Additionally, we introduce a newdataset featuring RGB and thermal sequences captured at different times,viewpoints, and illumination conditions, providing a robust benchmarkfor future work in multimodal 3D scene reconstruction. Code and modelsare publicly available at https://doi.org/10.5281/ZENODO.21077295.

3D occupancy prediction provides dense semantic scene rep-resentations for autonomous driving, yet camera-only methods sufferfrom occlusion and adverse weather, while existing fusion approachestreat 4D radar as a generic point cloud and neglect its distinct physicalmeasurements. We propose RIGS, a radar-informed Gaussian splattingframework that systematically exploits 4D radar across three stages ofthe pipeline. In scene modeling, we extract range and power from the 4Dradar tensor for depth initialization and fuse them with images throughimage–radar–Gaussian tri-modal cross-attention. In occupancy refine-ment, we introduce evidential ellipsoidal BKI with anisotropic kernelsand use RCS power as physical existence evidence for adaptive false-positive filtering, yielding uncertainty-aware predictions with Dirichletepistemic and semantic uncertainty. In velocity estimation, we proposetemporal displacement-guided Doppler de-aliasing to recover high-speedradial velocities and complement tangential components, supervised bya multi-level loss over sparse radar points, dense voxels, and connectedcomponents. Experiments on K-Radar show that RIGS achieves stronglong-range occupancy among camera–radar methods with uncertainty-aware refinement and competitive velocity estimation, validated by sys-tematic incremental ablations.