ECCV 2026 Papers

Vision–Language Models (VLMs) provide rich semantic pri-ors but are underexplored in Semi-supervised Semantic Segmentation.Recent attempts to integrate VLMs to inject high-level semantics over-look the semantic misalignment between visual and textual representa-tions that arises from using domain-invariant text embeddings withoutadapting them to dataset- and image-specific contexts. This lack of do-main awareness, coupled with limited annotations, weakens the model’ssemantic understanding by preventing effective vision-language align-ment. As a result, the model struggles with contextual reasoning, showsweak intra-class discrimination, and confuses similar classes. To addressthese challenges, we propose the Hierarchical Vision–Language Trans-former (HVLFormer), which achieves domain-aware and domain-robustalignment between visual and textual representations within a query-driven mask-transformer architecture. Firstly, we transform text embed-dings from a pre-trained VLM into multi-scale, dataset-aware textualobject queries that capture class semantics from coarse to fine gran-ularity and enhance the model’s semantic understanding. Next, thesequeries are refined using image-specific visual context, aligning globaltextual semantics with local scene structures and improving class dis-crimination. Finally, to achieve domain-robustness, we introduce cross-view and modal consistency regularization, which enforces predictionconsistency within the mask-transformer architecture across augmentedviews. It ensures that language queries remain robust to intra-class vari-ations and diverse visual scenes without overfitting to the small labeledset while maintaining stable vision–language alignment under pertur-bations during decoding. With limited training data, HVLFormer out-performs state-of-the-art methods on public benchmarks. Project page:numnz.github.io/HVLFormer.

Spatial reasoning remains a persistent challenge for manyvision–language models (VLMs), and improving it typically requires fine-tuning with substantial additional parameters. Our preliminary analysisreveals that rescaling activations in selected transformer layers—withoutmodifying pretrained weights—can significantly influence downstreamperformance. Motivated by this observation, we propose ScAle, an ultra-lightweight adaptation method that learns a small set of scalar coeffi-cients to modulate last-token attention and MLP activations in a fullyfrozen backbone. We evaluate our method on the synthetic spatial rea-soning benchmark SpatialEval and on real-world VQA datasets (CO-COQA and VGQA) across multiple model families. Our method, ScAle,achieves up to 134.1% relative accuracy gains using only 1K train-able parameters without requiring millions of trainable parameters as instandard LoRA. Despite its extreme compactness, our approach recov-ers a substantial fraction of standard PEFT performance while preserv-ing strong non-spatial VQA accuracy. These results demonstrate thatbounded activation reweighting provides a simple, architecture-agnostic,and highly parameter-efficient alternative for adapting pretrained VLMs.Code is available at https://github.com/rchowdhubnor/ScAle.git.

We present ReinDriveGen, a framework that enables full con-trollability over dynamic driving scenes, allowing users to freely edit actortrajectories to simulate safety-critical corner cases such as front-vehiclecollisions, drifting cars, vehicles spinning out of control, pedestrians jay-walking, and cyclists cutting across lanes. Our approach constructs adynamic 3D point cloud scene from multi-frame LiDAR data, introducesa vehicle completion module to reconstruct full 360° geometry from par-tial observations, and renders the edited scene into 2D condition imagesthat guide a video diffusion model to synthesize realistic driving videos.Since such edited scenarios inevitably fall outside the training distri-bution, we further propose an RL-based post-training strategy with apairwise preference model and a pairwise reward mechanism, enablingrobust quality improvement under out-of-distribution conditions with-out ground-truth supervision. Extensive experiments demonstrate thatReinDriveGen outperforms existing approaches on edited driving scenar-ios and achieves state-of-the-art results on novel ego viewpoint synthesis.

Recent advances in generative super-resolution (SR) havegreatly improved visual realism, yet existing evaluation and optimizationframeworks remain misaligned with human perception. Full-Referenceand No-Reference metrics often fail to reflect perceptual preference, ei-ther penalizing semantically plausible details due to pixel misalignmentor favoring visually sharp but inconsistent artifacts. Moreover, most SRmethods rely on ground-truth (GT)–dependent distribution matching,which does not necessarily correspond to human judgments. In this work,we propose RefReward-SR, a low-resolution (LR) reference-aware rewardmodel for preference-aligned SR. Instead of relying on GT supervisionor NR evaluation, RefReward-SR assesses high-resolution (HR) recon-structions conditioned on their LR inputs, treating the LR image asa semantic anchor. Leveraging the visual–linguistic priors of a Multi-modal Large Language Model (MLLM), it evaluates semantic consistencyand plausibility in a reasoning-aware manner. To support this paradigm,we construct RefSR-18K, the first large-scale LR-conditioned preferencedataset for SR, providing pairwise rankings based on LR–HR consistencyand HR naturalness. We fine-tune the MLLM with Group Relative Pol-icy Optimization (GRPO) using LR-conditioned ranking rewards, andfurther integrate GRPO into SR model training with RefReward-SR asthe core reward signal for preference-aligned generation. Extensive ex-periments show that our framework achieves substantially better align-ment with human judgments, producing reconstructions that preservesemantic consistency while enhancing perceptual plausibility and visualnaturalness.

Vision-Language-Action (VLA) models provide a unifiedparadigm for robotic manipulation, yet their real-world deploymentremains limited by execution efficiency. While existing efforts predomi-nantly focus on compute-centric efficiency to reduce per-step inferencelatency, the intrinsic policy efficiency of these models remains largelyunexplored. Policy efficiency is fundamentally affected by two factors,namely the effective executable length of predicted action chunks and thetotal physical steps required to complete a task. These two factors jointlydetermine the number of forward inference calls during execution. Weobserve that current VLA policies struggle with planning unreliabilityand action redundancy, suffering from severe tail degradation in actionchunks and tending to generate redundant physical steps. To address this,we propose PolicyTrim, a reinforcement learning-based post-trainingframework that extends reliable action chunk length and reduces redun-dant physical steps. For reliable chunk extension, we employ a dynamicexploration strategy that rewards the successful completion of longer exe-cutable lengths, progressively pushing the trustworthy prediction horizonto its empirical limit. For step efficiency, we design a redundancy-awarereward that favors successful task completion with fewer steps whilepenalizing unreproducible shortcuts, effectively eliminating redundantphysical actions. Experiments on three benchmarks and three VLA mod-els show that PolicyTrim improves action chunk utilization by 3×, re-duces physical execution steps by 51.4%, and delivers up to a 5.83× end-to-end deployment speedup without compromising task success rates.Project Page: https://inceptionwang.github.io/PolicyTrim/

3D Gaussian Splatting (3DGS) enables efficient novel-viewsynthesis, yet its extension to simultaneous RGB and thermal infrared(TIR) reconstruction is hindered by a fundamental spectral mismatch:RGB relies on reflected illumination while TIR captures emitted radi-ance. This disparity in frequency characteristics and physical sensingmechanisms causes gradient interference when naively sharing a unifiedrepresentation. Common methods typically treat TIR as an auxiliary in-tensity channel, ignoring the physical radiative transfer governing ther-mal imaging. Critically, enforcing a single opacity field across modalitiesis physically invalid due to cross-spectral visibility disparities, constrain-ing representational capacity and yielding thermally implausible recon-structions. To address these limitations, we propose Physically GroundedDual-Opacity Gaussian Splatting, a framework that unifies RGB andTIR reconstruction under a shared geometric scaffold with modality-aware visibility modeling. We introduce Radiative Attribute Parameter-ization, explicitly modeling each Gaussian’s thermal response throughemissivity, temperature, and reflectance. Dual-Opacity Rendering en-ables spectrally aware visibility handling by assigning modality-specificopacities to shared location parameters, resolving cross-modal occlusionconflicts. Training jointly optimizes photometric reconstruction, physicalpriors, and RGB-guided geometric regularization to resolve parameterambiguities. Experiments demonstrate competitive performance in bothradiometric accuracy and photorealistic novel view synthesis, highlight-ing the potential of 3DGS for joint thermal field with high-fidelity RGBreconstruction.

Recent advances in text-to-video (T2V) generation have achieved good visual quality, yet synthesizing videos that faithfully followphysical laws remains an open challenge. Existing methods mainly basedon graphics or prompt extension struggle to generalize beyond simplesimulated environments or learn implicit physics reasoning. The scarcityof training data with rich physics interactions and phenomena is alsoa problem. In this paper, we first introduce a Physics-Augmented videodata construction Pipeline, PhyAugPipe, that leverages a vision–languagemodel (VLM) with chain-of-thought reasoning to collect a training dataset,PhyVidGen-135K. Then we formulate a principled Physics-aware Group-wise Direct Preference Optimization, PhyGDPO, framework that usesreal-world video as winning case to guarantee correct physics learningand builds upon the groupwise Plackett–Luce probabilistic model tocapture holistic preferences beyond pairwise comparisons. We design aPhysics-Guided Rewarding (PGR) scheme that uses VLM-based physicalrewards to direct the optimization to focus on challenging physics. Plus,we propose a LoRA-Switch Reference (LoRA-SR) scheme that avoidsfull-model duplication as reference for efficient training. Experimentsshow that our method outperforms state-of-the-art methods. Our codeand data is at https://github.com/caiyuanhao1998/Open-PhyGDPO

Recent advances in 3D feedforward reconstruction neural net-works have achieved remarkable success in dense reconstruction from imageswithout any camera parameters. Yet, equipping these models with robustsemantic understanding remains an open problem. Here we introduce anapproach that performs 3D reconstruction and 3D panoptic segmentationin a unified framework. We build on existing 3D reconstruction models andaugment them with a set-based mask decoder. The approach is jointly trainedwith a geometric and semantic loss, which are shown to be mutually beneficial.More precisely, the features are initialized from the geometric informationand then finetuned to capture jointly geometry and semantics. We demon-strate the generality of our approach by successfully applying our frameworkboth to online and all-to-all attention reconstruction backbones. Our methodachieves state-of-the-art performance in 3D panoptic segmentation acrossScanNet, ScanNet200, and ScanNet++ datasets. Ablation studies show thatsuch joint training of a unified model equips 3D feedforward reconstructionneural networks with panoptic segmentation and yields mutually beneficialimprovements.

Existing diffusion-based 3D scene generation methods pri-marily operate in 2D image/video latent spaces, which makes maintain-ing cross-view appearance and geometric consistency inherently chal-lenging. To bridge this gap, we present OneWorld, a framework thatperforms diffusion directly within a coherent 3D representation space.Central to our approach is the 3D Unified Representation Autoencoder(3D-URAE); it leverages pretrained 3D foundation models and augmentstheir geometry-centric nature by injecting appearance and distilling se-mantics into a unified 3D latent space. Furthermore, we introduce token-level Cross-View-Correspondence (CVC) consistency loss to explicitly en-force structural alignment across views, and propose Manifold-Drift Forc-ing (MDF) to mitigate train–inference exposure bias and shape a robust3D manifold by mixing drifted and original representations. Comprehen-sive experiments demonstrate that OneWorld generates high-quality 3Dscenes with superior cross-view consistency compared to state-of-the-art2D-based methods.

EgoExo proficiency estimation aims to assess action qual-ity by integrating fine-grained motion cues from egocentric (1st-person)views with spatial context from multiple exocentric (3rd-person) views.Simply adding more exocentric views degrades EgoExo performance, asredundant or noisy perspectives dilute useful motion cues. Our anal-ysis identifies two key causes: (1) Multiview redundancy — Fromthe data perspective, certain views provide limited or noisy informa-tion, diluting discriminative cues; (2) Overfitting — From the featureperspective, conventional fusion increases representational complexity,causing the model to memorise view-specific patterns rather than learngeneralisable representations. To address these issues, we propose twocomplementary modules. AdaMVS adaptively identifies and fuses themost informative view tokens under weak supervision from the data per-spective, while VIB-GB combines Gradient Blending and VariationalInformation Bottleneck regularisation from the feature perspective tocompress redundant signals and suppress overfitting during training.Experiments on EgoExo-4D and EgoExo-Fitness demonstrate that ourmethod learns both which view to look at and how to fuse them,achieving new state-of-the-art results. Our source code is available athttps://github.com/dx199771/AdaMVS

Reconstructing 3D objects from brain signals offers a promis-ing avenue for understanding human visual cognition. While prior workhas shown initial success using EEG signals for 3D reconstruction, ex-isting methods typically employ a uniform diffusion decoder, overlook-ing the evolving semantic granularity of both EEG representations andthe diffusion denoising process. In this paper, we propose Mind2Cloud, anovel EEG-to-point-cloud generation framework based on two-granularitydiffusion decoding. The core of Mind2Cloud is a time-aware decoder thatintegrates a global Transformer branch and a local Point-Voxel CNN(PVCNN) branch across diffusion timesteps through a learnable fusionmask. Specifically, Transformer layers are incorporated into the earlyupsampling stages to capture global object structure under high uncer-tainty, while PVCNN modules are used in later stages to refine local ge-ometric details. Inspired by the hierarchical nature of EEG-based visualrepresentations, this design dynamically adapts its spatial granularity inaccordance with the coarse-to-fine trajectory of diffusion denoising. Wefurther introduce an adversarial refinement module to enhance geometricrealism and semantic consistency. Extensive experiments on the EEG-3Ddataset across all 12 subjects demonstrate that Mind2Cloud outperformsprior work in both geometric accuracy and semantic alignment, settinga new benchmark for EEG-to-point-cloud generation. Our source code isavailable at https://github.com/duasoi/Mind2Cloud.

While recent generative models produce high-fidelity videos,they struggle with the complex narrative control required for coherentmulti-shot audio-visual generation. Existing methods suffer from tem-poral misalignment, limited controllability, and incomplete scripting. Inthis paper, we propose MAVIN, the first framework for multi-shot audio-visual generation with customized narrative control. To resolve tempo-ral misalignment, we propose boundary-aware attention, which lever-ages hierarchical captions and boundary-aware token routing to renderaudio-visual elements within their respective temporal boundaries. Toimprove the controllability for multi-subject scenarios, we propose ID-aware propagation, utilizing identity embeddings and an identity-awaremask to bind specific identities to consistent visual appearances and vocaltimbres. To provide comprehensive audio-visual narratives, we present amulti-agent scripting pipeline to transform free-form user inputs into hi-erarchical captions. Furthermore, we construct MAVINSet, a multi-shotaudio-visual dataset for robust training and evaluation. Extensive exper-iments demonstrate that MAVIN achieves state-of-the-art performance,opening up a new avenue for integrating generative models into profes-sional filmmaking workflows.

We present HippoCamp, a new benchmark designed to eval-uate agents’ capabilities on multimodal file management. Unlike exist-ing agent benchmarks that focus on tasks like web interaction, tool-use, or software automation in generic settings, HippoCamp evaluatesagents in user-centric environments to model individual user profilesand search from massive personal files for context-aware reasoning. Ourbenchmark instantiates device-scale file systems over real-world profilesspanning diverse modalities, comprising 42.4 GB of data across over2K heterogeneous files. Building upon the raw files, we construct 581QA pairs to assess agents’ capabilities in search, evidence perception,and multi-step reasoning. To facilitate fine-grained analysis, we pro-vide 46.1K fine-grained trajectory annotations for step-wise failure di-agnosis. We evaluate a wide range of state-of-the-art multimodal large✉ Corresponding authors.language models (MLLMs) and agentic methods on HippoCamp. Ourcomprehensive experiments reveal a significant performance gap: eventhe most advanced commercial models achieve merely a 48.3% accu-racy in user profiling, struggling particularly with long-horizon retrievaland cross-modal reasoning within dense personal file systems. Further-more, our step-wise failure diagnosis identifies multimodal perceptionand evidence grounding as the primary bottlenecks. Ultimately, Hip-poCamp exposes the critical limitations of current agents in realistic,user-centric environments and provides a robust foundation for develop-ing next-generation personal AI assistants. Our dataset is publicly avail-able at: https://huggingface.co/datasets/MMMem-org/HippoCamp.

Few-shot Chinese handwriting generation aims to render char-acters in a target style using limited references. Existing image-levelmethods often suffer from limited stylistic fidelity and poor general-ization. We propose Hierarchical Style Aggregation for Chinese hand-writing generation (HiSAC), a versatile framework that decouples stylemodeling from task-specific generation. Our core idea is to leverage thecompositional nature of Chinese characters: by decomposing charactersinto radicals, components, and strokes, we establish fine-grained spatialcorrespondences between target content and reference glyphs at eachlevel, enabling precise style transfer. To further capture nuanced stylecharacteristics, we design a multi-band frequency encoder that extractsstylistic representations across different spectral ranges. The aggregatedmulti-granularity features can be seamlessly integrated with task-specificdecoders (e.g., Transformer for online trajectory generation, Diffusion foroffline image synthesis) without architectural redesign. By jointly model-ing style in both spatial and frequency domains, our approach enhancesrealism in global structure and stroke dynamics. Extensive experimentsdemonstrate that HiSAC outperforms existing methods both quantita-tively and qualitatively, and generalizes well to out-of-vocabulary andcross-language scripts. Code has been released at https://github.com/IIP-Lab-XDU/HiSAC.

Precise spatial understanding in Earth Observation is essen-tial for translating raw aerial imagery into actionable insights for criti-cal applications like urban planning, environmental monitoring and dis-aster management. However, Multimodal Large Language Models ex-hibit critical deficiencies in fine-grained spatial understanding withinRemote Sensing, primarily due to a reliance on limited or repurposedlegacy datasets. To bridge this gap, we introduce a large-scale datasetgrounded in verifiable cadastral vector data, comprising 3.8 million an-notated objects across 510k high-resolution images with 135 granularsemantic categories. We validate this resource through a comprehensiveinstruction-tuning benchmark spanning seven spatial grounding tasks.Our evaluation establishes a robust baseline using a standard LLaVAarchitecture. We show that while current RS-specialized and commer-cial models (e.g., Gemini) struggle in zero-shot settings, high-fidelity su-pervision effectively bridges this gap, enabling standard architecturesto master fine-grained spatial grounding without complex architecturalmodifications. Data, pretrained model and code are available at: https://huggingface.co/datasets/RogerFerrod/GroundSet

Autoregressive models in vision have gained widespread at-tention under the scalable next-token prediction paradigm, representingimages as sequences of discrete tokens. Yet their optimization remainschallenging due to the non-differentiability of discrete token samplingand the prohibitive memory costs of full-sequence fine-tuning. We pro-pose G3 AFT, a novel end-to-end fine-tuning framework for discrete-tokenautoregressive models. Within this framework, we introduce two com-plementary techniques, i.e. Global Scope Gradient Recording (GSGR)and Noise Suppressed Gradient Projection (NSGP). GSGR selectivelyupdates periodically sampled tokens by leveraging coarse global signalsthat summarize the overall layout and long-range structure of the image.This design alleviates the prohibitive memory overhead of full-sequencefine-tuning while maintaining long-range coherence. In parallel, NSGPenhances surrogate gradient flow by explicitly aligning updates withsalient sampling alternatives and filtering out spurious signals introducedby softmax relaxations. By suppressing noisy gradient components andreinforcing causally consistent updates, NSGP provides a more faithfulapproximation to discrete sampling, ensuring faithful gradient propaga-tion across long autoregressive trajectories. To the best of our knowl-edge, this is the first method under the next-token prediction paradigmfor discrete-token autoregressive image generation that enables fully dif-ferentiable optimization across the entire generation process. Extensiveexperiments validate that G3 AFT achieves superior image quality andquantitative performance, providing a practical framework for fully dif-ferentiable fine-tuning of autoregressive image generation models. Codeis available at https://github.com/Wotoosh/G3AFT.

While 3D Gaussian Splatting (3DGS) has enabled real-time high-fidelity novel view synthesis, maintaining already-deployed static 3DGS assets remains difficult. When a local region is missing or blurred, retraining the whole scene from scratch is costly, while directly finetuning the asset on newly captured repair images causes severe degradation of unobserved views, and geometric stitching introduces boundary and depth conflicts. We propose Flash-Refine, a practical in-situ repair pipeline for static 3DGS assets that operates directly on a native PLY model without access to historical training images. Given newly registered repair views, Flash-Refine uses Point-Wise Consensus Masking and an Adaptive Depth-Bounded Prior to identify a bounded active region, applies Gradient Locking to frozen background Gaussians, propagates the mask through native densification, and protects frozen Gaussians from close-up pruning. Under this constrained active set, gradient-gated densification naturally allocates new parameters to the defective region, enabling rapid local completion from peripheral seeds. Extensive experiments on Deep Blending and Mip-NeRF 360 show that Flash-Refine achieves the best Dropped and Kept PSNR among practical repair methods across four scenes, while requiring only 2–4 minutes on an RTX 4090. We further analyze the method’s scope and limitations: it assumes static scenes and preregistered repair views, and although frozen background parameters remain unchanged, newly densified active Gaussians may still induce scene-dependent visibility interactions in unobserved views.

Vision-based 3D occupancy prediction fundamentally relieson the 2D-to-3D view transformation. Current paradigms predominantlyutilize explicit physical projection, which artificially restricts the routingmatrix to strict, sparse camera rays. While computationally efficient, thisimposes a severe Locality Bottleneck, preventing the network from con-structing holistic contextual understanding and degrading sharply whencamera extrinsics are unreliable or absent. To break this bottleneck, weabstract view transformation as unconstrained bipartite routing and pro-pose Factorized Dense Routing (FDR). By approximating dense2D-to-3D mixing through hierarchical tensor contractions, FDR guar-antees a fully-global receptive field with tractable, sub-quadratic com-plexity. Crucially, the mandatory spatial contraction in dense routingexposes a fundamental Resolution-Context Trade-off. To address this,we introduce a Resolution-Context Decoupled Architecture. Wefactorize the 3D space into a global macroscopic topological anchor (viaFDR) and precise local geometric planes (via explicit projection). Thisdecoupling enables global semantic inference and exact surface localiza-tion to complement each other without mutual compromise. Extensiveexperiments demonstrate that our framework achieves state-of-the-artperformance on the Occ3D-nuScenes and Occ3D-Waymo benchmarks.More notably, in an uncalibrated setting where physical extrinsics arewithheld, our global routing internalizes the implicit multi-camera rigtopology and exhibits substantially stronger structural robustness thanphysical-projection baselines under the same protocol.

Human motion capture from head-mounted devices (HMDs)offers a scalable way to acquire real-world human motion and interac-tion data, which is crucial for applications in embodied AI and VR/AR.Existing approaches focus on either egocentric body tracking, estimat-ing the motion of the subject wearing the device, or exocentric track-ing, capturing the movements of people in the wearer’s surroundings. Sofar, these two paradigms have largely been explored in isolation. In thispaper, we propose a novel distributed framework that jointly leveragesego- and exocentric multi-modal signals for human motion estimationfrom HMDs. Unlike traditional motion capture systems requiring bulkymulti-camera setups or obtrusive mocap suits, our approach, EgoExo-MoCap, is as simple as two (or more) people, each wearing a pair ofsmart glasses. The method leverages head (plus potentially wrist) track-ing signals for accurate estimation of global motion in the 3D world andcombines context-aware image features based on DINOv3 to achieve ro-bustness in the presence of noise and occlusions. Extensive experimentson two in-the-wild datasets show that our approach can robustly recon-struct motion even in challenging scenarios.