ECCV 2026 Papers

Recent progress in video large language models (Video-LLMs) has enabled strong offline reasoning over long and complex videos. However, real-world deployments increasingly require streaming perception and proactive interaction, where video frames arrive online and the system must decide not only what to respond, but also when to respond. In this work, we revisit proactive activation in streaming video as a structured sequence modeling problem, motivated by the observation that temporal transitions in streaming video naturally form span-structured activation patterns. To capture this span-level structure, we model activation signals jointly over a sliding temporal window and update them iteratively as new frames arrive. We propose STRIDE (Structured Temporal Refinement with Iterative DEnoising), which employs a lightweight masked diffusion module at the activation interface to jointly predict and progressively refine activation signals across the window. Extensive experiments on diverse streaming benchmarks and downstream models demonstrate that STRIDE shows more reliable and temporally coherent proactive responses, significantly improving “when-to-speak” decision quality in online streaming scenarios. The STRIDE codebase and demonstrations are available on our project webpage.

Weight-only post-training quantization (PTQ) has proven highly effective for deploying large language models under memory-bound inference constraints. However, extending PTQ to Multimodal LLMs at ultra-low bit-widths (2–3 bits) presents a significant challenge. Existing uniform-precision methods suffer severe accuracy collapse due to their implicit assumption that all output channels within a layer are equally sensitive to quantization noise. This uniform bit-width assignment fails to account for the heterogeneous sensitivity of different channels. By lifting the analysis to the Transformer block level, we reveal a two-level anisotropy—inter-layer and intra-layer inter-channel. This anisotropy is further amplified by an outlier–sensitivity resonance mechanism, causing different output channels to contribute vastly different amounts of quantization error to the block output. Together, these findings provide the rigorous foundation for per-output-channel mixed-precision quantization. Building on the derived optimality condition, we propose GloBitQ, a framework that unfolds the anisotropic loss into principled bit allocation. It integrates three intermediate proxies: power-law balanced layer weighting, rank–value hybrid channel scoring, and global top-τ bit assignment. The resulting allocation provably approximates the optimum of the block-level quantization objective. On six multimodal benchmarks spanning Qwen2-VL-7B, Qwen2.5-VL-7B, and LLaVA-OneVision, GloBitQ consistently surpasses GPTQ, GPTAQ, and VLMQ: the 3-bit model performs within 2.1% of full precision(81.70 vs. 83.76), while the 2-bit model remains stable where prior methods collapse—all within a calibration-only, finetuning-free, hardware-compatible pipeline. Code is available in the supplementary material.

While modern multimodal models integrate vision with lan-guage, audio, or touch, olfaction remains largely unexplored due to thelack of paired visuo-olfactory data. We introduce SmellNet-V, a scal-able visuo-olfactory dataset built on the insight that odor identity islargely invariant to visual transformations within a semantic category.This allows us to synthetically pair smell-only samples with semanticallyaligned in-the-wild web images, converting a unimodal olfactory datasetinto a cross-modal benchmark without costly co-collection. Building onthis dataset, we propose See & Sniff, a self-supervised framework thatlearns joint visuo–olfactory representations via dense local alignment andnaturally produces smell saliency maps for spatial grounding of odorsources. We further introduce pixel-level smell localization task and abenchmark for evaluation. Our method surpasses smell-only baselines by7% in smell classification from smell alone and generalizes to cross-modalretrieval and smell localization, establishing visuo-olfactory learning asa new direction in multimodal perception.

Continual learning in visual question answering (VQACL) re-quires a single vision–language model to acquire new multimodal reason-ing skills from a task stream while retaining prior capabilities. However,naïve sequential finetuning suffers from catastrophic forgetting. Existingcontinual VQA methods primarily rely on replay or parameter regulariza-tion but largely overlook how task-specific updates accumulate and inter-act in parameter space, particularly whether successive updates are syn-ergistic or conflicting across layers. To address this, we introduce SCoT(Similarity-guided Conflict-aware Task Consolidation), a continual learn-ing framework that represents each task as a parameter update relative to apretrained anchor model and integrates tasks through layer-wise parameter-space reasoning. For each layer, SCoT measures alignment between incom-ing and accumulated task vectors, removes only destructive components viaconditional projection when conflicts arise, and adaptively modulates con-solidation strength using similarity-guided weighting. This preserves ben-eficial transfer while suppressing harmful interference, enabling stable yetadaptive continual learning. Experiments on VQAv2 and NExT-QA demon-strate strong continual VQA performance, reducing forgetting to near-zero(0.07 and -1.90) while achieving rare positive backward transfer (+5.64 and+6.97), outperforming strong continual-learning and task-vector baselines.Project page: https://anand-patel05.github.io/SCoT

SceneHI is a framework that lifts high-resolution, illumina-tion-aware priors from 2D diffusion models to perform 3D texture syn-thesis. It is the first to demonstrate that high-resolution textures, previ-ously limited to 2D synthesis, can be generated directly on 3D objectswithout model fine-tuning or optimization. Designed for complex, multi-object environments, SceneHI uniquely combines 3D-consistency, high-resolution fidelity, and physically plausible baked shadows within a singlegenerative pipeline. To enforce strict geometric coherence, we introducean exact analytical pixel-to-texel mapping that aligns diffusion trajecto-ries across multiple viewpoints. We utilize High-Resolution Latent Tex-tures (HRLTs) as a persistent canvas for gradually denoised textures,while camera views perform the denoising steps in latent pixel space.This ensures a shared base texture that can be subsequently refined tohigh resolution without compromising multi-view consistency. Finally, alight-aware generative pass embeds realistic geometry-consistent shad-ows directly into the atlases, bridging the gap to production workflows.SceneHI achieves high visual fidelity while reducing generation time by80% compared to existing scene-level methods. The code for our work isavailable at https://github.com/Thanos-DB/SceneHI.

Modern dense predictors largely follow a pyramidal pixel-decoding paradigm: they progressively lift low-resolution representationsto dense, high-resolution outputs. While effective, this design tightlycouples computation with output resolution, making dense predictionincreasingly costly to scale. In this work, we argue that strong denseprediction can be achieved by separating capacity from resolution: per-forming high-capacity reasoning in a compact latent space, and readingout to pixels with a lightweight operator. To instantiate this principle,we introduce Latent Dense Prediction Transformer (L-DPT), a minimalarchitecture that decouples dense decoding from pixel-space rendering.Instead of progressively decoding toward the pixel grid, L-DPT performsattention-based dense decoding entirely in latent space, keeping compu-tation largely stable as resolution increases while enabling more flexiblefeature integration from the encoder. The final output is produced viaa lightweight pixel readout, such as pixel shuffle, so scaling to extremeresolutions does not amplify decoding cost. As a result, L-DPT achievesstronger dense prediction accuracy while enabling inference on profes-sional ultra-high-resolution images up to 100 megapixels. More broadly,we view L-DPT as an instance of a scalable design principle rather thana task-specific decoder for dense prediction.

Dataset distillation (DD) condenses large corpora into com-pact, information-rich subsets for efficient training and reuse. However,under noisy supervision, DD risks condensing corrupted associations to-gether with useful signals, degrading robustness. Conventional noisy-label remedies (sample selection, loss weighting, label correction) tightlycouple noise estimation with model optimization, often require clean an-chors, and can amplify confirmation bias—assumptions that are mis-aligned with DD’s goal of compact, plug-and-play supervision. We there-fore propose a trajectory-based DD framework that jointly suppressesnoise and preserves transferable knowledge without relabeling or cleansubsets. It comprises two complementary components: Selective Guid-ance Reweighting (SGR), which fuses global forgetting patterns (second-split forgetting) with local neighborhood consistency into a progressivereweighting scheme that prioritizes clean supervision along the teachertrajectory; and Teacher-Inspired Auxiliary Targets (TIAT), which injectauxiliary residual guidance distilled from intermediate teacher dynamicsto reinforce informative signals while remaining internally consistent. To-gether, SGR and TIAT produce distilled datasets with cleaner and richerrepresentations under noisy supervision. The framework is robust, label-preserving, computationally lightweight, and broadly applicable, yield-ing consistent gains over state-of-the-art DD baselines across symmetric,asymmetric, and real-world noise.

Indoor scene generation is crucial for robot simulation andmodern interior design. However, complex layouts together with scarce3D scene data make learning-based generation challenging. Existing meth-ods often rely on hand-crafted rules or focus on isolated sub-tasks (e.g.,floorplan synthesis or single-room furnishing), producing whole-homescenes that lack global coherence, realism, and simulation readiness. Tomitigate these limitations, we propose a unified hierarchical frameworkthat decomposes indoor scene synthesis into controllable stages. First, wecurate a large-scale dataset of 300K real residential floorplans to traina large language model for whole-home floorplan generation. With de-tailed descriptions and a K-D tree–based representation, our method en-ables fine-grained, controllable whole-home floorplan generation. Build-ing upon the generated whole-home floorplan, we leverage image gener-ation models to draft furniture layouts from multi-level roaming view-points, and then generate the layouts of small manipulable objects ondifferent supporting surfaces (e.g. cabinets, desks, and dining tables) forembodied AI simulation. During furniture and object layout generation,a VLM-based refiner iteratively corrects furniture and object placement,and a 3D generative model enables flexible replacement of individualassets. We further attach basic physical attributes and simple surfacetexture and lighting setups to complete the pipeline for embodied AIuse. Experiments and user studies demonstrate that our pipeline pro-duces indoor spaces with greater layout diversity and stronger 3D designappeal, outperforming prior methods on both quantitative and qualita-tive metrics. Finally, alongside our generation pipeline, we will releasethe floorplan dataset and 5K fully furnished scenes to the community.Project page: https://kairos-homeworld.github.io/.

Estimation of Human-Object Interactions in 3D (3D HOI)is a fundamental problem in 3D computer vision with applications inAR/VR, robotics, and embodied AI. However, reconstructing these in-teractions in 3D remains challenging due to depth ambiguities, occlu-sions, and object shape variability. Existing approaches are primarilyconcerned with reprojection and contact constraints, fitting parametrichuman models and object templates to 2D images. In this paper, weexplore a different avenue. We present MILO, a framework that lever-ages the visual capabilities of Large Reconstruction Models (LRMs) torecover detailed 3D human-object interactions from a single image. Ourkey observation is that LRMs provide a powerful geometric scaffold thatpreserves relative human-object arrangement and proximity cues. Thissignificantly simplifies the reconstruction procedure, reframing the prob-lem as interpreting the LRM mesh: we segment it into human and objectcomponents, fit a parametric body model to the human part, and op-tionally align an object template to the object part (if such a templateis available). MILO achieves strong reconstruction accuracy and out-performs existing baselines across multiple benchmarks and interactionscenarios. Our code is available at https://ac5113.github.io/MILO.

Cameras capture scene-referred linear raw images, which areprocessed by onboard image signal processors (ISPs) into display-referred8-bit sRGB outputs. Although raw data is more faithful for low-levelvision tasks, collecting large-scale raw datasets remains a major bottle-neck, as existing datasets are limited and tied to specific camera hard-ware. Generative models offer a promising way to address this scarcity;however, existing diffusion frameworks are designed to synthesize photo-finished sRGB images rather than physically consistent linear representa-tions. This paper presents RawGen, to our knowledge the first diffusion-based framework enabling text-to-raw generation for arbitrary targetcameras, alongside sRGB-to-raw inversion. RawGen leverages the gener-ative priors of large-scale sRGB diffusion models to synthesize physicallymeaningful linear outputs, such as CIE XYZ or camera-specific raw rep-resentations, via specialized processing in latent and pixel spaces. Tohandle unknown and diverse ISP pipelines and photo-finishing effectsin diffusion-model training data, we build a many-to-one inverse-ISPdataset where multiple sRGB renditions of the same scene generatedusing diverse ISP parameters are anchored to a common scene-referredtarget. Fine-tuning a conditional denoiser and specialized decoder on thisdataset allows RawGen to obtain camera-centric linear reconstructionsthat effectively invert the rendering pipeline. We demonstrate RawGen’ssuperior performance over traditional inverse-ISP methods that assume afixed ISP. Furthermore, we show that augmenting training pipelines withRawGen’s scalable, text-driven synthetic data can benefit downstreamlow-level vision tasks.

Dexterous grasping is a fundamental ability for robots toperform complex manipulation tasks. However, simultaneously maximiz-ing grasp success rates and maintaining grasp-type diversity remains asignificant challenge. In response, we introduce RAGrasp, a retrieval-augmented framework that comprises two core components: a Geometry-Aware hand pose Initialization strategy(GAI) and a Diversity-Awaregrasp-type Selection network(DAS). First, GAI constructs a distribution-aligned retrieval dataset and leverages object geometric similarity toretrieve high-quality initial samples, significantly improving grasp suc-cess rates. Second, DAS adopts a retrieval-augmented dual-head classi-fier that formulates grasp-type selection as a collaborative optimizationproblem of single-label and multi-label grasp-type classification, improv-ing diversity and allowing flexible diversity adjustment. Extensive experi-ments show that RAGrasp yields an 8.2% relative improvement in successrate on BODex, alongside average relative gains of 13.0% and 11.9% onDexonomy under type-unconditioned and type-conditioned settings.

Estimating the 6D pose of unseen objects is a fundamentalyet challenging problem for open-world robotics and embodied percep-tion. Model-based methods are accurate but depend on CAD assets orheavy onboarding, while most model-free approaches are still limitedto pairwise single-anchor matching and thus fail under occlusion andlarge viewpoint changes with low query–reference overlap. Therefore, wepresent PANY, a unified model-free framework that seamlessly sup-ports both RGB and RGB-D inputs, operates on one or sparse pose-freereference views, and generalizes effectively to novel objects. Built on amulti-view transformer geometry backbone, PANY moves beyond pair-wise matching by learning view-consistent geometry and cross-view align-ment cues that remain stable under wide baselines and limited overlap.When additional unposed assist views are available, PANY aggregatesthem via pose-graph canonical registration to increase geometric cover-age and reinforce the final pose. Extensive experiments show that PANYachieves state-of-the-art performance across multiple benchmarks, sub-stantially outperforming existing model-free methods, improving poseaccuracy by +12% on YCB-V and over +20% on LM-O. Further-more, PANY consistently performs well under both single-reference andsparse-reference settings, demonstrating strong robustness in real-worldenvironments.

Neural Parametric Head Models (NPHMs) are a recent ad-vancement over mesh-based 3d morphable models (3DMMs) to facilitatehigh-fidelity geometric detail. However, fitting NPHMs to visual inputsis notoriously challenging due to the expressive nature of their under-lying latent space, which heavily limits NPHM’s practical use. To thisend, we propose Pix2NPHM, a vision transformer (ViT) network that di-rectly regresses NPHM parameters, given a single image as input, finallybridging the gap from theoretical to practical use. Compared to exist-ing 3DMM regressors, the neural parametric space allows our methodto reconstruct more recognizable facial geometry and accurate facial ex-pressions. For broad generalization, we exploit domain-specific ViTs asbackbones, which are pretrained on geometric prediction tasks. We trainPix2NPHM on a mixture of 3D data, including a total of over 100KNPHM registrations, and large-scale 2D video datasets, for which nor-mal estimates serve as pseudo ground truth. While Pix2NPHM runs atinteractive frame rates, it is possible to improve geometric fidelity by asubsequent optimization against estimated surface normals and canoni-cal point maps. As a result, we achieve unprecedented face reconstructionquality that can run at scale on in-the-wild data.

Recently, self-evolving large multimodal models (LMMs) havereceived attention for improving visual reasoning in a purely unsuper-vised setting. However, multi-role self-play and self-consistency rewardschemes in existing self-evolving LMMs optimize answer agreement with-out ensuring the decoder attends to visual content, relying instead onstatistical language priors to produce self-consistent outputs. This leadsto a persistent failure mode we term visual under-conditioning, wherethe decoder relies on language priors rather than the image during gen-eration, manifesting as insufficient attention to visual tokens. As a result,current self-evolving LMMs struggle on vision–language understandingtasks such as image captioning and visual question answering. To addressthis, we propose VISE (Visual Invariance Self-Evolution), a purely un-supervised self-evolving framework that directly regularizes the model’svisual conditioning policy through two complementary invariance-basedrewards: a geometric invariance reward that enforces spatial consistencyunder known transformations, and a semantic invariance reward thatpenalizes evidence-agnostic generation by requiring the model to recog-nize the absence of evidence when predicted regions are perturbed. VISEoperates within a single model without specialist roles, external rewardmodels, or annotations, and is trained on raw unlabeled images. Ex-periments on 18 benchmarks demonstrate the efficacy of our approach.Using Qwen3-VL-2B as the base model, VISE achieves gains of +16.85CIDEr on COCO and +19.66 CIDEr on TextCaps, reduces object hal-lucination by 5.0 Chair-I points, and generalizes across four model fam-ilies and scales. Our code and models are available at https://mbzuai-oryx.github.io/VISE/.

We present One4D, a unified framework for 4D generationand reconstruction that produces dynamic 4D content as synchronizedRGB frames and pointmaps. By consistently handling varying sparsitiesof conditioning frames through a Unified Masked Conditioning (UMC)mechanism, One4D can seamlessly transition between 4D generationfrom a single image, 4D reconstruction from a full video, and mixed gen-eration and reconstruction from sparse frames. Our framework adapts apowerful video generation model for joint RGB and pointmap generation,with carefully designed network architectures. The commonly used diffu-sion finetuning strategies for depthmap or pointmap reconstruction oftenfail on joint RGB and pointmap generation, quickly degrading the basevideo model. To address this challenge, we introduce Decoupled LoRAControl (DLC), which employs two modality-specific LoRA adapters toform decoupled computation branches for RGB frames and pointmaps,connected by lightweight, zero-initialized control links that graduallylearn mutual pixel-level consistency. Trained on a mixture of syntheticand real 4D datasets under modest computational budgets, One4D pro-duces high-quality RGB frames and accurate pointmaps across both gen-eration and reconstruction tasks. This work represents a step towardgeneral, high-quality geometry-based 4D world modeling using video dif-fusion models. Project page: https://mizhenxing.github.io/One4D.

Generative world models increasingly rely on 4D occupancyfor realistic autonomous driving simulation. However, existing genera-tion frameworks are limited in two respects: geometric-condition-basedmethods depend on explicit trajectories and cannot generalize to free-form instructions, while text-based methods operate at a coarse attributelevel and fail to orchestrate complex, sequential multi-agent interactions.We propose OccDirector, a framework that generates 4D occupancy dy-namics conditioned solely on natural language scripts, without requir-ing any geometric priors. OccDirector encodes free-form scripts via afrozen VLM and processes occupancy tokens through a Spatio-TemporalMMDiT backbone. A history-prefix anchoring strategy further ensuresthat generated sequences remain consistent with any provided contextover long horizons. To support training and evaluation, we introduceOccInteract-85k, a dataset of 85k clips annotated with multi-level lan-guage instructions spanning static layouts to intricate multi-agent be-haviors, alongside a novel VLM-based evaluation benchmark. Extensiveexperiments demonstrate state-of-the-art generation quality and stronginstruction-following fidelity across diverse driving scenarios.

Diffusion models have recently advanced text-to-video (T2V)generation, yet they still struggle with fine-grained compositional align-ment, such as attribute binding, spatial relations, and object interactions.While reward-based fine-tuning improves alignment, it is susceptible toreward hacking and adapts poorly to new prompt distributions. In thiswork, we propose NoisEasier, a test-time scaling framework that im-proves T2V generation through differentiable reward-guided noise opti-mization without modifying the underlying model. By combining effi-cient short-step generators with a multi-objective reward formulation,NoisEasier enables stable and practical test-time optimization under re-alistic inference budgets. Our key insight is that jointly optimizing theentire stochastic trajectory accelerates reward convergence and improvescompositional alignment over optimizing only the initial latent, with neg-ligible additional computational and time cost. Experiments on VBenchand T2V-CompBench demonstrate consistent improvements across mul-tiple backbones, achieving over 10% average gains on challenging dimen-sions such as attribute binding, object interaction, and numeracy. Over-all, NoisEasier serves as both a flexible alternative and a complementaryenhancement to reward-based fine-tuning, establishing test-time scalingas an effective paradigm for controllable text-to-video generation.

Thermal-to-visible face translation presents fundamental challenges including geometric discontinuities, semantic attribute mismatches, and identity degradation. We propose MTVDiff, a novel multimodal latent diffusion framework that synergistically integrates depth and textual information to address these limitations while preserving identity characteristics. The MTVDiff framework presents three core technical contributions: (1) a Dual-Branch Cross-Attention Fusion (DBCAF) module for multi-scale thermal-depth feature extraction and fusion; (2) a Gated Text-to-Visual Feature Alignment mechanism for semantically-guided generation; and (3) Spatial Feature Transformations (SFT) for adaptive multimodal prior integration. Extensive experiments on the MCXFace and SpeakingFaces datasets demonstrate that our multimodal approach significantly outperforms existing GAN-based and diffusion-based approaches across multiple metrics, achieving substantial improvements in both image quality and face verification performance, with FID reductions of up to 48.3% and Rank-1 accuracy improvements of up to 8.9%. Our work provides a robust solution for face recognition systems operating under varying illumination conditions and advances the state-of-theart in cross-spectral facial image translation through effective multimodal integration.

Makeup-transfer applies the reference makeup to the sourceface while preserving the source identity. Despite advances in full-faceediting by diffusion-based methods, strong regional controllability, makeupfidelity, and identity preservation remain challenging. The reasons are(i) pixel-to-attention misalignment that causes spillover into non-targetareas and weakens regional control; (ii) unclear transfer/preservationconcept separation under two-image conditioning, leading to couplingbetween makeup attributes and identity; and (iii) the lack of a high-resolution dataset that is identity-consistent and region-labeled for fine-grained supervision. In this paper, we propose MagicMakeup, a dif-fusion transformer-based framework for region-controllable and high-fidelity makeup transfer, built on spatial constraints and concept dis-entanglement. To enable precise region-specific editing while preservingidentity, we propose Token-Aligned Region Gating, which aligns pixelmasks with attention and applies region-specific logit gating. To clarifythe concepts of transfer and preservation, we further introduce Cross-Modal Perception Guidance, which aligns text and image features to en-hance cross-modal concept perception. We also design a pipeline for thegeneration of 1024 × 1024 data pairs through region-specific makeup re-moval and establish a unified benchmark in synthetic and real settings.Extensive quantitative and qualitative experiments show that Magic-Makeup improves regional controllability, makeup fidelity, and identitypreservation, with strong robustness across styles, races, and poses.