ECCV 2026 Papers

Large Multimodal Models (LMMs) have achieved remark-able success on images and short videos, yet scaling them to long videosremains challenging due to frame-centric tokenization and limited con-text windows. 3D geometry provides a natural compression mechanismfor visual streams: depth and camera pose enable observations from mul-tiple views and time steps to be fused into a persistent, world-aligned rep-resentation. While recent 3D LMMs leverage geometry-aware representa-tions to improve spatial reasoning, they continue to lag behind specialist3D perception systems on grounding and segmentation tasks. We arguethat a key limitation is geometry-aware decoding: existing methods com-municate 3D predictions through language tokens, proposal selection, orlightweight grounding queries, creating a bottleneck between languagereasoning and dense geometric prediction. Building on these insights, weintroduce Qwen-3D, a geometry-aware LMM that compresses visual in-formation within the Qwen backbone using multi-view geometric cues,enabling efficient long-horizon visual reasoning over static scenes. Qwen-3D augments visual tokens with 3D Rotary Positional Embeddings, al-lowing attention to operate directly in 3D scene space rather than acrossindependent image frames and thereby facilitating scalable cross-viewand temporal reasoning. To bridge language and geometry, Qwen-3D in-corporates a query-based segmentation decoder that grounds languagedirectly in the underlying 3D scene representation, unifying referentialgrounding, instance segmentation, and visual question answering acrossboth images and videos. Across a diverse set of benchmarks, Qwen-3Dsurpasses existing 3D LMMs and outperforms several large proprietary2D models. Notably, Qwen-3D achieves these improvements while main-taining strong performance on standard 2D vision–language benchmarksby jointly training on 2D and 3D data. Our code and checkpoints can befound at the project website https://qwen-3d.github.io/.

We introduce Point-supervised Multi-Object Tracking (PS-MOT) as a cost-effective alternative to traditional bounding box su-pervision, shifting the focus from spatial fitting to topological center-driven representation. However, PS-MOT faces challenges, e.g., spatialambiguity and identity drift due to the lack of explicit geometric struc-ture and scale constraints. To address these, we propose PS-Track, ahierarchical pipeline transitioning from points to instances across data,model, and loss levels. At the data level, we introduce Temporal-FeedbackPrompting (TFP) to evolve points into temporally consistent pseudo-labels using negative spatial cues and motion priors. At the model level,we design the Point-Excited Wavelet Attention (PEWA) module, whichleverages semantic correlations to activate high-frequency components,“hallucinating” object boundaries. At the loss level, Uncertainty-GuidedGaussian Learning (UGL) models pseudo-labels as probabilistic distri-butions, dynamically calibrating supervision intensity. Experiments onDanceTrack, EmboTrack, SportsMOT, and JRDB demonstrate that PS-Track provides a feasible and effective point-supervised alternative acrossdiverse tracking scenarios, establishing a new state-of-the-art for point-supervised tracking. The source code is available at https://github.com/xifen523/PS-MOT.

Although autoregressive (AR) models have demonstrated re-markable success in image generation, extending these models to layout-conditioned generation remains challenging due to the sparse natureof layout conditions and the risk of feature entanglement. We presentStructured Masking for AR-based Layout-to-Image (SMARLI), a novelframework that effectively integrates spatial layout constraints into theAR generation process. To equip AR models with layout control, a struc-tured masking strategy is applied to the attention computation to governthe interaction among the global prompt, layout, and image tokens. Thisdesign prevents the misassociation of different regions with their corre-sponding descriptions while enabling the sufficient injection of layoutconstraints into the generation process. To alleviate the exposure bias ofAR models and further enhance generation quality and layout accuracy,we incorporate a Group Relative Policy Optimization (GRPO) post-training scheme. We adapt it to the next-set-based paradigm and intro-duce a specifically designed layout reward, which is coordinated with animage quality reward to guide policy optimization in a balanced manner.Experimental results demonstrate that SMARLI seamlessly integrateslayout tokens with text and image tokens without compromising genera-tion quality, and the proposed masking strategy and post-training schemecan also be transferred to standard next-token-based AR models. Theproposed framework achieves superior layout control while maintainingthe structural simplicity and generation efficiency of AR models.

Recent advancements in Unified Multimodal Models (UMMs)have significantly advanced text-to-image (T2I) generation, particularlythrough the integration of Chain-of-Thought (CoT) reasoning. How-ever, existing CoT-based T2I methods largely rely on abstract natural-language planning, which lacks the precision required for complex spa-tial layouts, structured visual elements, and dense textual content. In thiswork, we propose CoCo (Code-as-CoT), a code-driven reasoning frame-work that represents the reasoning process as executable code, enablingexplicit and verifiable intermediate planning for image generation. Givena text prompt, CoCo first generates executable code that specifies thestructural layout of the scene, which is then executed in a sandboxed en-vironment to render a deterministic draft image. The model subsequentlyrefines this draft through fine-grained image editing to produce the fi-nal high-fidelity result. To support this training paradigm, we constructCoCo-10K, a curated dataset containing structured draft–final imagepairs designed to teach both structured draft construction and correc-tive visual refinement. Empirical evaluations on StructT2IBench, OneIG-Bench, and LongText-Bench show that CoCo achieves improvements of+68.83%, +54.8%, and +41.23% over direct generation, while also out-performing other generation methods empowered by CoT. These resultsdemonstrate that executable code is an effective and reliable reasoningparadigm for precise, controllable, and structured text-to-image genera-tion. The code is available at: https://github.com/micky-li-hd/CoCo

Reconstructing 4D animals from monocular videos is chal-lenging due to large inter-species variation, complex articulations, andthe lack of reliable templates. Existing approaches typically rely on eitherstrict category-specific priors that restrict generalization, or unconstrainedgenerative models that sacrifice input fidelity. To bridge this gap, wepresent a progressive test-time optimization framework built on 3D Gaus-sian Splatting for high-fidelity 4D animal reconstruction from a singlevideo. Our key insight is that a coarse shape prior suffices when coupledwith a progressive strategy that disentangles articulated pose from non-rigid deformation. Specifically, we employ a symmetry-aware temporalencoding that exploits bilateral cues while absorbing camera estimationdrift and a part-conditioned deformation mechanism guided by learn-able part anchors and a learnable skinning field. Extensive experimentsdemonstrate that our approach generalizes robustly across diverse species,achieving superior geometric accuracy, temporal consistency, and visualfidelity compared to existing baselines, even under severe prior mismatch.Project page: https://syl-322.github.io/ReWild4D/

Despite rapid progress in video diffusion models (VDMs),ensuring semantic adherence and physical commonsense remains a fun-damental challenge, as even state-of-the-art systems frequently violatereal-world physical laws, such as dynamics, thermodynamics, and op-tics. While Direct Preference Optimization (DPO) has become a popu-lar post-training strategy for aligning generative models, existing videoDPO methods to improve physical commonsense suffer from high compu-tational costs, poorly matched preference pairs, and ambiguous global su-pervision that fails to localize physical violations. We propose PhysPO,a physics-aware local preference optimization framework for physicallyconsistent video generation. We first introduce CounterPhyPipe, a physics-aware counterfactual data construction pipeline that forms video pref-erence pairs with consistent global semantics and local physical viola-tions, enabling meaningful comparisons for preference learning. Thenwe leverage high-frequency decomposition to introduce a physical mask,which localizes regions where physical state transitions may occur. Fi-nally, we develop PhysPO, a physics-aware DPO framework that con-centrates supervision on physically relevant regions while enforcing neu-trality on background regions. This mechanism reduces gradient noiseand mitigates shortcut optimization, encouraging the model to focus ongenuine physical discrepancies rather than superficial cues. Extensiveexperiments demonstrate that PhysPO significantly improves physicalcommonsense without compromising semantic adherence. Our projectpage is here3 .

Dense video captioning aims to generate temporally groundeddescriptions of video events, benefiting both event-level video understand-ing and generation. In this domain, autoregressive video large languagemodels have emerged as a prevalent paradigm due to their strong gen-erative and cross-modal modeling capacity. However, generating densecaptions under the token-by-token paradigm severely limits inference effi-ciency and hinders scalability as video length and event density increase.In this work, we propose a parallelized autoregressive framework thatnot only improves generation efficiency but also enhances temporallygrounded captioning performance. Our key insight is to exploit the weaklocal dependencies across temporally distinct events to restructure thecausal dependency graph, thereby enabling lossless parallel generation.Specifically, tokens with weak cross-event dependencies can be decoded inparallel, while tightly coupled tokens within each event retain sequentialdecoding to preserve local semantic coherence. To realize this insight, weintroduce two key components for lossless parallel decoding: (1) a latentglobal planning mechanism that automatically learns the event-level struc-ture and produces compact tokens encoding global inter-event causalitywhile adaptively aggregating event-level audio-visual semantics, guidingsubsequent dependency restructuring and parallel decoding; and (2) anevent-factorized parallel decoding mechanism that effectively balanceslocal focus with global inter-event awareness. Experiments on variousbenchmarks demonstrate the clear advantage of our approach in bothefficiency and performance in omni-modal event grounding and captioning.Project website: https://github.com/showlab/PadCaptioner.

3D Visual Grounding (3DVG) is a critical bridge from vision-language perception to robotics, requiring both language understandingand 3D scene reasoning. Traditional supervised models leverage explicit3D geometry but exhibit limited generalization, owing to the scarcity of3D vision-language datasets and the limited reasoning capabilities com-pared to modern vision-language models (VLMs). We propose a gener-alizable 3DVG framework, PanoGrounder, that couples multi-modalpanoramic representation with pretrained 2D VLMs for strong vision-language reasoning. Panoramic renderings, augmented with 3D semanticand geometric features, serve as an intermediate representation between2D and 3D, and offer two major benefits: (i) they can be directly fed toVLMs with minimal adaptation and (ii) they retain long-range object-to-object relations thanks to their 360-degree field of view. We devise athree-stage pipeline that places a compact set of panoramic viewpointsconsidering the scene layout and geometry, grounds a text query on eachpanoramic rendering with a VLM, and fuses per-view predictions into asingle 3D bounding box via lifting. Our approach achieves state-of-the-artresults on ScanRefer and Nr3D, and demonstrates strong generalizationto unseen 3D datasets and text rephrasings.

Existing feature coding methods typically minimize feature-space reconstruction error, implicitly assuming that preserving featuresimilarity ensures downstream task performance. However, in VisionTransformers (ViTs), global correlation mixing makes downstream pre-dictions more sensitive to small feature perturbations, so similar fea-ture distortion can lead to significantly different task performance. Inparticular, our analysis reveals that under fixed-group product quanti-zation, task sensitivity is highly uneven across channel subspaces. Con-sequently, near-uniform per-group rates may waste bits on insensitivedirections while leaving critical ones underrepresented, causing rapidtask-performance degradation at low bitrate. To address this issue, wepropose an Orthogonal Reparameterized product quantization for inter-mediate Feature Coding (ORFC). It introduces a learnable orthogonaltransform that rotates the representation before grouped quantization,enabling fixed-size groups to better align with task-relevant directions.We further use the output deviation of the remaining layers as a proxydistortion, and optimize an entropy-constrained rate-distortion objectiveto jointly learn the orthogonal transform and grouped quantization in anend-to-end manner. Extensive experiments across multiple ViT architec-tures and downstream tasks demonstrate that ORFC achieves a superiorrate-task trade-off, especially in the low-bitrate regime. The code is avail-able at https://github.com/zhangletian2/ORFC.

Optimizing vertex deformations from video for mesh anima-tion is constrained by rendering-based reconstruction losses. While ex-isting approaches improve mesh representations, supervision signals, oranimation paradigms, their supervision remains confined to the 2D do-main. Such 2D supervision is limited: it provides no signal for occludedregions and only indirect cues for visible areas. Consequently, these meth-ods often suffer from severe shape and motion artifacts. To address thislimitation, we propose Shape Flow Guidance (SFG), a sequence of 3Dshapes derived from videos, which serves as explicit 3D supervision formesh animation. Specifically, SFG is elicited by intervening in the sam-pling process of a pretrained mesh generator in a training-free manner.We further tailor a skeletal animation model that separates local defor-mation from global transformation. This model enables SFG to guidecomplex local motion while reserving rendering-based losses for simpleglobal motion. Extensive experiments confirm that our method signifi-cantly outperforms prior work qualitatively, quantitatively, and in termsof processing speed. Qualitative results are available on our project page:https://sfgmesh.pages.dev/.

Most existing mesh-based methods for garment simulationsuffer from sensitivities tied to mesh discretisation and resolution. Re-cent approaches such as NeuralClothSim employ continuous neural fieldsas a promising alternative, but remain constrained by a 2D curvilinearparameterisation that models a single cloth panel, limiting their abil-ity to represent garments with multiple panels stitched together. Thispaper introduces NeuralGarSim, a quasistatic garment simulator for-mulated directly in 3D Euclidean space. Our method accepts diverseinput representations—including distance fields, meshes, point clouds,and Gaussians—as undeformed garment states, making it compatible withoff-the-shelf reconstruction frameworks without additional preprocessing.We then represent the garment deformation as a neural field and define anonlinear Kirchhoff–Love shell model directly in R3 by applying tangentialdifferential calculus on the garment surface. This ensures that the neuralgarment simulation is parameterisation-space-free and naturally extendsto arbitrary topologies. By minimising a potential energy functional, ourmethod learns a 3D neural deformation field that predicts physicallyconsistent deformations across garments with multiple panels, seams,and holes. It supports multiple material models and trains 2–4× fasterthan its predecessor, NeuralClothSim, while retaining the continuous,consistent, and memory-adaptive behaviour1 .

Achieving precise, versatile whole-body character control inphysics-based animation remains challenging. Recent diffusion-based poli-cies generate rich and expressive motions but typically rely on gradient-based test-time guidance to satisfy task objectives, which is slow andcan reduce robustness. We introduce NaP-Control (Navigating Diffu-sion Prior for Versatile and Fast Character Control), abbreviated asNaP. Our method uses reinforcement learning to manipulate the latentnoise of a task-agnostic diffusion policy prior, steering it toward task-specific behaviors for fast, robust control with high motion fidelity. Incontrast to methods that rely solely on offline training, NaP interactswith the environment during training to correct motions and optimizetask rewards, improving success rates and enabling adaptation to chal-lenging scenarios. By directly predicting task-optimized diffusion noise,NaP eliminates iterative guidance during denoising and enables efficientinference. Experiments show that NaP attains higher success rates andfaster inference while preserving natural motion across diverse tasks.

Recent Vision-Language Models (VLMs) have demonstrated significant progress in video understanding. However, we notice that VLMs perform coarse perception and reasoning when answering finegrained motion questions. We investigate the reasons for this phenomenon and discover that motions in video change rapidly, so fixed frame rate sampling easily misses inter-frame motion details. Moreover, when answering motion-centric questions, VLMs tend to introduce motion-irrelevant contextual information and lack a structured and precise representation of motions, making it difficult to distinguish what motion the subject performs and when. To address these problems, we propose MotionChain, a training-free method that decouples motion question answering into a sequence of ⟨time range, subject, motion, environment⟩ tuples. Concretely, MotionChain leverages optical flow variations to calibrate when motions occur and extracts motion-relevant key frames as supplementary input, effectively alleviating the problem of missing critical motion information. For fine-grained motion question answering (QA), each tuple element in MotionChain can be converted into a visually grounded claim, providing the model with visual evidence. Experiments on motion-centric video QA benchmarks demonstrate the superiority of our MotionChain.

Multi-exposure fusion (MEF) expands the luminance rangebeyond what a single exposure can capture. Combining images takenat different exposure levels requires handling geometric differences whilenaturally merging their complementary brightness information. It of-ten demands generative completion where details are missing. Diffusion-based generative methods address these challenges, however, they arecomputationally expensive and struggle to preserve fine structures in sat-urated regions. We propose LIIFusion, a coarse-to-fine framework thatbalances fusion quality and efficiency in generative MEF. The coarsestage performs low resolution generative fusion, enhanced by an adaptiveexposure correction that recovers structure lost in saturated over-exposedareas. The fine stage adapts a local implicit image function into a multi-exposure fusion function: conditioned on the HR OE/UE sources and thecoarse output, it queries arbitrary target coordinates and fuses sourceevidence regard- less of the HR input resolution. LIIFusion achieves upto 3.5× speed-up over existing generative methods while maintainingor improving structural fidelity and perceptual quality. We believe thisframework provides an effective pathway toward making generative MEFmore practical in real-world applications.

Multimodal large language models (MLLMs) have rapidlyadvanced video understanding, achieving strong zero-shot and few-shotrecognition across standard benchmarks. Yet their ability to deny anaction by recognizing when an activity is not happening despite strongcontextual cues remains largely unexplored. We introduce UCF101-AD, a large-scale benchmark consisting of paired Action-Presence andAction-Denial clips, designed to evaluate this capacity for denial. Eachnegative video in UCF101-AD preserves the same contextual and mo-tion cues (persons, objects, locations) as its positive counterpart, butthe defining action itself is explicitly absent. Evaluating 20 state-of-the-art MLLMs reveals a consistent failure: models that exceed 85% accu-racy on the positive action classes collapse below 50% on its action-denial counterpart, indicating a strong inclination to affirm plausibleactions rather than verify that they truly occur. This exposes a criti-cal blind spot in modern video understanding: the inability to reasoncausally about whether a motion actually happens. To probe this is-sue, we explore a causal graph formulation, CausalAct, which expressesscene structure through natural-language prompts linking context, in-teraction, and motion. Incorporating such causal cues substantially re-duces false positives, demonstrating that denial is a learnable reason-ing skill. UCF101-AD provides a new lens for diagnosing and improv-ing causal reasoning in multimodal models. Dataset and relevant code:https://github.com/raiyaan-abdullah/Learn-to-Deny.

Latent diffusion models have emerged as a dominant frame-work for high-fidelity image and video synthesis, operating in compactlatent spaces with variational autoencoders (VAEs) to enhance compu-tational efficiency without compromising visual quality. However, con-ventional VAEs are suboptimal for video data as they employ fixed com-pression ratios that cannot adapt to the varying complexity of spatio-temporal content. We present KATok (Keep-or-Drop? Adaptive Tok-enizer for Compact Video Representation), a transformer-based VAEthat incorporates an adaptive token selector which is jointly learned withlatent tokens. By evaluating each token’s content-richness as keep-or-dropprobability, the token selector effectively discards uninformative tokens,naturally allowing data-dependent compression. Applying adaptive tok-enization to diffusion models may cause spatial misalignment, as tokendropping can disturb the original spatio-temporal structure. To allevi-ate this issue, we propose two position-prediction strategies: cascadedand joint generation, to ensure spatial consistency. We empirically showthat our model achieves strong reconstruction and generation quality at astate-of-the-art compression ratio. Further analysis on video data revealsthat this improvement is primarily achieved by reducing spatio-temporalredundancy and removing uninformative tokens, as supported by bothquantitative and qualitative results.

Current diffusion models face a fundamental tension betweenthe computational efficiency of latent-space generation and the fine-detailfidelity of pixel-space generation. Latent Diffusion Models (LDMs) relyon VAE compression, which can discard high-frequency visual cues andlimit pixel-level quality, while pixel-space diffusion often suffers from diffi-cult optimization when global structure and high-frequency details mustbe learned jointly. In this paper, we propose the Hybrid Latent-PixelDiffusion Transformer (Hi-DiT), a unified Diffusion Transformer thatbridges these two regimes within a single architecture. Motivated by thetemporal heterogeneity of denoising—early timesteps primarily establishcoarse global structure, whereas late timesteps increasingly emphasizehigh-frequency detail—we introduce a dual-stream design with temporalspecialization. Hi-DiT performs semantic planning in a compact latentpathway at early stages, and activates a pixel pathway at later stagesto synthesize high-frequency details using time-consistent noisy pixelembeddings. A Time-Gated Injection mechanism schedules the partic-ipation of pixel tokens only in the low-noise regime, and a lightweightsub-pixel prediction head enables efficient dense detail generation. Ex-tensive experiments demonstrate that Hi-DiT achieves state-of-the-artperformance, obtaining an FID of 1.06 on ImageNet 256 × 256, 1.26 onImageNet 512 × 512, and 5.15 on MS-COCO text-to-image generation.Code is available at: https://github.com/HiDream-ai/Hi-DiT.

Natural-language queries about 3D environments become actionable when responses are verifiable and metric. Verifiability requires explicit grounding to the referred 3D region, while metric answers report physical measurements in real-world units (e.g., size, thickness, clearance, and distance). Existing 3D large multimodal models (LMMs) approaches remain limited: conversational systems typically respond without explicit 3D grounding, while 3D grounding models are not designed for interactive, metric-aware dialogue. In this paper, we present Ground3DLMM, a unified model that takes a point cloud and an optional RGB image as input and supports 3D spatial conversation with (i) pointgrounded responses and (ii) metric numeric outputs at both object and part granularity, including multi-object queries. To evaluate this intersection of grounding and measurement, we define the 3D Grounded Measurement task, which requires predicting the referred 3D region and the corresponding metric quantities in real-world units. We introduce a large-scale dataset built on ScanNet and ScanNet++ datasets with dense object and part annotations and roughly 2.5M question-answer pairs spanning eight tasks, along with a manually verified test set. Extensive experiments on multiple datasets and tasks show that our proposed Ground3D-LMM model provides a strong baseline for grounded, metric-aware 3D conversational understanding. Our dataset and model are publicly available.

Existing instruction-based video editing datasets commonlyfocus on single-task appearance editing, failing to meet the complex cre-ative demands of real-world scenarios. To bridge this gap, we presentGoku, a large-scale dataset featuring 2 million high-quality, instruction-aligned video editing pairs, which is the first to extend task boundariesfrom basic appearance editing to multi-task and structural manipulations(e.g., precise control of subject movement). To tackle the data synthesischallenges inherent in these complex tasks, we design an efficient datasynthesis pipeline that decomposes complex edits into controllable sub-problems and introduce a progressive filtering system for data reliabil-ity throughout the whole process. Furthermore, we explore the optimalnetwork structures on Goku, and propose Goku-Edit. To deeply com-prehend complex editing instructions, Goku-Edit leverages an MLLMas its text encoder and adopts a decoupled dual-branch design: a dedi-cated mask branch handles structural control, freeing the main branchfor appearance rendering. A comprehensive video editing benchmark,Goku-Bench, is also proposed with 1,000 human-verified test cases and7 novel editing-specific metrics. Evaluated on Goku-Bench, Goku-Editobtains up to +8% improvement on other open-source models in termsof instruction following.