ECCV 2026 论文列表

We present ’RenderFormer-V2’, a unified learned transformer-based neural rendering model, complementary to modern physics-basedrendering systems, that can handle diverse light-transport effects suchas caustics, volumetric scattering, environment lighting, textured anddisplaced surfaces and out-of-distribution materials without per-scenetraining or specialized code. RenderFormer-V2 models global light trans-port as a sequence-to-sequence transformation. Following its predecessor,RenderFormer-V2 also employs a two stage process: a view-independentstage that resolves intra-scene primitive to primitive transport, and aview-dependent stage that transforms the internal neural scene repre-sentation into image pixels. Different from RenderFormer, our modelemploys a novel combined windowed-attention and rendering-informedattention sink in the view-independent stage to improve scalability whilemaintaining render accuracy. To further improve versatility, RenderFormer-V2 supports heterogeneous scene primitives, including environment mapsand participating media, and it employs a material encoding indepen-dent of the underlying surface reflectance model that encodes materialappearance via a novel neural embedding. We demonstrate the versatil-ity of RenderFormer-V2 on a variety of scenes and perform an extensiveablation of the improved attention mechanism.

The reconstruction of dynamic 3D scenes using 3D GaussianSplatting has shown significant promise. A key challenge, however, re-mains in modeling realistic motion, as most methods fail to align the mo-tion of Gaussians with real-world physical dynamics. This misalignmentis particularly problematic for monocular video datasets, where failingto maintain coherent motion undermines local geometric structure, ulti-mately leading to degraded reconstruction quality. Consequently, manystate-of-the-art approaches rely heavily on external priors, such as opti-cal flow or 2D tracks, to enforce temporal coherence. In this work, wepropose a novel method to explicitly preserve the local geometric struc-ture of Gaussians across time in 4D scenes. Our core idea is to introducea view-space ray grouping strategy that clusters Gaussians intersectedby the same ray, considering only those whose α-blending weights ex-ceed a threshold. We then apply constraints to these groups to maintaina consistent spatial distribution, effectively preserving their local geom-etry. This approach enforces a more locally coherent motion model byensuring that local geometry remains stable over time, eliminating thereliance on external guidance. We demonstrate the efficacy of our methodby integrating it into two distinct baseline models. Extensive experimentson challenging monocular datasets show that our approach significantlyoutperforms existing methods, achieving superior temporal consistencyand reconstruction quality.

Predicting human gaze in video is fundamental to advanc-ing scene understanding and multimodal interaction. While traditionalsaliency maps provide spatial probability distributions and scanpaths of-fer ordered fixations, both abstractions often collapse the fine-grainedtemporal dynamics of raw gaze. Furthermore, existing models are typi-cally constrained to short-term windows (≈ 3–5s), failing to capture thelong-range behavioral dependencies inherent in real-world content. Wepresent a generative framework for infinite-horizon raw gaze predictionin videos of arbitrary length. By leveraging an autoregressive diffusionmodel, we synthesize gaze trajectories characterized by continuous spa-tial coordinates and high-resolution timestamps. Our model is condi-tioned on a saliency-aware visual latent space. Quantitative and qualita-tive evaluations demonstrate that our approach significantly outperformsexisting approaches in long-range spatio-temporal accuracy and trajec-tory realism. Project website: https://www.immersivecomputinglab.org/publication/infinite-gaze/.

Multimodal Large Language Models (MLLMs) inherit richrelational priors from their language backbones, yet often fail when askedto apply these relationships in visual contexts. We trace this failure to astructural blind spot: projection-based alignment trains each visual to-ken to carry the right semantics, but never asks whether the relationshipsbetween concepts survive the crossing from language to vision. To ad-dress this, we propose MIRROR (Mapping Inter-concept Relationsfrom language to visual Representation via Optimal-transport-basedRegularization), a geometric regularization framework that transfers re-lational priors from language to vision by exploiting the rich relationalstructure encoded in language representations. Specifically, we derive asurrogate loss from the proposed Semi-Inverse Gromov–Wasserstein(SI-GW) problem, an inverse geometric problem that aligns visual rep-resentations with language-derived relational priors. We show that thisformulation admits a unique closed-form solution that prescribes theideal visual relational structure implied by language geometry and cross-modal coupling. The structure of the formulation also enables efficientcomputation, making it applicable to long token sequences. ApplyingSI-GW inside decoder-only Transformers requires careful design. We in-troduce targeted strategies at the layer, head, and token levels to ensurestable extraction without additional parameters or inference cost. MIR-ROR improves relational consistency while preserving performance ongeneral vision-language tasks.

This paper presents CountEx, a discriminative visual count-ing framework designed to address a key limitation of existing prompt-based methods: the inability to explicitly exclude visually similar distrac-tors. While current approaches allow users to specify what to count viainclusion prompts, they often struggle in cluttered scenes with confus-able object categories, leading to ambiguity and overcounting. CountExenables users to express both inclusion and exclusion intent, specifyingwhat to count and what to ignore, through multimodal prompts includ-ing natural language descriptions and optional visual exemplars. At thecore of CountEx is a novel Discriminative Query Refinement module,which jointly reasons over inclusion and exclusion cues by first identi-fying shared visual features, then isolating exclusion-specific patterns,and finally applying selective suppression to refine the counting query.To support systematic evaluation of fine-grained counting methods, weintroduce CoCount, a benchmark comprising 1,780 videos and 10,086 an-notated frames across 97 category pairs. Experiments show that Coun-tEx achieves substantial improvements over state-of-the-art methods forcounting objects from both known and novel categories. The data, code,and model are available at https://github.com/bbvisual/CountEx.

Video generation is central to AI-powered content creation.The alignment with human preferences is one of the key metrics for mea-suring the quality of the generated videos. Despite significant progress invisual quality, three key challenges remain: 1) The reliability of rewardsignals is constrained by the quality of human preference data, which isoften corrupted by subjective noise and bias. 2) Standard scalar rewardmodels collapse multi-aspect human preference into a single value, lead-ing to the loss of dynamic trade-offs across multiple dimensions of humanpreference. 3) In policy optimization, the widely adopted KL-divergenceimposes only local constraints, failing to capture holistic human pref-erence. To address these challenges, we propose a unified, preference-aware learning framework for video generation. First, we propose elite-guided filtering to calibrate preference data and construct reliable su-pervision for reward-model training. We then model video quality asa multidimensional reward distribution to capture the uncertain na-ture of human preference, and use the Wasserstein distance to alignit with the empirical human preference distribution. Finally, we intro-duce Wasserstein-based distributional alignment in GRPO, guiding thepolicy’s video generation to match the global structure of human videopreference. Experiments on reward modeling and video generation showthat our approach improves the reliability of reward signals and theperceptual consistency of generated videos. Our code is available athttps://github.com/alignhs26/ahs.

Generating temporally continuous and socially coherent hu-man motion from text remains a fundamental challenge, particularly inrealistic streams where people act alone, enter interactions, and later dis-engage. Most existing methods generate fixed-length motion clips understatic agent configurations, which makes them brittle to solo–social tran-sitions and unsuitable for incremental generation over long horizons. Wepropose ARMS, an Anchor–Relational Motion Streaming framework thatunifies solo motion and human–human interaction within a single causalgenerative process. ARMS introduces a dynamics-asymmetric represen-tation that decouples per-person temporal evolution from inter-personalignment via a partner-referenced relative-translation term, enablingseamless switching of social coupling without sacrificing long-horizonstability or spatial consistency between agents. On top of a causal la-tent space, a causal relational diffusion model progressively refines mo-tion segment by segment using only past context, capturing both intra-person temporal dependencies and inter-person relations. Mode-awarerelational gating activates or masks cross-agent connections, allowingthe same model to support both solo and interaction generation. Ex-periments show that ARMS improves transition smoothness and socialcoherence compared to interaction-centric baselines, while also achievingcompetitive results on human–human interaction benchmarks.

Human-human collaboration is a fundamental aspect of ev-eryday life, essential to success in a wide range of goal-directed activitiesfrom household tasks to professional teamwork. While much researchhas focused on modeling coordination and task execution, the cognitiveprocesses that support such collaboration, particularly Theory of Mind(the ability to infer others’ mental states), remain difficult to study innatural settings. To address this gap, we introduce a novel egocentric andexocentric video dataset capturing real-world collaboration in cookingscenarios. The dataset integrates multi-perspective video, high-qualityaudio, gaze tracking, and 3D scene and object scans, with annotations forshared gaze attention, social cues and interactions between agents, as wellas agent-object interactions. We establish benchmarks for Joint AttentionEstimation, Action Anticipation, and Collaborative Handover Prediction,enabling research on multimodal perception, proactive assistance, andcollaborative planning. By providing temporally aligned, richly annotatedmultimodal data, CoMind facilitates the development and evaluation ofAI systems capable of modeling complex social interactions and reasoningabout human behaviors in collaborative environments. Our dataset andbenchmarks are made available at https://comind.ethz.ch/.

Vision encoders are a critical component of vision-languagemodels, and scaling their capacity effectively improves performance. How-ever, dense scaling increases compute cost and inference latency. Mixture-of-Experts (MoE) architectures offer a compelling alternative, having en-abled efficient scaling in LLMs, yet the MoE design space for CLIP-stylevision encoders remains underexplored at State-of-the-Art (SOTA) lev-els. In this work, we systematically study MoE designs for vision encoderscaling and find that fine-grained MoE topologies yield substantial gainsover both dense and standard MoE counterparts. We further proposean auxiliary-loss-free balancing variant for better expert utilization, anddesign a specialized MoE kernel to mitigate inference latency overhead.To enhance video capabilities while preserving image knowledge, we in-troduce frame-level distillation paired with a novel freezing mechanism.We pretrain a series of Mixture-of-Experts Vision Encoders (MoE-ViE)across a range of sizes, all consistently outperforming their dense counter-parts. Our largest model matches the zero-shot performance of a SOTAencoder 1.7× its size at 76% of its latency. When aligned with an LLM,MoE-ViE surpasses all compared encoders on image and video bench-marks, including those with up to 5× more activated parameters.

Generating motion-controlled videos—where user-specified actions drive physically plausible scene dynamics under freely chosen viewpoints—demands two capabilities: (1) disentangled motion control, allowing users to separately control the object motion and adjust camera viewpoint; and (2) motion causality, i.e., generative action–consequence dependency, ensuring that user-driven actions trigger coherent reactions from other objects rather than merely displacing pixels. Existing methods fall short on both fronts: they entangle camera and object motion into a single tracking signal and treat motion as kinematic displacement without modeling causal relationships between object motion. We introduce MoRight, a unified framework that addresses both limitations through disentangled motion modeling. Object motion is specified in a canonical static-view and transferred to an arbitrary target camera viewpoint via temporal cross-view attention, enabling disentangled camera and object control. We further decompose motion into active (user-driven) and passive (consequence) components, training the model to learn motion causality from data. At inference, users can either supply active motion and MoRight predicts consequences (forward reasoning), or specify desired passive outcomes and MoRight recovers plausible driving actions (inverse reasoning), all while freely adjusting the camera viewpoint. Experiments on three benchmarks demonstrate state-of-the-art performance in generation quality, motion controllability, and interaction awareness.

Building virtual cells with generative models to simulate cel-lular behavior in silico is emerging as a promising paradigm for acceler-ating drug discovery. However, prior image-based generative approachescan produce implausible cell images that violate basic physical and bio-logical constraints. To address this, we propose to post-train virtual cellmodels with reinforcement learning (RL), leveraging biologically mean-ingful evaluators as reward functions. We design seven rewards spanningthree categories—biological function, structural validity, and morpho-logical correctness—and optimize the state-of-the-art CellFlux modelto yield CellFluxRL. CellFluxRL consistently improves over CellFluxacross all rewards, with further performance boosts from test-time scal-ing. Overall, our results present a virtual cell modeling framework thatenforces physically-based constraints through RL, advancing beyond “vi-sually realistic” generations towards “biologically meaningful” ones.

Video understanding is often plagued by severe temporal re-dundancy, where processing dense frame sequences is both semanticallyinefficient and computationally expensive. This challenge is further am-plified when only a small subset of frames is truly relevant to the givenquery. In this paper, we propose a Query- and Content-Aware (QCA)keyframe selection framework that can select a compact yet information-rich set of frames from long videos. QCA first partitions the video intotemporal segments and estimates the information contribution of eachsegment by jointly modeling query relevance and content deviation, anddynamically allocates keyframe budget to each segment. Within eachsegment, QCA anchors on the most query-relevant frame and iterativelyincorporates additional frames to maximize diversity while maintaininghigh semantic relevance to the query. Crucially, our method requires noadditional training and can be seamlessly integrated into existing Video-LLMs. Extensive experiments across multiple long video understandingbenchmarks demonstrate that our proposed approach achieves state-of-the-art performance and has strong generalization ability. For instance,QCA achieves 67.8% on LongVideoBench using 128 frames, while GPT-4o achieves 66.7% using 256 frames. Our codes are available in GitHub.

Aligning video generative models to human preferences heav-ily relies on Reinforcement Learning (RL), which suffers from extensivecomputational overhead. Existing workflows typically treat RL and dis-tillation as disconnected stages: applying RL before distillation incursprohibitive computational costs, whereas applying RL after distillationfrequently leads to model collapse. To overcome these limitations, wepropose a unified, single-stage optimization framework grounded in Dis-tribution Matching (DM). In the standard DM framework, distillationupdates the model via a gradient direction that minimizes the gap be-tween the real and fake models, guiding generations toward clarity andhigh fidelity. Building upon this, we introduce DM-Align, which derivesa complementary gradient direction to guide the model toward human-preferred samples. Inspired by DPO and GRPO, our method leveragesthe distributional gap—formulated from either preference pairs or intra-group exploration—to directly construct this preference-guided gradient.By synergizing these two gradient directions, our approach eliminatesthe need for multi-step reward evaluation and complex ODE-SDE con-versions inherent in traditional RL. Comprehensive experiments acrossmultiple foundational video models demonstrate that this sample-guidedframework robustly enhances both distillation quality and preferencealignment, consistently outperforming both standalone variants and se-quential two-stage pipelines.

Radar sensors provide reliable perception under adverse weatherand lighting conditions, but their sparse, noisy, and weakly semanticmeasurements make dense semantic segmentation challenging. Most ex-isting radar segmentation methods rely on grid-based encodings andpairwise interactions, which struggle to capture the higher-order rela-tional structure formed by multiple radar returns from the same phys-ical object. We introduce a unified higher-order structural alignmentframework for multi-view radar segmentation. The proposed method re-fines radar feature representations using learnable hypergraphs to cap-ture higher-order dependencies among spatially related responses. To en-sure consistency across heterogeneous radar projections, we further alignview-specific features using Unbalanced Optimal Transport (UOT), en-abling correspondence-free alignment under varying measurement den-sities and partial observations. An adaptive attention mechanism thenfuses complementary radar views while emphasising structurally informa-tive responses under sparsity and noise. The resulting architecture learnsstructurally consistent representations across Range Angle (RA), RangeDoppler (RD), and Angle Doppler (AD) views and is trained using super-vised segmentation together with cross-view consistency regularisation.Experiments on the CARRADA and RADIal benchmarks demonstrateconsistent improvements over strong radar-specific baselines, achieving63.8% mIoU on CARRADA and 83.4% mIoU on RADIal, im-proving the previous best methods by +1.7 and +2.3 mIoU, respec-tively. These results highlight the importance of higher-order relationalmodelling for robust radar perception.

Current expressive avatar systems rely heavily on visual cuesand often fail when faces are occluded or emotions remain internal. Wepresent Mind-to-Face, the first framework to decode non-invasive elec-troencephalogram (EEG) signals directly into high-fidelity facial expres-sions. We build a dual-modality recording setup that captures synchro-nized EEG and multi-view facial video during emotion-eliciting stimuli,providing precise supervision for neural-to-visual learning. Our modeluses a CNN-Transformer encoder to map EEG signals into dense 3Dposition maps that sample over 65k vertices, capturing fine-scale geom-etry and subtle emotional dynamics, and renders them through a mod-ified 3D Gaussian Splatting pipeline for photorealistic, view-consistentresults. Extensive evaluations show that EEG alone can reliably predictdynamic, subject-specific facial expressions, including subtle emotionalresponses, demonstrating that neural signals contain far richer affectiveand geometric information than previously assumed. Mind-to-Face es-tablishes a new paradigm for neural-driven avatars, enabling personal-ized, emotion-aware telepresence and cognitive interaction in immersive

Visual Prompting (VP) has emerged as a parameter-efficient paradigm for adapting pre-trained models to downstream tasks. While existing VP methods predominantly operate in the input space, extending VP to deep activations confronts a fundamental Capacity-Integrity Paradox: increasing prompt capacity for better task adaptation inevitably triggers a parameter explosion and severe semantic disruption, where deep prompts override pre-trained representations. To strike a delicate tradeoff within this Capacity-Integrity Paradox, we conducted a series of preliminary experiments. Interestingly, these experiments reveal a strong correlation among prompt channels: applying shared prompts across grouped channels not only drastically reduces parameters but also improves performance. Motivated by this structural correlation and the need to systematically master the tradeoff, we propose Low-Rank Tensor Visual Prompting (LoTeP). By modeling deep prompts as low-rank tensors, LoTeP elegantly achieves extreme parameter compression. Concurrently, it enables precise control over the prompt capacity via layerwise rank decaying, effectively preserving the semantic integrity of deep activations. Extensive experiments demonstrate the superior generality, effectiveness, and efficiency of our approach. Overall, LoTeP consistently outperforms the state-of-the-art LoR-VP by an average margin of over 2.5% across all evaluated scenarios, while adding less than 0.1% of the backbone parameters over the LoR-VP baseline.

A major challenge in autonomous driving is the “long tail” ofsafety-critical edge cases, which often emerge from unusual combinationsof common traffic elements. Synthesizing these scenarios is crucial, yetcurrent controllable generative models provide incomplete or entangledguidance, preventing the independent manipulation of scene structure,object identity, and ego actions. We introduce CompoSIA, a composi-tional driving video simulator that disentangles these traffic factors, en-abling fine-grained control over diverse adversarial driving scenarios. Tosupport controllable identity replacement of scene elements, we proposea noise-level identity injection, allowing pose-agnostic identity generationacross diverse element poses, all from a single reference image. Further-more, a hierarchical dual-branch action control mechanism is introducedto improve action controllability. Such disentangled control enables ad-versarial scenario synthesis—systematically combining safe elements intodangerous configurations that entangled generators cannot produce. Ex-tensive comparisons demonstrate superior controllable generation qualityover state-of-the-art baselines, with a 17% improvement in FVD for iden-tity editing and reductions of 30% and 47% in rotation and translationerrors for action control. Furthermore, downstream stress-testing revealssubstantial planner failures: across editing modalities, the average colli-sion rate of 3s increases by 173%.

World models that forecast scene evolution by generatingfuture video frames devote the bulk of their capacity to photometricdetails, yet the resulting predictions often remain geometrically inconsis-tent. We present VGGT-World, a geometry world model that side-stepsvideo generation entirely and instead forecasts the temporal evolution offrozen geometry-foundation-model (GFM) features. Concretely, we re-purpose the latent tokens of a frozen VGGT as the world state and traina lightweight temporal flow transformer to autoregressively predict theirfuture trajectory. Two technical challenges arise in this high-dimensional(d=1024) feature space: (i) standard velocity-prediction flow matchingcollapses, and (ii) autoregressive rollout suffers from compounding ex-posure bias. We address the first with a clean-target (z-prediction) pa-rameterization that yields a substantially higher signal-to-noise ratio,and the second with a two-stage latent flow-forcing curriculum that pro-gressively conditions the model on its own partially denoised rollouts.Experiments on KITTI, Cityscapes, and TartanAir demonstrate thatVGGT-World significantly outperforms the strongest baselines in depthforecasting while running 3.6-5× faster with only 0.43B trainable pa-rameters, establishing frozen GFM features as an effective and efficientpredictive state for 3D world modeling. The code will be open-source athttps://github.com/SimonSun0810/VGGT-World.

Low-light image enhancement is an essential computer visiontask to improve image contrast and to decrease the effects of colorbias and noise. Many existing interpretable deep-learning algorithmsexploit the Retinex theory as the basis of model design. However,previous Retinex-based algorithms, that consider reflected objects asideal Lambertian ignore specular reflection in the modeling process andconstruct the physical constraints in image space, limiting generalizationof the model. To address this issue, we preserve the specular reflectioncoefficient and reformulate the original physical constraints in theimaging process based on the Kubelka-Munk theory, thereby constructingconstraint relationship between illumination, reflection, and detection, theso-called triple physical constraints (TPCs) theory. Based on this theory,the physical constraints are constructed in the feature space of the modelto obtain the TPC network (TPCNet). Comprehensive quantitativeand qualitative benchmark and ablation experiments confirm that theseconstraints effectively improve the performance metrics and visual qualitywithout introducing new parameters, and demonstrate that our TPCNetoutperforms other state-of-the-art methods on 10 datasets. The code isavailable at https://github.com/2020shijingyi/TPCNet