ECCV 2026 论文列表

Although Multimodal Large Language Models (MLLMs) havemade impressive progress in understanding videos—such as recognizingobjects, actions, and events—they still struggle with spatial reasoning.In particular, they have difficulty forming a consistent 3D understandingof a scene from separate 2D video frames. Unlike humans, who can natu-rally infer depth, distance, object size, and their own movement throughspace, these models often lack the ability to accurately reconstruct a co-herent 3D environment. This limitation makes it challenging for them toestimate real-world measurements, such as how far an object is or howmuch the camera has moved, when relying only on 2D image sequences.To bridge these gaps, we propose VoxAnchor, a framework for explicitspatial-semantic grounding. Specifically, we reify the 2D video sequenceinto a spatiotemporally continuous representation by unprojecting visualtokens, geometric features, and view-consistent semantics into a unified3D voxel grid. To facilitate precise spatial reasoning, we propose a Voxel-Aware Attention mechanism, which constrains feature interactions withinphysically consistent volumetric units. This approach enables the modelto effectively aggregate fragmented observations and infer metric-awarespatial relationships from monocular video. VoxAnchor sets a new state-of-the-art on VSI (66.8%) and VSTI (66.3%) benchmarks, outperform-ing significantly more parameter-intensive models and confirming thatexplicit geometric grounding effectively bridges the 2D-to-3D gap. Codewill be available at https://github.com/Embrace-Arch/VoxAnchor.

Recovering dense 3D geometry from unposed images remainsa foundational challenge in computer vision. Current state-of-the-artmodels are predominantly trained on perspective datasets, which implic-itly constrains them to a standard pinhole camera geometry. As a result,these models suffer from significant geometric degradation when appliedto wide-angle imagery captured via non-rectilinear optics, such as fisheyeor panoramic sensors. To address this, we present CAM3R, a Camera-Agnostic, feed-forward Model for 3D Reconstruction capable of process-ing images from wide-angle camera models without prior calibration.Our framework consists of a two-view network which is bifurcated into aRay Module (RM) to estimate per-pixel ray directions and a Cross-viewModule (CVM) to infer radial distance with confidence maps, pointmaps,and relative poses. To unify these pairwise predictions into a consistent3D scene, we introduce a Ray-Aware Global Alignment framework forpose refinement and scale optimization while strictly preserving the pre-dicted local geometry. Extensive experiments on various camera modeldatasets, including panorama, fisheye and pinhole imagery, demonstratethat CAM3R establishes a new state-of-the-art in pose estimation andreconstruction.

Deep Neural Network (DNN) classifiers suffer from poor calibration when their softmax outputs (predictive confidence) deviate from the empirical likelihoods. This manifests itself as either overconfident incorrect predictions or under-confident correct predictions. Label smoothing (LS) enhances model calibration by introducing entropy regularization during training through redistributing probability mass from the ground-truth label to the remaining classes. LS, including Margin-based LS (MbLS), have restrictive assumptions: they rely on predefined, uniform smoothing rules and only tackle overconfidence. In reality, samples exhibit diverse characteristics, such as difficulty/ambiguity, that interact with the evolving nature of the model being trained. In training, samples may have various degrees of underor overconfidence. To overcome this, a mechanism that identifies the specific confidence state of each sample and determines the appropriate degree of smoothing in each training step is needed, tailoring the adjustment to the individual sample. We propose FeDLaS: Feature-Modulated Bidirectional Label Smoothing, a plug-and-play algorithm for label smoothing-based losses. In FeDLaS, we introduce a Feature Norm-based Confidence Indicator (NCI) to control smoothing and a Bidirectional Calibration Gating (BCG) module to detect both over and under-confidence. Our algorithm can be integrated with LS and MbLS based losses when applied to standard DNNs, enhancing performance. Extensive experiments on standard and fine-grained high-resolution vision benchmarks show that FeDLaS consistently improves calibration compared to modern baselines, reducing Expected Calibration Error (ECE) and Adaptive ECE while maintaining Top-1 accuracy. Code: github.com/nadarasarbahavan/FEDLAS

In the pursuit of robust and generalizable category-level ob-ject pose estimation, most existing methods adopt parametric formula-tions that learn effective representations from data, yet they primarilyencode category-level patterns into fixed shape priors or static param-eter weights, which limits their scalability to highly diverse instances.In this paper, we rethink category-level pose estimation from a memory-centric perspective and present MemPose, a memory-augmented frame-work that explicitly incorporates category-level geometric memory intothe pose estimation pipeline. We introduce an external memory bufferthat stores and dynamically updates structural representations frompreviously observed instances, enabling the model to leverage accumu-lated experience to support current perception. Extensive experimentson four challenging benchmarks (REAL275, CAMERA25, Housecat6Dand Wild6D) demonstrate the superiority of our proposed method overprevious state-of-the-art approaches.

Attention-based models often develop attention sinks, wherea small number of tokens repeatedly attract attention and accumulateunusually large activations. In vision transformers, these outliers areclosely related to registers, which have been diagnostically linked toglobal, low-frequency image structure. Existing work has largely studiedregisters through interpretability analyses and linear probes, leaving openwhether they can be operationalized as plug-and-play signals for genera-tion without retraining. We revisit this question in tokenized image gen-eration. Using OpenCLIP and DINOv2 on ImageNet, we find that test-time register features exhibit stronger low-frequency concentration thanboth [CLS] readouts and patch-mean features, and show a consistent (al-beit moderate) correlation with pixel-space DCT low-frequency energy.Motivated by these diagnostics, we introduce RegToken, a training-free procedure that converts register structure into a small set of globalprior tokens by (i) NFN-based layer localization, (ii) TokenRank-guidedsubspace extraction, and (iii) a projection-and-conservation update onthe register subspace. Inserted into a frozen compact 1D token gener-ation pipeline, RegToken improves ImageNet generation and alignmentmetrics (e.g., FID-5k 20.5 → 20.1, SigLIP 3.6 → 3.9) without modify-ing pretrained weights, and accelerates test-time optimization (Steps@τ74 → 52). Overall, our results suggest that structures often viewed asattention artifacts can be repurposed as lightweight global priors for to-kenized generation.

Advances in radiance fields have enabled photorealistic novelview synthesis. In several domains, large-scale real-world datasets havebeen developed to support comprehensive benchmarking and to facilitateprogress beyond scene-specific reconstruction. However, for distractor-free radiance fields, a large-scale dataset with clean and cluttered imagesper scene remains lacking, limiting the development. To address thisgap, we introduce DF3DV-1K, a large-scale real-world dataset compris-ing 1,048 scenes, each providing clean and cluttered image sets for bench-marking. In total, the dataset contains 89,924 images captured using con-sumer cameras to mimic casual capture, spanning 17 scenario types, 128distractor types, and 161 scene themes across indoor and outdoor envi-ronments. A curated subset of 41 scenes, DF3DV-41, is systematicallydesigned to evaluate the robustness of distractor-free radiance field meth-ods under challenging scenarios. Using DF3DV-1K, we benchmark ninerecent distractor-free radiance field methods and 3D Gaussian Splatting,identifying the most robust methods and the most challenging scenarios.Beyond benchmarking, we demonstrate an application of DF3DV-1K byfine-tuning a diffusion-based 2D enhancer to improve radiance field meth-ods, achieving average improvements of 0.96 dB PSNR and 0.057 LPIPSon the held-out set (e.g., DF3DV-41) and the On-the-go dataset. We hopeDF3DV-1K facilitates the development of distractor-free vision and pro-motes progress beyond scene-specific approaches. The dataset and leader-board are available at https://johnnylu305.github.io/df3dv1k_web/.

Modern text-to-image models excel in visual fidelity andprompt adherence. However, this strict adherence comes at the cost ofdiversity: generated samples tend to collapse into a single visual inter-pretation. Existing methods to improve diversity produce outputs drivenby incidental variations rather than meaningful design choices. This mo-tivates a new variant of the diversity task where structure is enforced onthe generated samples.We introduce a method for controlled diversity that enables SemanticBrowsing, where users can navigate structured image galleries and expe-rience creative exploration through a systematic traversal of meaningful,interpretable axes of variation. Achieving this level of semantic controlrequires a deep understanding of the scene. We exploit the fact thatrecent text-to-image models are trained on elaborated captions, effec-tively decoupling semantic decision-making from pixel generation. Thisenables a paradigm shift: instead of relying on stochastic variation withinthe text-to-image model, we induce diversity directly at the text level.By leveraging rich textual representations, we allow a Vision LanguageModel (VLM) to operate on the full scene context. To overcome thegeneric outputs typical of standard VLMs, we employ an agentic work-flow that explicitly enforces structured variation attuned to the originalprompt. We demonstrate that our method produces diverse and naviga-ble design spaces where every variation corresponds to a specific, user-understandable semantic decision.

We present ’RenderFormer-V2’, a unified learned transformer-based neural rendering model, complementary to modern physics-basedrendering systems, that can handle diverse light-transport effects suchas caustics, volumetric scattering, environment lighting, textured anddisplaced surfaces and out-of-distribution materials without per-scenetraining or specialized code. RenderFormer-V2 models global light trans-port as a sequence-to-sequence transformation. Following its predecessor,RenderFormer-V2 also employs a two stage process: a view-independentstage that resolves intra-scene primitive to primitive transport, and aview-dependent stage that transforms the internal neural scene repre-sentation into image pixels. Different from RenderFormer, our modelemploys a novel combined windowed-attention and rendering-informedattention sink in the view-independent stage to improve scalability whilemaintaining render accuracy. To further improve versatility, RenderFormer-V2 supports heterogeneous scene primitives, including environment mapsand participating media, and it employs a material encoding indepen-dent of the underlying surface reflectance model that encodes materialappearance via a novel neural embedding. We demonstrate the versatil-ity of RenderFormer-V2 on a variety of scenes and perform an extensiveablation of the improved attention mechanism.

The reconstruction of dynamic 3D scenes using 3D GaussianSplatting has shown significant promise. A key challenge, however, re-mains in modeling realistic motion, as most methods fail to align the mo-tion of Gaussians with real-world physical dynamics. This misalignmentis particularly problematic for monocular video datasets, where failingto maintain coherent motion undermines local geometric structure, ulti-mately leading to degraded reconstruction quality. Consequently, manystate-of-the-art approaches rely heavily on external priors, such as opti-cal flow or 2D tracks, to enforce temporal coherence. In this work, wepropose a novel method to explicitly preserve the local geometric struc-ture of Gaussians across time in 4D scenes. Our core idea is to introducea view-space ray grouping strategy that clusters Gaussians intersectedby the same ray, considering only those whose α-blending weights ex-ceed a threshold. We then apply constraints to these groups to maintaina consistent spatial distribution, effectively preserving their local geom-etry. This approach enforces a more locally coherent motion model byensuring that local geometry remains stable over time, eliminating thereliance on external guidance. We demonstrate the efficacy of our methodby integrating it into two distinct baseline models. Extensive experimentson challenging monocular datasets show that our approach significantlyoutperforms existing methods, achieving superior temporal consistencyand reconstruction quality.

Predicting human gaze in video is fundamental to advanc-ing scene understanding and multimodal interaction. While traditionalsaliency maps provide spatial probability distributions and scanpaths of-fer ordered fixations, both abstractions often collapse the fine-grainedtemporal dynamics of raw gaze. Furthermore, existing models are typi-cally constrained to short-term windows (≈ 3–5s), failing to capture thelong-range behavioral dependencies inherent in real-world content. Wepresent a generative framework for infinite-horizon raw gaze predictionin videos of arbitrary length. By leveraging an autoregressive diffusionmodel, we synthesize gaze trajectories characterized by continuous spa-tial coordinates and high-resolution timestamps. Our model is condi-tioned on a saliency-aware visual latent space. Quantitative and qualita-tive evaluations demonstrate that our approach significantly outperformsexisting approaches in long-range spatio-temporal accuracy and trajec-tory realism. Project website: https://www.immersivecomputinglab.org/publication/infinite-gaze/.

Multimodal Large Language Models (MLLMs) inherit richrelational priors from their language backbones, yet often fail when askedto apply these relationships in visual contexts. We trace this failure to astructural blind spot: projection-based alignment trains each visual to-ken to carry the right semantics, but never asks whether the relationshipsbetween concepts survive the crossing from language to vision. To ad-dress this, we propose MIRROR (Mapping Inter-concept Relationsfrom language to visual Representation via Optimal-transport-basedRegularization), a geometric regularization framework that transfers re-lational priors from language to vision by exploiting the rich relationalstructure encoded in language representations. Specifically, we derive asurrogate loss from the proposed Semi-Inverse Gromov–Wasserstein(SI-GW) problem, an inverse geometric problem that aligns visual rep-resentations with language-derived relational priors. We show that thisformulation admits a unique closed-form solution that prescribes theideal visual relational structure implied by language geometry and cross-modal coupling. The structure of the formulation also enables efficientcomputation, making it applicable to long token sequences. ApplyingSI-GW inside decoder-only Transformers requires careful design. We in-troduce targeted strategies at the layer, head, and token levels to ensurestable extraction without additional parameters or inference cost. MIR-ROR improves relational consistency while preserving performance ongeneral vision-language tasks.

This paper presents CountEx, a discriminative visual count-ing framework designed to address a key limitation of existing prompt-based methods: the inability to explicitly exclude visually similar distrac-tors. While current approaches allow users to specify what to count viainclusion prompts, they often struggle in cluttered scenes with confus-able object categories, leading to ambiguity and overcounting. CountExenables users to express both inclusion and exclusion intent, specifyingwhat to count and what to ignore, through multimodal prompts includ-ing natural language descriptions and optional visual exemplars. At thecore of CountEx is a novel Discriminative Query Refinement module,which jointly reasons over inclusion and exclusion cues by first identi-fying shared visual features, then isolating exclusion-specific patterns,and finally applying selective suppression to refine the counting query.To support systematic evaluation of fine-grained counting methods, weintroduce CoCount, a benchmark comprising 1,780 videos and 10,086 an-notated frames across 97 category pairs. Experiments show that Coun-tEx achieves substantial improvements over state-of-the-art methods forcounting objects from both known and novel categories. The data, code,and model are available at https://github.com/bbvisual/CountEx.

Video generation is central to AI-powered content creation.The alignment with human preferences is one of the key metrics for mea-suring the quality of the generated videos. Despite significant progress invisual quality, three key challenges remain: 1) The reliability of rewardsignals is constrained by the quality of human preference data, which isoften corrupted by subjective noise and bias. 2) Standard scalar rewardmodels collapse multi-aspect human preference into a single value, lead-ing to the loss of dynamic trade-offs across multiple dimensions of humanpreference. 3) In policy optimization, the widely adopted KL-divergenceimposes only local constraints, failing to capture holistic human pref-erence. To address these challenges, we propose a unified, preference-aware learning framework for video generation. First, we propose elite-guided filtering to calibrate preference data and construct reliable su-pervision for reward-model training. We then model video quality asa multidimensional reward distribution to capture the uncertain na-ture of human preference, and use the Wasserstein distance to alignit with the empirical human preference distribution. Finally, we intro-duce Wasserstein-based distributional alignment in GRPO, guiding thepolicy’s video generation to match the global structure of human videopreference. Experiments on reward modeling and video generation showthat our approach improves the reliability of reward signals and theperceptual consistency of generated videos. Our code is available athttps://github.com/alignhs26/ahs.

Generating temporally continuous and socially coherent hu-man motion from text remains a fundamental challenge, particularly inrealistic streams where people act alone, enter interactions, and later dis-engage. Most existing methods generate fixed-length motion clips understatic agent configurations, which makes them brittle to solo–social tran-sitions and unsuitable for incremental generation over long horizons. Wepropose ARMS, an Anchor–Relational Motion Streaming framework thatunifies solo motion and human–human interaction within a single causalgenerative process. ARMS introduces a dynamics-asymmetric represen-tation that decouples per-person temporal evolution from inter-personalignment via a partner-referenced relative-translation term, enablingseamless switching of social coupling without sacrificing long-horizonstability or spatial consistency between agents. On top of a causal la-tent space, a causal relational diffusion model progressively refines mo-tion segment by segment using only past context, capturing both intra-person temporal dependencies and inter-person relations. Mode-awarerelational gating activates or masks cross-agent connections, allowingthe same model to support both solo and interaction generation. Ex-periments show that ARMS improves transition smoothness and socialcoherence compared to interaction-centric baselines, while also achievingcompetitive results on human–human interaction benchmarks.

Human-human collaboration is a fundamental aspect of ev-eryday life, essential to success in a wide range of goal-directed activitiesfrom household tasks to professional teamwork. While much researchhas focused on modeling coordination and task execution, the cognitiveprocesses that support such collaboration, particularly Theory of Mind(the ability to infer others’ mental states), remain difficult to study innatural settings. To address this gap, we introduce a novel egocentric andexocentric video dataset capturing real-world collaboration in cookingscenarios. The dataset integrates multi-perspective video, high-qualityaudio, gaze tracking, and 3D scene and object scans, with annotations forshared gaze attention, social cues and interactions between agents, as wellas agent-object interactions. We establish benchmarks for Joint AttentionEstimation, Action Anticipation, and Collaborative Handover Prediction,enabling research on multimodal perception, proactive assistance, andcollaborative planning. By providing temporally aligned, richly annotatedmultimodal data, CoMind facilitates the development and evaluation ofAI systems capable of modeling complex social interactions and reasoningabout human behaviors in collaborative environments. Our dataset andbenchmarks are made available at https://comind.ethz.ch/.

Vision encoders are a critical component of vision-languagemodels, and scaling their capacity effectively improves performance. How-ever, dense scaling increases compute cost and inference latency. Mixture-of-Experts (MoE) architectures offer a compelling alternative, having en-abled efficient scaling in LLMs, yet the MoE design space for CLIP-stylevision encoders remains underexplored at State-of-the-Art (SOTA) lev-els. In this work, we systematically study MoE designs for vision encoderscaling and find that fine-grained MoE topologies yield substantial gainsover both dense and standard MoE counterparts. We further proposean auxiliary-loss-free balancing variant for better expert utilization, anddesign a specialized MoE kernel to mitigate inference latency overhead.To enhance video capabilities while preserving image knowledge, we in-troduce frame-level distillation paired with a novel freezing mechanism.We pretrain a series of Mixture-of-Experts Vision Encoders (MoE-ViE)across a range of sizes, all consistently outperforming their dense counter-parts. Our largest model matches the zero-shot performance of a SOTAencoder 1.7× its size at 76% of its latency. When aligned with an LLM,MoE-ViE surpasses all compared encoders on image and video bench-marks, including those with up to 5× more activated parameters.

Generating motion-controlled videos—where user-specified actions drive physically plausible scene dynamics under freely chosen viewpoints—demands two capabilities: (1) disentangled motion control, allowing users to separately control the object motion and adjust camera viewpoint; and (2) motion causality, i.e., generative action–consequence dependency, ensuring that user-driven actions trigger coherent reactions from other objects rather than merely displacing pixels. Existing methods fall short on both fronts: they entangle camera and object motion into a single tracking signal and treat motion as kinematic displacement without modeling causal relationships between object motion. We introduce MoRight, a unified framework that addresses both limitations through disentangled motion modeling. Object motion is specified in a canonical static-view and transferred to an arbitrary target camera viewpoint via temporal cross-view attention, enabling disentangled camera and object control. We further decompose motion into active (user-driven) and passive (consequence) components, training the model to learn motion causality from data. At inference, users can either supply active motion and MoRight predicts consequences (forward reasoning), or specify desired passive outcomes and MoRight recovers plausible driving actions (inverse reasoning), all while freely adjusting the camera viewpoint. Experiments on three benchmarks demonstrate state-of-the-art performance in generation quality, motion controllability, and interaction awareness.

Building virtual cells with generative models to simulate cel-lular behavior in silico is emerging as a promising paradigm for acceler-ating drug discovery. However, prior image-based generative approachescan produce implausible cell images that violate basic physical and bio-logical constraints. To address this, we propose to post-train virtual cellmodels with reinforcement learning (RL), leveraging biologically mean-ingful evaluators as reward functions. We design seven rewards spanningthree categories—biological function, structural validity, and morpho-logical correctness—and optimize the state-of-the-art CellFlux modelto yield CellFluxRL. CellFluxRL consistently improves over CellFluxacross all rewards, with further performance boosts from test-time scal-ing. Overall, our results present a virtual cell modeling framework thatenforces physically-based constraints through RL, advancing beyond “vi-sually realistic” generations towards “biologically meaningful” ones.

Video understanding is often plagued by severe temporal re-dundancy, where processing dense frame sequences is both semanticallyinefficient and computationally expensive. This challenge is further am-plified when only a small subset of frames is truly relevant to the givenquery. In this paper, we propose a Query- and Content-Aware (QCA)keyframe selection framework that can select a compact yet information-rich set of frames from long videos. QCA first partitions the video intotemporal segments and estimates the information contribution of eachsegment by jointly modeling query relevance and content deviation, anddynamically allocates keyframe budget to each segment. Within eachsegment, QCA anchors on the most query-relevant frame and iterativelyincorporates additional frames to maximize diversity while maintaininghigh semantic relevance to the query. Crucially, our method requires noadditional training and can be seamlessly integrated into existing Video-LLMs. Extensive experiments across multiple long video understandingbenchmarks demonstrate that our proposed approach achieves state-of-the-art performance and has strong generalization ability. For instance,QCA achieves 67.8% on LongVideoBench using 128 frames, while GPT-4o achieves 66.7% using 256 frames. Our codes are available in GitHub.