paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

ECCV 2026

Staying VIGILant: Mitigating Visual Laziness in MLLMs via Information-Theoretic Alignment

Xi Xiao, Chen Liu, Chih-Ting Liao, Yunbei Zhang, Qizhen Lan, Yuxiang Wei, Lin Zhao, Janet Wang, Jianyang Gu, Muchao Ye, Tianyang Wang, Hao Xu

Multimodal large language models (MLLMs) extend large language models (LLMs) with visual perception, enabling joint reasoning over images and text. Despite inheriting strong reasoning capabilities from LLMs, they remain prone to hallucinations that contradict their visual inputs. Mechanistic studies indicate that this weakness stems from visual laziness: MLLMs encode the correct visual evidence internally, but overly rely on strong language priors during response. Existing alignment methods, such as direct preference optimization, primarily optimize outcome-level rewards based on text. This introduces an optimization bias toward linguistic shortcuts, leading to responses that often contradict the visual evidence. To address this, we propose Visual Information Gain In aLignment (VIGIL), a reinforcement-learning (RL) post-training framework that shifts the focus from numerical reward fitting to causal visual grounding. VIGIL introduces a geometric constraint that explicitly maximizes the mutual information between the visual input and the generated response. We achieve this by penalizing “blind confidence” instances where the model remains improperly certain even when textual-visual attention is masked to create a counterfactual blind state. Extensive experiments show that VIGIL consistently outperforms recent alignment methods across hallucination and reasoning benchmarks without compromising text-only capabilities. Our approach matches the full-data performance of state-of-the-art methods using only 25% of the preference data and even demonstrates emergent spatial grounding capabilities without explicit bounding box supervision.

ECCV 2026

History-Aware Transformation of ReID Features for Multiple Object Tracking

Ruopeng Gao, Yuyao Wang, Chunxu Liu, Limin Wang

In Multiple Object Tracking (MOT), Re-identification (ReID) features are widely employed as a powerful cue for object association. However, they are often wielded as a one-size-fits-all hammer, applied uniformly across all videos through simple similarity metrics. We argue that this overlooks a fundamental truth: MOT is not a general retrieval problem, but a context-specific task of discriminating targets within a single video. To this end, we advocate for the adjustment of visual features based on the context specific to each video sequence for better adaptation. In this paper, we propose a history-aware feature transformation method that dynamically crafts a more discriminative subspace tailored to each video’s unique sample distribution. Specifically, we treat the historical features of established trajectories as context and employ a tailored Fisher Linear Discriminant (FLD) to project the raw ReID features into a sequence-specific representation space. Extensive experiments demonstrate that our training-free method dramatically enhances the discriminative power of features from diverse ReID backbones, resulting in marked and consistent gains in tracking accuracy. Our findings provide compelling evidence that MOT inherently favors context-specific representation over the direct application of generic ReID features. We hope our work inspires the community to move beyond the naive application of ReID features and towards a deeper exploration of their purposeful customization for MOT.

ECCV 2026

Towards Purified Multi-Label Test-Time Adaptation of Vision-Language Models

Yiwen Liang, Hui Chen, Yizhe Xiong, Mengyao Lyu, Yuhan Cao, Zijia Lin, Shuaicheng Niu, Sicheng Zhao, Jungong Han, Guiguang Ding

Test-time adaptation (TTA) has been widely explored in single-label recognition, effectively mitigating distribution shifts, especially when combined with vision-language models. However, real-world images often contain multiple objects, while the more practical multilabel test-time adaptation (MLTTA) has received little attention so far. Recent cache-based TTA methods have shown promising efficiency and effectiveness, yet directly extending them to multi-label scenarios suffers from a one-to-many mapping problem: a shared global representation entangling co-occurring objects is stored as class-wise cache prototypes, inducing dominant-label bias and compromised cache calibration. While introducing region-level cues helps isolate class-specific evidence, such regional evidence can also be unreliable under distribution shifts, making its identification and utilization non-trivial. To address these issues, we introduce PuRF, a novel PuRiFication-driven cache-based method for multi-label test-time adaptation of vision-language models. Specifically, PuRF first performs region purification to identify reliable regions, providing comprehensive regional cues for multi-label recognition and enabling fine-grained alignment. Based on these purified regions, PuRF conducts cache purification to enhance cache representation and adaptability, where episodic purification builds a discriminative regionbased cache, and temporal refreshing further promotes long-term cache adaptability. Experiments demonstrate that PuRF consistently outperforms state-of-the-art methods, achieving a notable 4.05% mAP improvement on ViT-B/32 across five datasets. The code is available at: https://github.com/Evelyn1ywliang/PuRF-MLTTA.

ECCV 2026

GenSP: Consistent Spherical Parameterization via Learning Shape Generative Models

Sai Karthikey Pentapati, Shashank Gupta, Rajesh Sureddi, Yuezhi Yang, Alan Bovik, Qixing Huang

We introduce GenSP, a data-driven framework that learns consistent spherical parameterizations across a collection of genus-0 shapes. Instead of optimizing the parameterization of each shape independently, our method learns a neural generative model that predicts a continuous mapping from the unit sphere to shapes in a dataset. Under this formulation, spherical parameterizations are obtained through the inverse mappings of the learned generator, which encourages similar shapes to share consistent parameterizations. To make this formulation practical, we address several key challenges in learning such a generative model. First, we introduce a continuous neural deformation model that predicts surface points from sphere coordinates and latent shape codes, avoiding discretization artifacts common in mesh-based formulations. Second, we augment the training space with intermediate shapes that bridge the sphere and input shapes, allowing the model to learn meaningful deformations across a heterogeneous shape collection. Third, we compute reliable initial correspondences by propagating mappings along a spanning tree of training shapes in the latent space. Experiments on the ShapeNet [9] dataset demonstrate that our approach significantly reduces geometric distortion and improves cross-shape consistency compared with state-of-the-art spherical parameterization methods.

ECCV 2026

Hybrid Advantage Estimation with Unified Critic for VLM Agentic Reinforcement Learning

Wenxuan Zhang, Yuhui Wang, Donggang Jia, Xiaoqian Shen, Jian Ding, Ivan Viola, Jurgen Schmidhuber, Mohamed Elhoseiny

Large Vision-Language Models (VLMs) now act as agents in interactive environments, where success requires coherent reasoning and decision-making across turns. Although end-to-end training in agentic environments can improve such multi-turn decision-making abilities, current methods mainly rely on either token-wise optimization over concatenated token trajectories or turn-wise optimization with uniform withinturn credit. In this work, we establish theoretical formulations for the two levels of optimization and derive a hybrid advantage that serves both objectives. Furthermore, with an appropriate choice of discount factor and learning target, we prove that a unified critic model can estimate values for both turn-wise and token-wise. As such, we propose HyGAE, an actor-critic framework that jointly optimizes tokenand turn-level objectives with the hybrid advantage and unified critic. We conduct extensive evaluations of HyGAE across five multi-turn decisionmaking environments, where it achieves an average success rate of 91% and a significant improvement of 10% over other methods. Furthermore, we provide an in-depth analysis showing that the exact analytic form of the hybrid advantage and return is crucial for optimization. Project Page: https://wx-zhang.github.io/hygae-web/.

ECCV 2026

ECC: Encoder-Centric Corruption for Fine-Grained Vision in VLMs

Hyesong Choi, Daeun Kim, Sungmin Cha, Kwang Moo Yi, Dongbo Min

Modern Vision-Language Models (VLMs) often suffer from granularity mismatch, failing to encode high-density visual details not explicitly described in abstract text prompts, which results in low-frequency bias and object hallucination. While additive noise naturally introduces high-frequency variations, suggesting corruption-to-reconstruction (C2R) as a potential remedy, existing noise-based extensions of masked image modeling (MIM) surprisingly yield little improvement on recognition tasks. We identify the cause as a structural limitation: recent trends toward decoder-style designs prevent noise-induced signals from being deeply integrated into the encoder representations. To address this issue, we propose Encoder-Centric Corruption (ECC), a framework guided by three principles: (i) Encoder-Centricity, performing restoration within the encoder to shape transferable features; (ii) FeatureLevel Noising, injecting noise at intermediate layers to capture finegrained textures; and (iii) Task Disentanglement, separating mask reconstruction and denoising via a disruption loss. ECC surpasses prior MIM and noise-based methods by up to 8.1%. Moreover, integrating ECC into large-scale frameworks such as CLIP and AIM-v2 consistently improves performance, demonstrating its effectiveness as a scalable pretraining strategy for next-generation foundation models. Code is available at https://github.com/doihye/ecc.

ECCV 2026

The Sterkfontein Caves Dataset: A Novel View Rendering Challenge from the Cradle of Humankind

Ireton Liu, Brian Xu, Dominic Stratford, Steven James, Richard Klein, James Tompkin

We introduce the challenging Sterkfontein Caves dataset comprising ten underground scenes from a UNESCO World Heritage Site, and use it to find a new simple baseline method that beats existing low-light reconstruction methods upon it. Each scene is of complex surface geometry and high-frequency texture from cave rock structures, including humanmade markings on the rock face. The captured images exhibit varied or uncontrolled lighting over a large dynamic range, with glare artefacts, and low signal-to-noise ratios from the challenging dark real-world capture scenario.Weproposeaviewsynthesisbenchmarkforlow-lightRAWandsRGB reconstruction. All tested NeRF and Gaussian splatting baseline methods struggleonthisdata,withthebestperformingmethodintermsofreliability andaveragePSNRbeingourRaw-Nerfactomethod.Wediscusstheseerrors in detail to find directions of future work for the community in overcoming the significant challenges that remain in low-light high-detail scenes. Our dataset is publicly available at https://visual.cs.brown.edu/sterkfontein.

ECCV 2026

SKEL-CF: Coarse-to-Fine Biomechanical Skeleton and Surface Mesh Recovery

Da Li, Ji-Ping Jin, Xiaodong Cun, Xuanlong Yu, Wei Liu, Rui Fan, Jiangang Kong, Kai Chen, Xi Shen

Parametric 3D human models such as SMPL have driven significant advances in human pose and shape estimation, yet their simplified kinematics limit biomechanical realism. The recently proposed SKEL model addresses this limitation by re-rigging SMPL with an anatomically accurate skeleton. However, estimating SKEL parameters directly remains challenging due to limited training data, perspective ambiguities, and the inherent complexity of human articulation. In this work, we propose SKEL-CF, a new framework for estimating SKEL parameters. SKEL-CF adopts a standard transformer-based encoder–decoder architecture. The encoder first produces coarse predictions of the camera extrinsics and SKEL parameters. The decoder then iteratively refines these predictions across multiple layers, with explicit and auxiliary supervision applied at each layer. To provide anatomically consistent training data, we convert the existing SMPL-based dataset into a SKEL-aligned version, called HMR-SKEL. This new dataset offers high-quality supervision for SKEL estimation. In addition, to reduce depth and scale ambiguity, we explicitly incorporate camera intrinsic estimation into the SKEL-CF pipeline and show that it is important for accurate reconstruction. Extensive experiments validate the effectiveness of the proposed design. On the challenging MOYO dataset, SKEL-CF achieves 85.0 MPJPE / 51.4 PA-MPJPE, significantly outperforming the previous SKEL-based state-of-the-art HSMR (104.5 / 79.6). These results establish SKEL-CF as a promising framework for human motion analysis, facilitating the use of computer vision techniques in biomechanicsrelated analysis. Our implementation is available on the project page: https://pokerman8.github.io/SKEL-CF/.

ECCV 2026

RT-DocLayout: Real-Time End-to-End Document Layout Analysis with Reading Order in the Wild

Cheng Cui, Tingquan Gao, Xueqing Wang, Changda Zhou, Hongen Liu, Ting Sun, Yubo Zhang, Zelun Zhang, Jiaxuan Liu, Manhui Lin, Yue Zhang, Suyin Liang, Yiqing Xiang, Yi Liu

Accurate document layout analysis remains a critical bottleneck for document parsing systems, due to the intricate coupling among heterogeneous document layout elements, geometric distortions (e.g., paper warping and bending, perspective variations), and reading order within diverse layout structures. Existing approaches typically rely on fragmented multi-stage pipelines or computationally heavy generative Transformer architectures, leading to error propagation and limited efficiency. In this paper, we present RT-DocLayout, a highly efficient end-to-end framework for document layout analysis, designed as a front-end for document parsing tasks. The proposed model unifies classification, detection, pixel-level segmentation, and reading order prediction for layout elements within a single 33M-parameter architecture. Built upon the RT-DETR, our key contribution is a unified multi-task formulation within a single query-based decoder that simultaneously classifies, regresses bounding box, generates masks, and constructs relationship to reason reading order. By jointly learning geometric and structural representations, RT-DocLayout introduces multi-task optimization that substantially improves robustness under real-world document distortions. Extensive experiments on OmniDocBench v1.5 and Real5-OmniDocBench demonstrate state-ofthe-art performance in document layout analysis while maintaining realtime inference speed(132.1 FPS). When coupled with downstream OCR engines, RT-DocLayout significantly improves full-document reconstruction quality, providing a scalable and practical foundation for real-world document intelligence systems. The code and models are available at https://github.com/PaddlePaddle/ PaddleOCR.

ECCV 2026

Accelerating Diffusion Models via Equal-Risk Caching

Can Zhang, Liangshun Zou

Diffusion models achieve strong image and video generation quality but remain slow at inference due to multi-step denoising. Caching accelerates inference by reusing intermediate features across timesteps, yet its effectiveness depends critically on refresh scheduling (when to perform full recomputation). Existing schedules fall into three paradigms: fixed-interval, online heuristic, and global optimization. The first two may miss temporal sensitivity structure or accumulated segment-level degradation, while the last incurs substantial offline cost. We propose Equal-Risk Caching (ERC), which builds a one-dimensional temporal risk profile from a single fixed-length probing cache curve and uses cumulative risk mass as a global surrogate for segment sensitivity. ERC places refresh boundaries by equal-mass partitioning on the cumulative risk axis, automatically allocating denser recomputation to high-risk regions and longer reuse to low-risk regions. This reduces the offline profiling cost of schedule construction from O(T ·L) to O(T). Experiments on multiple diffusion models demonstrate an overall competitive quality– speed trade-off.

ECCV 2026

M2Tok: Multi-head Multi-codebook Discrete Action Tokenization for Vision-Language-Action Models

Chunpu Xu, Zhixuan Liang, Yuhao Zhang, Chi-Min Chan, Jiashuo Wang, Yang Xiao, Mengkang Hu, Xiaokang Yang, Yao Mu

Recent advancements have successfully adapted autoregressive language models to process multimodal signals, such as images and actions. Since raw action signals are continuous, effective tokenization is essential to map high-dimensional inputs into compact discrete tokens for autoregressive processing. However, existing discrete action tokenizers often suffer from high reconstruction loss, failing to preserve the finegrained dynamics required for precise control. This “discretization bottleneck” significantly limits the performance ceiling of downstream VisionLanguage-Action (VLA) models. To address this, we propose M2Tok, a Multi-head Multi-codebook Action Tokenizer designed to minimize reconstruction error and enhance policy performance. Our approach introduces two key structural innovations: (1) we decompose the latent action features into multiple heads, enabling the model to implicitly align specific heads with distinct action dimensions; (2) we assign independent codebooks to each head for quantization. By leveraging the combinatorial nature of multiple codebooks, we significantly expand the representational expressivity of the tokenizer, leading to substantially lower reconstruction loss compared to previous methods. We evaluate the M2Tok-based VLA on the RoboTwin, Simpler-Env, and 3 zero-shot real-world tasks. Experimental results demonstrate our method not only achieves superior reconstruction fidelity but also significantly boosts the success rate of VLA models. Comprehensive ablation studies further confirm the effectiveness of the multi-head and multi-codebook mechanisms. Code is available at https://github.com/cpaaax/M2Tok.

ECCV 2026

RECO: Region-Aware Compensation for Extrinsic Perturbations in Roadside 3D Detection

Junsheng Du, Yuhuan Lu, Zhaocheng He

In intelligent transportation systems, roadside 3D object detection provides wide-area perception crucial for traffic understanding, cooperative early warning, and safe autonomous driving. However, existing methods suffer from high sensitivity to camera extrinsics; even slight deviations (whether manifesting as transient jitter or persistent drift) can be significantly amplified by projective geometry. This cascade results in severe feature misalignment and degraded localization. To mitigate this limitation, we propose RECO, a region-aware extrinsic compensation framework that corrects extrinsics using piecewise 6-DoF pose offsets. RECO predicts a learnable range boundary to partition the scene into near and far regions, estimating region-specific pose corrections. A differentiable sigmoid gate then smoothly blends the two compensated geometries to preserve continuous BEV sampling and facilitate stable optimization. To supervise the refinement of extrinsics, we introduce an auxiliary reprojection loss that compares 2D bounding boxes projected from 3D ground truth against 2D annotations, optimizing it jointly with the standard detection objective. Extensive experiments on the DAIR-V2X-I and Rope3D benchmarks under extrinsic perturbations demonstrate consistent improvements over state-of-the-art baselines across both yaw and z-axis deviations. RECO also generalizes from transient perturbations to persistent shifts, maintaining highly competitive performance under strict calibration uncertainty.

ECCV 2026

Fast Sam 3D Body: Accelerating SAM 3D Body for Real-Time Full-Body Human Mesh Recovery

Timing Yang, Sicheng He, Hongyi Jing, Jiawei Yang, Zhijian Liu, Chuhang Zou, Yue Wang

SAM 3D Body (3DB) achieves state-of-the-art accuracy in monocular 3D human mesh recovery, yet its inference latency of several seconds per image precludes real-time application. We present Fast SAM 3D Body, a training-free acceleration framework that reformulates the 3DB inference pathway to achieve interactive rates. By decoupling serial spatial dependencies and applying architecture-aware pruning, we enable parallelized multi-crop feature extraction and streamlined transformer decoding. Moreover, to extract the joint-level kinematics (SMPL) compatible with existing humanoid control and policy learning frameworks, we replace the iterative mesh fitting with a direct feedforward mapping, accelerating this specific conversion by over 10,000×. Overall, our framework delivers up to a 10.25× end-to-end speedup while maintaining largely on-par reconstruction fidelity, even surpassing 3DB on benchmarks such as LSPET. We demonstrate its utility by deploying Fast SAM 3D Body in a vision-only teleoperation system that—unlike methods reliant on wearable IMUs—enables real-time humanoid control and the direct collection of manipulation policies from a single RGB stream.

ECCV 2026

Head Avatars with Dynamic Explicit Hair

Vanessa Sklyarova, Haonan Chen, Berna Kabadayi, Tobias Kirschstein, Zicong Fan, Xi Wang, Gerard Pons-Moll, Matthias Niessner, Marc Pollefeys, Michael Black, Justus Thies

We present DynHair, a novel method for tracking and mod-eling dynamic hair for human head avatars. From video input, we recon-struct a dynamic head avatar with an explicit strand-based hair represen-tation using structured 3D Gaussian Splatting. In contrast to the faceregion of human head avatars, which can be modeled with 3D Gaus-sians that are attached or generated with respect to some expressive3D head model, hair is particularly challenging as it exhibits dynamicmotion effects. Therefore, we present a novel method that models thedynamic deformations of the hair strands using a temporal network thatis conditioned on angular velocity and acceleration of the head, as wellas relative gravity. Specifically, an LSTM encodes the motion historyand modulates per-point strand features via FiLM conditioning whichis then used by an MLP to produce physically plausible displacementsto the canonical hairstyle. We jointly optimize this motion and appear-ance representation of the hair, with a 3DGS-based representation ofthe face-region, via differentiable Gaussian splatting with photometric,geometric, and physics-based supervision. As a result, we obtain hair⋆ Equal contribution. † Equal supervision.tracking of the training video data and an animatable head avatar withcontrollable hair dynamics. In our experiments, we demonstrate state-of-the-art performance in terms of hair dynamics, temporal consistency,and generalization across subjects. Our data and code are available athttps://dynhair.is.tue.mpg.de/.

ECCV 2026

SNOW: Spatio-Temporal Scene Understanding with World Knowledge for Open-World Embodied Reasoning

Tin Stribor Sohn, Maximilian Dillitzer, Jason Corso, Johannes Bach, Eric Sax

Autonomous robotic systems require spatio-temporal understanding of dynamic environments to ensure reliable navigation and interaction. While Vision-Language Models (VLMs) provide open-world semantic priors, they lack grounding in 3D geometry and temporal dynamics. Conversely, geometric perception captures structure and motion but remains semantically sparse. We propose SNOW (Scene Understanding with Open-World Knowledge), a training-free and backboneagnostic framework for unified 4D scene understanding that integrates VLM-derived semantics with point cloud geometry and temporal consistency. SNOW processes synchronized RGB images and 3D point clouds, using HDBSCAN clustering to generate object-level proposals that guide SAM2-based segmentation. Each segmented region is encoded through our proposed Spatio-Temporal Tokenized Patch Encoding (STEP), producing multimodal tokens that capture localized semantic, geometric, and temporal attributes. These tokens are incrementally integrated into a 4D Scene Graph (4DSG), which serves as 4D prior for downstream reasoning. A lightweight SLAM backend anchors all STEP tokens spatially, providing the global reference alignment, and ensuring unambiguous spatial grounding across time. The resulting 4DSG forms a queryable, unified world model through which VLMs can directly interpret spatial scene structure and temporal dynamics. Experiments on a diverse set of benchmarks demonstrate that SNOW enables precise 4D scene understanding and spatially grounded inference, setting new stateof-the-art performance in several settings, highlighting the importance of structured 4D priors for embodied reasoning and autonomous robotics.

ECCV 2026

EGM: Efficient Visual Grounding Language Models

Guanqi Zhan, Changye Li, Zhijian Liu, Yao Lu, Yi Wu, Song Han, Ligeng Zhu

Visual grounding is an essential capability of Visual Language Models (VLMs) to understand the real physical world. Previous state-of-the-art grounding visual language models usually have large model sizes, making them heavy for deployment and slow for inference. However, we notice that the sizes of visual encoders are nearly the same for small and large VLMs and the major difference is the sizes of the language models. Small VLMs fall behind larger VLMs in grounding because of the difference in language understanding capability rather than visual information handling. To mitigate the gap, we introduce ‘Efficient visual Grounding language Models’ (EGM): generate many mid-quality tokens (from small models) to match the performance of large VLMs with few high-quality but expensive tokens. This method is deploymentfriendly, and yields better end-to-end latency: On the RefCOCO benchmark, our EGM-Qwen3-VL-8B demonstrates 91.4 IoU with an average of 737ms (5.9× faster) latency while Qwen3-VL-235B demands 4,320ms to reach 90.5 IoU. To validate our approach’s generality, we further set up a new amodal grounding setting that requires the model to predict both the visible and occluded parts of the objects. Experiments show our method consistently improves both vanilla and amodal grounding capabilities of small models to match or outperform larger models, thereby improving efficiency for visual grounding.

ECCV 2026

Analyzing and Improving Training-Free Fast Sampling of Text-to-Image Diffusion Models

Zhenyu Zhou, Defang Chen, Siwei Lyu, Chun Chen, Can Wang

Text-to-image diffusion models have achieved unprecedented success but still struggle to produce high-quality results under limited sampling budgets. Existing training-free sampling acceleration methods are typically developed independently, leaving the overall performance and compatibility among these methods unexplored. In this paper, we bridge this gap by systematically elucidating the design space, and our comprehensive experiments identify the sampling time schedule as the most pivotal factor. Inspired by the geometric properties of diffusion models revealed through the Frenet-Serret formulas, we propose constant total rotation schedule (TORS), a scheduling strategy that ensures uniform geometric variation along the sampling trajectory. TORS outperforms previous training-free acceleration methods and produces highquality images with 10 sampling steps on Flux.1-Dev and Stable Diffusion 3.5. Extensive experiments underscore the adaptability of our method to unseen models, hyperparameters, and downstream applications. Code is available at https://github.com/zju-pi/TORS.

ECCV 2026

Mapping Dark-Matter Clusters via Physics-Guided Diffusion Models

Diego Royo, Brandon Zhao, Adolfo Muñoz, Diego Gutierrez, Katherine Bouman

Galaxy clusters are powerful probes of astrophysics and cosmology through gravitational lensing: their mass, dominated by 85% dark matter, distorts background light. Yet, mass reconstruction lacks the scalability and large-scale benchmarks to process the hundreds of thousands of clusters expected from forthcoming wide-field surveys. We introduce a fully automated method to reconstruct clustersurfacemassdensityfromphotometryandgravitationallensingobservables. Central to our approach is DARKCLUSTERS-15K, our new dataset of 15,000 mock cluster observations with paired mass and photometry maps, the largest to date, spanning multiple redshifts and simulation frameworks. We train a plug-and-play diffusion prior on DARKCLUSTERS-15K that learns the statistical relationship between mass and light, and draw approximate posterior samples constrained by weakand strong-lensing observables, yielding principled reconstructions with well-calibrated empirical uncertainties. Our approach requires no expert tuning, runs in minutes rather than hours, achieves higher accuracy, and matches experttuned reconstructions of the MACS 1206 cluster. We release our method and DARKCLUSTERS-15K to support upcoming wide-field cosmological surveys.

ECCV 2026

Divide and Conquer: Decoupled Representation Alignment for Multimodal World Models

Junyuan Xiao, Dingkang Liang, Xin Zhou, Yixuan Ye, Tongtong Su, Guangmo Yi, Bin Xia, Qiang Lyu, Shurui Shi, Jun Huang, Jianlou Si, Wenming Yang

Emerging multi-modal world models attempt to jointly generate videos across diverse modalities (e.g., RGB, depth, and mask), yet they fail to fully exploit the priors of existing foundation models. We propose M2-REPA, the first representation alignment method tailored for multi-modal video generation. Our key insight is that foundation models trained on different modality spaces naturally capture distinct domain-specific priors, acting as complementary “experts.” Specifically, we first decouple modality-specific features from the diffusion model’s intermediate representations, then align each with its corresponding expert foundation model. To this end, we design two synergistic objectives: a multi-modal representation alignment loss that enforces feature-toexpert matching, and a modality-specific decoupling regularization that encourages complementarity across different modalities. This design enables joint optimization, exploiting priors from multiple foundation models. Extensive experiments demonstrate that our method outperforms baselines in visual quality and long-term consistency.

ECCV 2026

VERTIGO: Visual Preference Optimization for Cinematic Camera Generation

Mengtian Li, Yuwei Lu, Feifei Li, Chenqi Gan, Zhifeng Xie, Xi Wang

Cinematic camera control relies on a tight feedback loop between director and cinematographer, where camera motion and framing are continuously reviewed and refined. Recent generative camera systems can produce diverse, text-conditioned trajectories, but they lack this “director in the loop” and have no explicit supervision of whether a shot is visually desirable. This results in in-distribution camera motion but poor framing, off-screen characters, and undesirable visual aesthetics. In this paper, we introduce VERTIGO, the first framework for visual preference optimization of camera trajectory generators. Our framework leverages a real-time graphics engine (Unity) to render 2D visual previews from generated camera motion. A cinematically fine-tuned vision–language model then scores these previews using our proposed caption-based consistency similarity mechanism, which aligns renders with text prompts. This process provides the visual preference signals for Direct Preference Optimization (DPO) post-training. Both quantitative evaluations and user studies on Unity renders and diffusion-based Camera-to-Video pipelines show consistent gains in condition adherence, framing quality, and perceptual realism. Notably, VERTIGO reduces the character off-screen rate from 38% to nearly 0% while preserving the geometric fidelity of camera motion. User study participants further prefer VERTIGO over baselines across composition, consistency, prompt adherence, and aesthetic quality, confirming the perceptual benefits of our visual preference post-training.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.