paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

ECCV 2026

LUNA: Learning Universal 3D Human Animation Beyond Skinning

Peng Li, Rawal Khirodkar, Junxuan Li, Yuan Dong, Chen Cao, Yuan Liu, Wenhan Luo, Yike Guo, Shunsuke Saito

Creating photorealistic, animatable 3D human avatars from monocular images still largely depends on Linear Blend Skinning (LBS) and parametric body models which constrain expressivity and often introduce artifacts due to imperfect fitting. We propose LUNA, an LBSfree universal neural animation model that directly maps multiple 2D controls like images, keypoints, sketch and unseen characters into 3D Gaussian deformations, bypassing explicit body fitting. At its core, a transformer-based motion regressor disentangles global rigid motion from fine-grained local dynamics to capture both coherent movement and subtle non-rigid effects. To resolve the inherent ambiguity of 2D-to-3D lifting while scaling beyond fitted datasets, we introduce hybrid supervision that distills soft structural priors from an LBS teacher and a loss that supports training on both limited fitted data and large in-the-wild unlabeled videos. Extensive experiments show LUNA achieves competitive visual fidelity compared to LBS-based approaches, while delivering realistic human motion and zero-shot cross-identity generalization across diverse driving modalities. To the best of our knowledge, LUNA is the first end-to-end 3D animatable model that supports implicit 2D driving.

ECCV 2026

ROSE: Real-Time Open-World Scene Understanding from Monocular Video via Compact Multimodal 4D Scene Graphs

Ziyue Qiu, Yong Wang, Jin Pan

Vision-Language Models (VLMs) have advanced rapidly invisual and semantic understanding, yet they remain weak at dynamic 4Dspatial grounding in video. On spatio-temporal reasoning benchmarks,raw-video input often yields low and variable accuracy. 4D Scene Graphs(4DSGs) can supply missing structure, but two interface gaps remain:identity facets (appearance, geometry, motion) are often encoded throughlanguage-side descriptors rather than modality-separated inputs, andmost training targets static spatial relations instead of dynamic 4D at-tributes such as trajectories and velocity. We introduce Visual Spatio-Temporal Anchor (VISTA), a multimodal node encoding that representseach tracked object as a persistent, queryable entity, delegating appear-ance to anchor crops processed by the VLM’s vision encoder and geome-try and motion to explicitly structured tokens. We introduce Real-TimeOpen-World Scene Understanding (ROSE), a training-free monocularpipeline (single GPU) that runs at 10 Hz. Beyond inference-time prompt-ing, the 4DSGs produced by ROSE can serve as structured training-timecontext: fine-tuning a VLM on 4DSG-augmented data teaches it to usetrajectory and velocity tokens for dynamic reasoning, yielding a new stateof the art on DSR-Bench (67.5, +8.6 over the previous best), with gainsacross all 13 subtasks. ROSE achieves competitive accuracy across allthree benchmarks under real-time, single-GPU monocular constraints.

ECCV 2026

Anchored, Not Graded: How Vision-Language Models Fail at Slant-from-Texture Perception

Qian Zhang, Michal Golovanevsky, Fulvio Domini, James Tompkin

Human perception of surface slant from texture exhibits systematic, graded biases that emerge reliably in psychophysical experiments. Prior work showed that unsupervised CNNs reproduce several human-like biases,whilesupervisedCNNsdonot.DoVision-LanguageModels(VLMs) exhibit similar competences? Across multiple VLM families and model scales,zero-shotandin-contextpromptingbothproducedistinctivefailures: slantispredictedatonlyasmallsetofanchors(e.g.,0°,±25°,±45°)withlittle dependence on stimulus field of view, optical slant, or surface curvature. Supervised fine-tuning partially remediates the failure, but residual anchoring persists. While success in high-level vision-language benchmarks might notrequiresensitivitytolow-levelgeometriccues,weinterpretanchoringas a failure at the representation-to-output language interface: not necessarily anabsenceofgeometricencoding,butafailuretoexpressitinagradedform.

ECCV 2026

ZeroSplat: Generalized Referring Segmentation in 3D Gaussian Splatting

Jiayu Ding, Meilu Song, Xiaoyi Zhang, Hongbo Jin, Yichen Jin, Xiangtian Si

Recent advancements in 3D Gaussian Splatting (3DGS) have enabled language-guided scene understanding. However, existing Referring 3D Gaussian Splatting (R3DGS) methods are fundamentally restricted to single-target queries. To reflect the ambiguity of real-world instructions, we introduce the Generalized Referring 3D Gaussian Splatting Segmentation (GR3DGS) task, which requires dynamically segmenting an arbitrary number of targets (0, 1, or N). To facilitate comprehensive evaluation of this new task, we construct two new benchmarks: GR-LERF and GR-ScanNet. Crucially, existing R3DGS paradigms exhibit fundamental technical bottlenecks that severely limit their performance on the GR3DGS task: they lack intrinsic 3D point-level understanding by operating merely on 2D rendered pixels, and they incur prohibitive computational overhead by requiring per-scene optimization to embed heavy semantic features. To dismantle these bottlenecks, we propose ZeroSplat, a novel training-free and zero-feature framework. ZeroSplat lifts 2D Vision-Language Model (VLM) priors into 3D space through robust multi-view geometric constraints. This strategy enables intrinsic point-level understanding without incurring any additional feature storage. Extensive experiments demonstrate that ZeroSplat significantly outperforms state-of-the-art methods across generalized and single-target scenarios while maintaining exceptional efficiency. Project Page: https://inkmind-ai.github.io/ZeroSplat

ECCV 2026

The Dynamic Prior: Understanding 3D Structures for Casual Dynamic Videos

Zhuoyuan Wu, Xurui Yang, Jiahui Huang, Yue Wang, Jun Gao

Estimating accurate camera poses, 3D scene geometry, and object mo-tion from in-the-wild videos is a long-standing challenge for classical structurefrom motion pipelines due to the presence of dynamic objects. Recent learning-based methods attempt to overcome this challenge by training motion estimatorsto filter dynamic objects and focus on the static background. However, their per-formance is largely limited by the availability of large-scale motion segmentationdatasets, resulting in inaccurate segmentation and, therefore, inferior structural 3Dunderstanding. In this work, we introduce the Dynamic Prior (D!"#$%) to robustlyidentify dynamic objects without task-specific training, leveraging the powerfulreasoning capabilities of Vision-Language Models (VLMs) and the fine-grainedspatial segmentation capacity of SAM2. D!"#$% can be seamlessly integrated intostate-of-the-art pipelines for camera pose optimization, depth reconstruction, and4D trajectory estimation. Extensive experiments on both synthetic and real-worldvideos demonstrate that D!"#$% not only achieves state-of-the-art performanceon motion segmentation, but also significantly improves accuracy and robustnessfor structural 3D understanding. Code is available.

ECCV 2026

Geometrically Consistent Multi-View Scene Generation from Freehand Sketches

Ahmed Bourouis, Savas Ozkan, Andrea Maracani, Yi-Zhe Song, Mete Ozay

We tackle a new problem: generating geometrically consistent multi-view scenes from a single freehand sketch. Freehand sketches are the most geometrically impoverished input one could offer a multiview generator. They convey scene intent through abstract strokes while introducing spatial distortions that actively conflict with any consistent 3D interpretation. No prior method attempts this; existing multi-view approaches require photographs or text, while sketch-to-3D methods need multiple views or costly per-scene optimisation. We address three compounding challenges; absent training data, the need for geometric reasoning from distorted 2D input, and cross-view consistency, through three mutually reinforcing contributions: (i) a curated dataset of ∼9k sketch-to-multiview samples, constructed via an automated generation and filtering pipeline; (ii) Parallel Camera-Aware Attention Adapters (CA3) that inject geometric inductive biases into the video transformer; and (iii) a Sparse Correspondence Supervision Loss (CSL) derived from Structure-from-Motion reconstructions. Our framework synthesizes all views in a single denoising process without requiring reference images, iterative refinement, or per-scene optimization. Our approach significantly outperforms state-of-the-art two-stage baselines, improving realism (FID) by over 60% and geometric consistency (Corr-Acc) by 23%, while providing up to a 3.7× inference speedup.

ECCV 2026

CASA: Cross-Attention over Self-Attention for Efficient Vision-Language Fusion

Moritz Bohle, Amelie Royer, Juliette Marrie, Edouard Grave, Patrick Perez

Vision-language models (VLMs) are commonly trained by directly inserting image tokens from a pretrained vision encoder into the text stream of a language model. This allows text and image information to fully attend to one another within the model, but becomes rapidly costly for long multi-image conversations or streaming video applications, both in terms of memory and compute. VLMs leveraging cross-attention are an efficient alternative to token insertion as image tokens are not added to the KV cache. Despite being introduced early on, multimodal cross-attention models are scarce in the current VLM literature and often underperform their token insertion counterparts. In this work, we reinvestigate the effectiveness of cross-attention for vision-language modeling: (i) We analyze the core differences between the cross-attention and selfattention mechanisms, (ii) we train cross-attention VLMs both from a text-only LLM and by adapting a pretrained insertion-based VLM, showing that simple cross-attention is more competitive with token insertion than previously reported, and (iii) we demonstrate the practical advantages of cross-attention on real-time video captioning, where it naturally maintains low latency and near-constant memory cost.

ECCV 2026

EgoEverything: A Benchmark for Human Behavior–Inspired Long-Context Egocentric Video Understanding in AR Environment

Qiance Tang, Ziqi Wang, Jieyu Lin, Ziyun Li, Barbara Salvo, Sai Qian Zhang

Long-context egocentric video understanding has recently attracted significant research attention, with augmented reality (AR) highlighted as one of its most important application domains. Nevertheless, the task remains highly challenging due to the need for reasoning over extended temporal contexts and diverse, unstructured activities. Although several benchmarks exist, most egocentric datasets rely on human-worn cameras and focus mainly on visual content, with limited consideration of underlying user behavior when forming video-related queries. EgoEverything is a benchmark that uses real gaze traces as a weak attention prior, rather than as a direct proxy for user intention, when generating questions. It comprises over 5,000 multiple-choice question-answer pairs, spanning more than 100 hours of video. By integrating measured gaze traces with a lightweight spatial sampling prior, it more faithfully captures natural AR-style querying behavior and offers a realistic evaluation setting for long-context egocentric video understanding in AR. We release our dataset at https://sai-lab-nyu.github.io/EgoEverything/.

ECCV 2026

SyncFix: Multi-View Consistent Diffusion Refinement of 3D Reconstructions

Deming Li, Cheng Peng, Abhay Kumar Yadav, Rama Chellappa, Anand Bhattad

We present SyncFix, a framework that enforces cross-view consistency during the diffusion-based refinement of reconstructed scenes. SyncFix formulates refinement as a joint latent bridge matching problem, synchronizing distorted and clean representations across multiple views to fix the semantic and geometric inconsistencies. This means SyncFix learns a joint conditional over multiple views to enforce consistency throughout the denoising trajectory. Our training is done only on image pairs, but it generalizes naturally to an arbitrary number of views during inference. Moreover, reconstruction quality improves with additional views, with diminishing returns at higher view counts. Qualitative and quantitative results demonstrate that SyncFix consistently generates high-quality reconstructions and surpasses current state-of-the-art baselines, even in the absence of clean reference images. SyncFix achieves even higher fidelity when sparse references are available.

ECCV 2026

SPICE: Simple Polysemantic feature Interpretation via Clustering-based Explanations

Sehyun Lee, Dahee Kwon, Damin Lee, Jaesik Choi

One of the pivotal recent challenges in neural network interpretability is polysemanticity, where a single neuron is activated by multiple, often unrelated concepts, hindering clear functional understanding. Although prior work has explored this phenomenon, existing approaches remain architecture-specific and depend on manual heuristics such as a fixed number of concept clusters (K), limiting their generality and scalability—especially for modern Transformer-based models. To address these limitations, we introduce SPICE (Simple Polysemantic Feature Interpretation via Clustering-based Explanation), a generalizable framework for analyzing polysemanticity in deep vision architectures. SPICE avoids architecture-dependent propagation rules, enabling the first systematic comparison of polysemanticity across both CNNs and Transformers, and automatically determines the number of concept clusters per neuron, eliminating reliance on a preset K and supporting scalable analysis for large models. Using SPICE, we conduct a comprehensive investigation into how polysemanticity emerges, varies across depth and architecture, and forms through distinct computational pathways.

ECCV 2026

UniDDT: Unifying Multimodal Understanding and Generation with Decoupled Diffusion Transformer

Shuai Wang, Liang Li, Yang Chen, Ruopeng Gao, Yao Teng, Limin Wang

Unified Multimodal Models (UMMs) have emerged as a critical direction for general-purpose multimodal intelligence, integrating understanding and generation into a single framework. However, existing UMMs face prominent challenges: (1) the inherent learning conflicts between visual understanding and generation tasks, leading to suboptimal modeling in both tasks; (2) different understanding and generation visual spaces impeding scalability; (3) over-reliance on task-specific data that neglects the duality of text-image understanding and generation. To address these challenges, we propose UniDDT, which leverages a Noisy ViT encoder along with a LLM to unify semantic encoding for visual generation and understanding tasks, while employing a separate diffusion decoder to decouple diffusion decoding from text decoding. With this Noisy ViT encoder, UniDDT is able to leverage the latent space as a unified visual representation, enabling seamless compatibility between understanding and generation tasks. Thus, the scalability within the generation tasks and the semantic expressiveness within understanding tasks can be balanced. Also, we construct dual data structures from the same image-text pairs, fostering interdependence between the generation and understanding data to exploit their inherent duality. Extensive experiments demonstrate that UniDDT achieves effective unification of multimodal understanding and generation with enhanced semantic consistency and scalability. For visual generation tasks, our UniDDT achieves 0.86 GenEval score and 86.9 DPG overall score. For multimodal understanding tasks, our UniDDT achieves 1699.5 score on MME benchmark and 76.5 overall score on SEEDbench.

ECCV 2026

EgoDyn-Bench: Evaluating Ego-Motion Understanding in Vision-Centric Foundation Models for Autonomous Driving

Finn Rasmus Schäfer, Yuan Gao, Dingrui Wang, Thomas Stauner, Stephan Günnemann, Mattia Piccinini, Sebastian Schmidt, Johannes Betz

While Vision-Language Models (VLMs) have advanced highlevel reasoning in autonomous driving, their ability to ground this reasoning in the underlying physics of ego-motion remains poorly understood. We introduce EgoDyn-Bench4, a diagnostic benchmark for evaluating the semantic ego-motion understanding of vision-centric foundation models. By mapping continuous vehicle kinematics to discrete motion concepts via a deterministic oracle, we decouple a model’s internal physical logic from its visual perception. Our large-scale empirical audit spanning 20+ models, including closed-source MLLMs, open-source VLMs across multiple scales, and specialized VLAs, identifies a significant Perception Bottleneck: while models exhibit logical physical concepts, they consistently fail to accurately align them with visual observations, frequently underperforming classical non-learned geometric baselines. This failure persists across model scales and domain-specific training, indicating a structural deficit in how current architectures couple visual perception with physical reasoning. We demonstrate that providing explicit trajectory encodings substantially restores physical consistency across all evaluated models, revealing a functional disentanglement between vision and language: ego-motion logic is derived almost exclusively from the language modality, while visual observations contribute negligible temporal signal. This structural finding provides a standardized diagnostic framework and a practical pathway toward physically aligned embodied AI.

ECCV 2026

TetraSDF: Analytic Isosurface Extraction with Multi-resolution Tetrahedral Grid

Seonghun Oh, Youngjung Uh, Jin-Hwa Kim

Extracting an explicit surface that exactly matches the zerolevel set of a neural signed distance function (SDF) remains challenging. Sampling-based isosurfacing methods such as Marching Cubes introduce discretization error. In contrast, continuous piecewise affine (CPWA) analytic approaches typically require plain ReLU MLPs, which limits the ability to learn high-frequency SDFs in practice. We present TetraSDF, an analytic isosurface extraction framework for SDFs that retains the expressiveness of grid-based encoders while enabling exact zero-level set extraction, by representing the SDF with a ReLU MLP composed with a multi-resolution tetrahedral positional encoder. Our positional encoder’s barycentric interpolation preserves a global CPWA structure, allowing us to track ReLU linear regions within an encoder-induced polyhedral complex. We further introduce a fixed analytic input preconditioner derived from the encoder’s metric to reduce directional bias, thereby stabilizing training. Across multiple benchmarks, TetraSDF matches or surpasses existing grid-based encoders in SDF reconstruction accuracy, while faithfully recovering the network’s zero-level set as a triangle mesh.

ECCV 2026

GO-Renderer: Generative Object Rendering with 3D-aware Controllable Video Diffusion Models

Zekai Gu, Shuoxuan Feng, Yansong Wang, Hanzhuo Huang, Zhongshuo Du, Chengfeng Zhao, Chengwei Ren, Peng Wang, Yuan Liu

Reconstructing a renderable 3D model from images is a useful but challenging task. Recent feedforward 3D reconstruction methods have demonstrated remarkable success in efficiently recovering geometry, but still cannot accurately model the complex appearances of these 3D reconstructed models. Recent diffusion-based generative models can synthesize realistic images or videos of an object using reference images without explicitly modeling its appearance, which provides a promising direction for object rendering, but lacks accurate control over the viewpoints. In this paper, we propose GO-Renderer, a unified framework integrating the reconstructed 3D proxies to guide the video generative models to achieve high-quality object rendering on arbitrary viewpoints under arbitrary lighting conditions. Our method not only enjoys the accurate viewpoint control using the reconstructed 3D proxy but also enables high-quality rendering in different lighting environments using diffusion generative models without explicitly modeling complex materials and lighting. Extensive experiments demonstrate that GO-Renderer achieves state-of-the-art performance across the object rendering tasks, including synthesizing images on new viewpoints, rendering the objects in a novel lighting environment, and inserting an object into an existing video. Project page: https://igl-hkust.github.io/GO-Renderer.

ECCV 2026

POET: Preference Optimization for Enhanced Text-to-Image Generation

Ruibo Chen, Jiacheng Pan, Heng Huang, Zhenheng Yang

Recent advances in text-to-image (T2I) generation have achieved impressive results, yet existing models often struggle with simple or underspecified user prompts due to a distributional gap with their descriptive training captions. This frequently leads to suboptimal imagetext alignment, aesthetics, and overall visual quality. To bridge this gap, we propose POET (Preference Optimization for Enhanced Text-to-Image generation), an automated prompt rewriting framework that leverages large language models (LLMs) to refine user inputs before feeding them into frozen T2I backbones. POET introduces a carefully designed composite reward system and an iterative Direct Preference Optimization (DPO) training pipeline, enabling the rewriter to learn model-preferred prompt structures directly from multimodal feedback without requiring costly high-quality supervised fine-tuning (SFT) data. Extensive evaluations across diverse T2I models and benchmarks show that our prompt rewriter consistently improves image-text alignment, visual quality, and aesthetics, outperforming strong baselines. Furthermore, we demonstrate strong transferability by showing that a rewriter trained on one T2I backbone generalizes effectively to others without needing to be retrained. These findings highlight that POET is an effective, robust, and practical model-agnostic strategy for improving T2I systems.

ECCV 2026

Neural Collapse-Inspired Multi-Label Federated Learning under Label-Distribution Skew

Can Peng, Yuyuan Liu, Yingyu Yang, Pramit Saha, Qianye Yang, J. Noble

Federated Learning (FL) enables collaborative model training across distributed clients while preserving data privacy, but remains challenging when client data are highly heterogeneous. These challenges are further amplified in multi-label scenarios, where inter-label dependencies and mismatches between local and global label relationships introduce additional optimization conflicts. While most FL studies focus on single-label classification, many real-world applications are inherently multi-label and often exhibit severe label skew across clients. To address this important yet underexplored problem, we propose FedNCA-ML, a novel FL framework that aligns client representations and learns discriminative, well-clustered features inspired by Neural Collapse (NC) theory. NC describes an ideal latent geometry where each class’s features collapse to their mean, forming a maximally separated simplex. FedNCAML further introduces an attention-based module to extract class-specific representations, enabling more balanced learning under heavy label imbalance. These class-wise representations are then aligned via a shared NC-inspired structure, mitigating inter-client conflicts induced by heterogeneous local data and inconsistent label dependencies. In addition, we design regularisation losses to encourage compact and consistent feature clustering in the latent space. Experiments on five benchmark datasets under nine FL settings demonstrate the effectiveness of the proposed method, achieving improvements of up to 3.92% in class-wise AUC and 4.93% in class-wise F1 score. Code is available at https: //github.com/CanPeng123/multi_label_fl_fednca.

ECCV 2026

Motion Style Slider: Endpoint-Supervised Continuous Style Control for Human Motion Diffusion

Chen-Chieh Liao, YICHEN PENG, Yiyi Cai, Yûi Ono, Hiroki Hanaoka, Erwin Wu, Hideki Koike, Shuichi Kurabayashi

Existing human motion diffusion methods provide strong motion generation quality [35], and recent style transfer models can inject target style cues [11,33], but fine-grained continuous control of style intensity remains underexplored. In production, style intensity is subjective across artists and directors, so the practical requirement is not a universal absolute unit (e.g., globally correct “2×”), but a reliable monotonic control axis. We propose Motion Style Slider, a motion-to-motion style transfer framework for endpoint-supervised continuous control. Given a content motion and a style motion, we construct a style direction in a learned motion-style embedding space and condition diffusion generation with a scalar intensity α. The training objective combines diffusion denoising with latent intensity regularization to encourage smooth and monotonic style scaling without requiring intermediate-intensity groundtruth motions. Our framework is compatible with pretrained motion diffusion backbones and supports heterogeneous style datasets, including the multi-actor style motion dataset [20]. To test out-of-range usability, we additionally introduce a small real-capture over-reaction extension and evaluate large-α behavior against these unseen targets. Experiments measure controllability, interpolation/extrapolation behavior, content preservation, and motion realism, with ablations on direction construction and loss design.

ECCV 2026

Detect by Track: Making Detector-Free Matcher Trackable

Yusuke Sekikawa, HIDEKI SHIRAI, Ruka Eto, Yuzhe Hao, Kengo Mitsui, Nakamasa Inoue

Detector–free matchers (e.g., EDM, JamMa) offer a strong accuracy– latency trade-off for pairwise matching. However, their implicit keypoint selection—where keypoints emerge as a by-product of matching—often leads to multi-view inconsistency and fragmented tracks. This limitation hinders applications that require connected tracks, such as object tracking, SfM, and SLAM. Prior attempts to mitigate this issue rely on coarsely quantized matches combined with heavy post-processing for multiple images; thus, they are typically limited to batch-based SfM pipelines. We take a different approach and directly address multi-view inconsistency through a Detect-by-Tracking (DeT) mechanism. Given an image pair I , I , and a sub-pixel query q , DeT steers the local A B A similaritymatrixofadetector-freematchersothattherow-wisesoftmaxrepresents the correspondence probability of an arbitrary sub-pixel query q . This produces A connected tracks across views without heavy post-processing, while preserving the core strength of detector-free matching: implicit keypoint selection driven by matching. Moreover, DeT can be incorporated into off-the-shelf matchers without additional training. Experiments demonstrate that DeT maintains the base detector-free matcher’s matching accuracy while producing connected tracks.

ECCV 2026

IoUCert: Robustness Verification for Anchor-based Object Detectors

Benedikt Brückner, Alejandro J. Mercado, Yanghao Zhang, Panagiotis Kouvaros, Alessio Lomuscio

While formal robustness verification has seen significant success in image classification, scaling these guarantees to object detection remains notoriously difficult due to complex non-linear coordinate transformations and Intersection-over-Union (IoU) metrics. As a fundamental step towards verifying complete detection pipelines, we introduce IoUCert, a novel formal verification framework designed specifically to overcome these core mathematical bottlenecks. By isolating the object localisation task in single-object settings, we propose a coordinate transformation that circumvents precision-degrading relaxations of non-linear box prediction functions. This approach allows us to optimise bounds directly with respect to anchor box offsets, enabling a novel Interval Bound Propagation method that derives optimal IoU bounds. We demonstrate that IoUCert enables, for the first time, the robustness verification of foundational, anchor-based architectures including tractable variants of SSD, YOLOv2, and YOLOv3 against various input perturbations, providing a rigorous theoretical basis for future end-to-end detector verification.

ECCV 2026

PerceptionComp: A Video Benchmark for Complex Perception-Centric Reasoning

Shaoxuan Li, Zhixuan Zhao, Hanze Deng, Zirun Ma, Shulin Tian, Zuyan Liu, Yushi Hu, Haoning Wu, Yuhao Dong, Benlin Liu, Ziwei Liu, Ranjay Krishna

Deep video understanding requires long-horizon, perceptioncentric reasoning that repeatedly revisits a video to gather temporally distributed evidence. However, existing benchmarks are either relatively easy (perception-centric but often solvable after a single view) or logicheavy with simplified visuals, and thus do not faithfully measure multimodal test-time thinking that depends on repeated perception. We introduce PerceptionComp, a fully manually annotated benchmark designed so that no single moment is sufficient: answering requires evidence from multiple temporally separated segments under compositional constraints. PerceptionComp contains 1,114 five-choice questions over 279 high-scene-complexity videos spanning diverse domains. Videos are selected using automatic proxies for scene complexity (SAM2 instance counts and optical-flow magnitude), and each question requires 10–20 minutes of annotation. Human evaluation confirms the intended difficulty: PerceptionComp requires substantially longer response times than prior benchmarks, and under a single-view setting (no rewatching) human accuracy drops to near chance (18.97%), while experts can reach 100% accuracy with unrestricted rewatching and sufficient time. Stateof-the-art MLLMs perform notably worse: the best model in our evaluation (Gemini-3-Flash) reaches only 45.96% accuracy, and open-source MLLMs remain below 40%. Test-time reasoning helps but remains far from human-level (e.g., GPT-o3 exceeds GPT-4o by 11.04%; Gemini- 2.5-Pro exceeds Gemini-2.5-Flash by 6.19%), and increasing test-time compute via larger thinking-token budgets or more input frames further improves performance. Finally, among the strongest frontier models we tested (Gemini-3 variants and GPT-o3), accuracies cluster in the mid-40s, suggesting a bottleneck in perception-centric long-horizon video reasoning. PerceptionComp provides a focused testbed for diagnosing these limitations and advancing multimodal visual thinking.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.