paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

ECCV 2026

Describe-Then-Act: Proactive Agent Steering via Distilled Language-Action World Models

Massimiliano Pappa, Luca Romani, Valentino Sacco, Alessio Palma, Stephane Lathuiliere, Fabio Galasso, Xavier Alameda-Pineda, Indro Spinelli

Deploying safety-critical agents requires anticipating the consequences of actions before they are executed. While world models offer a paradigm for this proactive foresight, current approaches relying on visual simulation incur prohibitive latencies, often exceeding several seconds per step. In this work, we challenge the assumption that visual processing is necessary for failure prevention. We show that a trained policy’s latent state, combined with its planned actions, already encodes sufficient information to anticipate action outcomes, making visual simulation redundant for failure prevention. To this end, we introduce DILLO (DIstiLLed Language-ActiOn World Model), a fast steering layer that shifts the paradigm from “simulate-then-act” to “describe-thenact.” DILLO is trained via cross-modal distillation, where a privileged Vision Language Model teacher annotates offline trajectories and a latentconditioned Large Language Model student learns to predict semantic outcomes. This creates a text-only inference path, bypassing heavy visual generation entirely, achieving a 14× speedup over baselines. Experiments on MetaWorld and LIBERO demonstrate that DILLO produces high-fidelity descriptions of the next state and is able to steer the policy, improving episode success rate by up to 15pp and 9.3pp on average across tasks. Code is available at github.com/MaxPappa/DILLO.

ECCV 2026

WiFlow: Estimating Optical Flow using WiFi Channel State Information

Thomas Weigel, Simon Kiefhaber, Fabian Portner, Matthias Hollick, Simone Schaub-Meyer

Knowing where and how fast objects are moving within a scene is important across various domains. Usually, cameras are used to capture the data necessary for this task, but adding cameras often raises privacy concerns, and the quality of captured frames is heavily influenced by lighting conditions. In this work, we explore using WiFi channel state information (CSI) instead of camera frames for optical flow estimation. We propose WiFlow, a CSI based flow estimator, a preprocessor evaluation for CSI, and three model architectures that offer different trade-offs between accuracy and complexity. Further, we create the first dataset for training and evaluating CSI-based optical flow estimators, and our experiments provide insights into key design elements for this task. Code and data are available at https://visinf.github.io/wiflow.

ECCV 2026

YeTI: You Only Need Two Noisy Images for Real-World sRGB Noise Generation

Jaekyun Ko, Byung Wan Lim, Dongjin Kim, Soomin Lee, Tae Hyun Kim

Real-world sRGB image denoising remains challenging due to the nonlinear characteristics of sensor noise and the difficulty of acquiring aligned clean-noisy image pairs. Supervised denoisers often overfit to limited paired datasets, while self-supervised methods still depend on sufficiently diverse noisy observations. These limitations motivate scalable noise synthesis methods that can model real-world noise without clean ground truth or camera metadata. We propose YeTI, a real-world sRGB noise generation framework that learns from only two noisy observations of the same scene. YeTI uses a Reconstruction Autoencoder to disentangle scene structure and noise characteristics, and models the latent noise distribution with a one-step Conditional Diffusion Transformer trained using consistency objectives. Given a single noisy input at inference time, YeTI generates realistic, signal-dependent noise while preserving the underlying scene content. Extensive experiments demonstrate the effectiveness of YeTI across real-world benchmarks. We evaluate noise generation on SIDD and further assess generalization on SIDD+, MAI2021, and SID, covering smartphone and diverse consumer-camera sensors. Downstream denoising results on DND further show that denoisers trained with YeTI-synthesized images achieve strong real-world performance, highlighting the practical value of clean-image-free and metadata-free noise generation. Code is available at: https://github.com/ByungWanLim/YeTI-You-Only-NeedTwo-Noisy-Images-for-Real-World-sRGB-Noise-Generation

ECCV 2026

ELDiff: When Evidential Learning Meets Text-to-Image Diffusion

Qingtao Pan, Kai Ye, Zhihao Dou, Bing Ji, Shuo Li

In multi-object text-to-image (T2I) diffusion, ensuring semantic consistency between textual prompts and generated visual content is crucial for image synthesis. However, such consistency constraint is often underemphasized in the denoising process of diffusion models. Although token supervised diffusion models can mitigate this issue by learning object-wise consistency between the image content and object segmentation maps, it tends to suffer from the problems of segmentation map bias and semantic overlap conflict, especially when involving multiple objects. In this paper, we propose ELDiff, a new evidential learningsupervised T2I diffusion model, which leverages the advantages of uncertainty metric and conflict detection to enhance the fault tolerance of unreliable segmentation maps and suppress semantic conflicts, strengthening object-wise consistency learning. Specifically, a pixel evidence loss is proposed to restrain overconfidence in unreliable labels through evidential regularization, and a token conflict loss is designed to weaken the contradiction between semantics through optimizing a measured conflict factor. Extensive experiments show that our ELDiff outperforms existing training based and train-free based T2I diffusion models on SD v1.4, SD v2.1, SDXL, SD v3.5, and Qwen-Image, without requiring additional inference-time manipulations. Notably, ELDiff can be seamlessly extended to the existing training pipeline of T2I diffusion models. Code can be found at https://github.com/QingtaoPan/ELDiff.

ECCV 2026

Prior-Conditioned Gaussian Discriminants for Generalizable AI-generated Image Detection

Shashank Kotyan, Makoto Shing, Yuki Imajuku, Rujikorn Charakorn, Tarin Clanuwat

Diffusion-based generators have made synthetic images ubiquitous, but detectors often fail under simultaneous shifts in generator, prompt/style, and source-domain. We study AI-generated image detection as a transfer system described by training prior, frozen representation, and decision rule, and ask when classifier head training adds value beyond what is already separable in modern features. As a controlled diagnostic, we fit a prior-conditioned Gaussian discriminant ladder: closed-form heads built from firstand second-order feature statistics under nested covariance assumptions. On Percept-Lens, a unified protocol over 39 public datasets (7.1 million images), the best rung is frequently competitive with, and sometimes exceeds, released detector heads when matched on both prior and encoder. We further quantify strong sensitivity to the training prior, data-efficiency of moment-based heads, and representation dependence of Gaussian shift metrics, motivating (prior, encoder, head)-level reporting and stronger analytical baselines for AIGI transfer.

ECCV 2026

Seeing Isn't Orienting: A Cognitively Grounded Hierarchical Benchmark for Object Orientation in MLLMs

Nazia Tasnim, Keanu Nichols, Yuting Yan, Nicholas Ikechukwu, Elva Zou, Deepti Ghadiyaram, Bryan Plummer

Humans develop object orientation understanding progressively, from recognizing which way an object faces, to mentally rotating it, to reasoning about how multiple objects are oriented relative to each other. Yet current vision-language benchmarks treat orientation as an afterthought, conflating it with positional relationships and general scene understanding. We introduce Discriminative Orientation Reasoning Intelligence (DORI), a cognition-informed hierarchical benchmark that establishes object orientation as the primary evaluation target. Specifically, DORI decomposes orientation into four dimensions, each assessed at both coarse (categorical) and granular (metric) levels. This results in 33,656 multiple-choice questions over diverse open-vocabulary objects within 13,652 real-world and synthetic images taken from 14 sources. DORI’s coarse-to-granular design isolates orientation from confounds such as object recognition difficulty, scene clutter, and linguistic ambiguity through bounding-box isolation, standardized spatial reference frames, and structured prompts. Our evaluation of 26 state-of-the-art vision-language models reveals a consistent pattern: models competent on general spatial benchmarks remain near-random on object-centric orientation tasks. Even the best models achieve only 64.2% on coarse and 42.9% on granular judgments, with the largest drops on compound rotations and inter-object reference frame shifts. Large coarse-to-granular gaps further expose that models rely on categorical heuristics rather than geometric reasoning, a limitation invisible to existing benchmarks. These findings establish orientation understanding as an unsolved challenge in multimodal systems, with direct implications for robotic manipulation, 3D scene reconstruction, and human-AI interaction. Find the dataset here: https://huggingface.co/datasets/appledora/DORI-Benchmark

ECCV 2026

PartCHOI: Part-Aware Guidance for Clothed Human-Object Interaction Generation

Mingwen Shao, Xinyuan Chen, Qiao Zhang, Xiang Lv, Lingzhuang Meng, Qinglin Zhan, Chang Liu, Chao Dong

Existing single-view 3D human generation methods remain limited in modeling clothed human–object interactions (CHOI), due to two key challenges: (1) the lack of reliable guidance for human–object spatial relationships resulting in implausible layouts, and (2) the absence of physical interaction modeling between entities leading to unrealistic contacts. To address these issues, we propose PartCHOI, a part-aware method that formulates the CHOI task as generating a human-centered scene with objects treated as parts of the human body. Specifically, to model spatial relationships between objects and the human, we design a Human-Guided Pose Estimator (H-GPE) that efficiently infers objectpart poses by separating and localizing them within the coupled interaction scene. Furthermore, we devise an Object-Guided Refinement Module (O-GRM) to perform fine-grained human surface deformation for preventing penetration into objects, thus yielding realistic contact. Benefiting from the above design, PartCHOI generates plausible and detailed interactive scenes while avoiding lengthy layout optimization. Extensive experiments demonstrate our approach surpasses state-of-the-art methods in generation and interaction quality.

ECCV 2026

Analytic Bayesian Uncertainty for LiDAR Segmentation: A Single-pass Generative Approach

Hanieh Shojaei Miandashti, Qianqian Zou, Claus Brenner

Accurate predictions with well-calibrated uncertainties are vital for safety-critical LiDAR scene understanding in autonomous driving and robotics. While Bayesian Neural Networks and deep ensembles are effective, they are computationally expensive, requiring multiple forward passes or independently trained models. We propose GMM-NIG, a Bayesian generative classification head that extends the GMMSeg framework by placing conjugate Normal-Inverse-Gamma (NIG) priors over class-conditional GMM parameters to obtain posterior-predictive uncertainty. While the backbone remains deterministic, GMM-NIG enables analytic closed-form posterior updates in a single training run. In inference, GMM-NIG yields a mixture of Student-t distributions, providing wellcalibrated predictive uncertainty without extra computational overhead. On SemanticKITTI and nuScenes, GMM-NIG achieves state-of-the-art calibration with an Adaptive Calibration Error of 2.01%, outperforming deep ensembles, Monte Carlo Dropout, evidential models and post-hoc calibration methods. Beyond calibration, it increases segmentation mIoU by nearly 6% and yields the strongest failure detection performance.

ECCV 2026

TORA: Topological Representation Alignment for 3D Shape Assembly

Nahyuk Lee, Zhiang Chen, Marc Pollefeys, Sunghwan Hong

Flow-matching methods for 3D shape assembly learn pointwise velocity fields that transport parts toward assembled configurations, yet they receive no explicit guidance about which cross-part interactions should drive the motion. We introduce TORA, a topology-first representation alignment framework that distills relational structure from a frozen pretrained 3D encoder into the flow-matching backbone during training. We first realize this via simple instantiation, token-wise cosine matching, which injects the learned geometric descriptors from the teacher representation. We then extend to employ a Centered Kernel Alignment (CKA) loss to match the similarity structure between student and teacher representations for enhanced topological alignment. Through systematic probing of diverse 3D encoders, we show that geometryand contact-centric teacher properties, not semantic classification ability, govern alignment effectiveness, and that alignment is most beneficial at later transformer layers where spatial structure naturally emerges. TORA introduces zero inference overhead while yielding two consistent benefits: faster convergence (up to 6.9×) and improved accuracy in-distribution, along with greater robustness under domain shift. Experiments on five benchmarks spanning geometric, semantic, and inter-object assembly demonstrate state-of-the-art performance, with particularly pronounced gains in zero-shot transfer to unseen real-world and synthetic datasets. Project page: https://nahyuklee.github.io/tora

ECCV 2026

Learn2Fold: Structured Origami Generation with World Model Planning

Yanjia Huang, Yunuo Chen, Ying Jiang, Zhengzhong Tu, Yin Yang, Chenfanfu Jiang

The ability to transform a flat sheet into a complex threedimensional structure is a fundamental test of physical intelligence. Unlike cloth manipulation, origami is governed by strict geometric axioms and hard kinematic constraints, where a single invalid crease or collision can invalidate the entire folding sequence. As a result, origami demands long-horizon constructive reasoning that jointly satisfies precise physical laws and high-level semantic intent. Existing approaches fall into two disjoint paradigms: optimization-based methods enforce physical validity but require dense, precisely specified inputs, making them unsuitable for sparse natural language descriptions, while generative foundation models excel at semantic and perceptual synthesis yet fail to produce longhorizon, physics-consistent folding processes. Consequently, generating valid origami folding sequences directly from text remains an open challenge. To address this gap, we introduce Learn2Fold, a neuro-symbolic framework that formulates origami folding as conditional program induction over a crease-pattern graph. Our key insight is to decouple semantic proposal from physical verification. A large language model generates candidate folding programs from abstract text prompts, while a learned graph-structured world model serves as a differentiable surrogate simulator that predicts physical feasibility and failure modes before execution. Integrated within a lookahead planning loop, Learn2Fold enables robust generation of physically valid folding sequences for complex and out-of-distribution patterns, demonstrating that effective spatial intelligence arises from the synergy between symbolic reasoning and grounded physical simulation.

ECCV 2026

TIGER: Taming Identity, Geometry, and Generative Priors for High-Quality Face Video Restoration

Yang Zhou, Wenxue Li, Peng Zhang, Yifei Chen, Fei Wang, Daiguo Zhou

Face Video Restoration (FVR) aims to recover high-fidelity facial videos from degraded input while preserving identity and semantic consistency across frames. Existing methods often struggle to simultaneously address three key challenges: identity shift, viewpoint-entangled guidance, and perceptual realism. To tackle these issues, we propose TIGER, a structured tri-prior fusion framework that Tames Identity, Geometry, and gEnerative pRiors for high-quality FVR. Specifically, an Identity Prior is first established by injecting subject-discriminative embeddings into the latent space, effectively anchoring the subject’s identity against severe degradations. Then, to provide temporally consistent structural guidance for dynamic videos, TIGER constructs a Geometry Prior by lifting 2D reference cues into a disentangled 3D parameter space, creating a geometric anchor through cross-source parameter fusion. Moreover, to achieve better efficiency without compromising realism, we harness the video generation model’s Generative Prior through a one-step rectified flow. We further design a progressive three-stage training optimazition strategy that refines structural fidelity, textural reconstruction, and distribution-level realism to ensure robust optimization. We also construct a large-scale FVR dataset to facilitate robust training and standardized evaluation. Extensive experiments demonstrate that TIGER achieves state-of-the-art performance in both identity fidelity and temporal stability, delivering a high-quality, efficient and identityconsistent FVR. Project page: https://yzhoulv.github.io/Tiger/.

ECCV 2026

HVGCD:Rethinking Generalized Category Discovery through Hypothesis–Verification

zhang baoqiang, Kunze Huang, Luyao Tang, Xiaotong Tu

Generalized Category Discovery (GCD) aims to simultaneously recognize known categories and discover novel ones from partially labeled data without annotations. Existing methods rely on single-path global embeddings, implicitly performing only hypothesis formation and omitting an explicit verification mechanism, which leads to over-reliance on pretrained semantics and confident misclassification of unknown categories. We reformulate GCD as a Hypothesis–Verification representation framework and propose HVGCD, a structured representation framework that decomposes inference into a global hypothesis, hypothesisconditioned verification evidence, and statistical grounding. The verification branch reconstructs discriminative evidence via content-adaptive prototypes, enabling the model to verify and refine its hypotheses under open-world uncertainty, while a maximum-entropy grounding term stabilizes inference in open-world settings. As an efficient module, HVGCD integrates seamlessly into existing GCD pipelines and consistently improves performance on both coarseand fine-grained benchmarks.

ECCV 2026

BrainFIBRE: A Foundation Model via Information Decomposition for Brain Microstructure

Zijian Dong, Yi Lin, Fang Ji, Jianxiong Zhou, Eric Kwun Kei NG, Juan Zhou

Diffusion MRI is widely used to probe brain microstructure, with particular sensitivity to early cerebrovascular and neurodegenerative changes. Neurite Orientation Dispersion and Density Imaging (NODDI) decomposes the diffusion signal into three biophysically interpretable maps — neurite density index (NDI), orientation dispersion index (ODI), and free water fraction (FWF) — capturing neurite packing, fiber coherence, and extracellular fluid, respectively. These 3D maps provide a rich substrate for learning transferable microstructural representations for the human brain, yet effectively integrating them remains an open challenge: standard representation learning struggles to disentangle the unique information carried by each of the three brain microstructure maps (i.e., NDI, ODI, and FWF) from their shared and synergistic interactions. Here, we present BrainFIBRE (Brain Foundation Model via Information Decomposition for BRain MicrostructurE), the first foundation model for brain microstructure, pretrained on three NODDI-derived microstructure maps from the UK Biobank dataset (55,592 participants). To achieve this, we propose Self-supervised Partial Information Decomposition (SPID), which extends PID-guided multimodal learning to the self-supervised regime for the first time. A novel Counterfactual Candidate Construction (CCC) paradigm perturbs inter-modality alignment through modality dropping and swapping, providing the contrastive signal for a Mixture-of-Experts (MoE) architecture to disentangle unique, synergistic, and redundant information without any downstream label. Evaluated on both Caucasian and Asian cohorts, our model achieves state-of-the-art performance across diverse downstream tasks predicting age, sex, cerebrovascular disease (CeVD) and neurodegenerative markers, and cognitive performance, while yielding neurobiologically interpretable representations that reveal taskand cohort-specific interaction patterns among microstructural compartments. BrainFIBRE establishes a versatile foundation for neuroimaging analysis at the microstructural level. Code is available at https://github.com/hzlab/BrainFIBRE

ECCV 2026

Accelerating Multimodal Large Language Models with Prior-Corrected Token Reduction

Zengjie Chen, Yuxiang Cai, Jingcai Guo, Taotao Cai, Jianwei Yin, Zhi Chen

Visual token reduction has emerged as an effective strategy for accelerating Multimodal Large Language Models (MLLMs). Many existing methods prune tokens by ranking text–visual attention scores. However, we show that attention is often dominated by a model-induced prior: even without textual instruction, MLLMs tend to focus on certain task-agnostic regions. Consequently, the attention scores of instructionconditioned tokens are suppressed, increasing the risk that these tokens are discarded during pruning. To address this issue, we propose PriorCorrected Token Reduction (PriorTR), a training-free token reduction method that explicitly separates task-conditioned attention from the model-induced prior. PriorTR estimates the attention map of the prior, and contrasts it with the task-conditioned attention distribution to measure the additional usable information contributed by each visual token. Importantly, PriorTR computes both the model-induced prior and the task-conditioned posterior within a single forward pass by introducing a null token that serves as an instruction-agnostic probe in the attention block. This design avoids duplicated propagation. Extensive experiments across multiple multimodal benchmarks and MLLMs demonstrate that PriorTR consistently improves the trade-off between accuracy and efficiency over strong training-free baselines, particularly under aggressive token budgets.

ECCV 2026

Spanning Tree Autoregressive Visual Generation

Sangkyu Lee, Changho Lee, Janghoon Han, Hosung Song, Tackgeun You, Hwasup Lim, Stanley Jungkyu Choi, Honglak Lee, Youngjae Yu

We present Spanning Tree Autoregressive (STAR) modeling, which can incorporate prior knowledge of images, such as center bias and locality, to maintain sampling performance while also providing sufficiently flexible sequence orders to accommodate image editing at inference time. Approaches that expose conventional autoregressive (AR) models in visual generation to arbitrary sequence orders via random permutation suffer from degraded sampling performance or compromise the flexibility in sequence order choice at inference time. Instead, STAR utilizes traversal orders of uniform spanning trees in a lattice defined by the positions of image patches. Traversal orders are obtained via breadth-first search, allowing us to efficiently construct a spanning tree via rejection sampling whose traversal order ensures that the connected partial observation of the image appears as a prefix for native image inpainting support. Through the tailored yet structured sequence order randomization strategy, STAR preserves the capability of postfix completion while maintaining sampling performance, without any significant changes to the model architecture widely adopted in language AR modeling.‡

ECCV 2026

HilDA: Hierarchical Distillation with Diffusion for Advancing Self-Supervised LiDAR Pre-training

Maciej Wozniak, Jesper Ericsson, Hariprasath Govindarajan, Truls Nyberg, Thomas Gustafsson, Patric Jensfelt, Olov Andersson

Leveraging Vision Foundation Models (VFMs) for camera-toLiDAR knowledge distillation offers a promising solution to the scarcity of annotated data needed to represent the immense geometric and kinematic diversity of real-world autonomous driving (AD). However, current approaches typically treat VFMs as black-box teachers, relying exclusively on frame-wise feature similarity. Consequently, they do not fully exploit the teacher’s layer-wise semantic structure and global context, as well as the rich spatiotemporal information inherent in LiDAR sequences. We propose HilDA, a self-supervised pre-training framework for LiDAR backbones that better captures the semantic what and geometric where needed for driving tasks. HilDA combines hierarchical distillation comprising multi-layer distillation for progressive semantic alignment and global context distillation for scene-level semantics, with a temporal occupancy diffusion objective promoting spatiotemporal consistency. Models pre-trained with HilDA achieve state-of-the-art results on cross-modal distillation benchmarks and outperform models trained via prior distillation approaches on 3D object detection, scene flow, and semantic occupancy prediction. Code available at: https://maxiuw.github.io/hilda.

ECCV 2026

One Video, One World: Turning Monocular Video into Physical 4D Scenes

Junhao Chen, Boran Zhang, Mingjin Chen, Henghaofan Zhang, Saining Zhang, Congcong Zhu, Hao Zhao, Ruqi Huang, Zhihao Li, Yufei Wang

We introduce OVOW, the first training-free system that reconstructs instance-level, simulation-ready 4D mesh scenes from a single monocular video. Recent 4D reconstruction achieves impressive rendering quality, but its outputs (e.g., implicit fields, Gaussian primitives, or point clouds) lack the watertight topology, instance separation, and standardized physical interfaces required by physics simulators and embodied AI. OVOW closes this gap with a four-stage pipeline: a vision-language model discovers, labels, and motion-classifies all instances; category-aware reconstruction yields per-instance meshes for rigid objects and topology-consistent mesh sequences for deformable ones; an iterative render-match-optimize procedure recovers metric scale and 6-DoF pose trajectories; and physics-grounded assembly enforces ground contact and inter-object support. Crucially, we model all motion, rigid and non-rigid, through direct vertex deformation without category-specific priors or skeleton rigging, producing watertight mesh scenes ready for downstream physics simulation and editing. We further establish the first benchmark for structured Video-to-4D evaluation, with metrics for geometric correctness, instance separation, and physical plausibility beyond visual fidelity; the same pipeline doubles as a scalable engine for synthesizing paired video-to-4D simulation data for future 4D world models and embodied AI. Across two synthetic benchmarks (static and 4D), OVOW attains the best overall layout and geometry accuracy and the lowest photometric and semantic error among all baselines, and on monocular video runs one to two orders of magnitude faster than the baselines, while downstream physics simulation confirms its physical stability.

ECCV 2026

MetaPoint: Unlocking Precise Spatial Control in Visual Generation

Dewei Zhou, Xinyu Huang, Xun Wang, Ji Xie, Yabo Zhang, Liang Li, Kunchang Li, Zongxin Yang, Yi Yang

Generative visual models fundamentally struggle with precise spatial control. This arises from a core disconnect: models can process textual descriptions of space but cannot directly map numerical coordinates onto the 2D image canvas (as illustrated in Fig. 2). We introduce MetaPoint, a method that bridges this gap by representing a continuous 2D coordinate as a single, special token. Crucially, MetaPoint requires no new architectural components; it directly leverages the model’s inherent positional encoding schemes to interpret these coordinates, treating our token as a virtual point on the canvas. This lightweight approach enables pixel-level control of an object’s position with one token or its bounding box with two, all without requiring architectural changes or bespoke attention masking. The MetaPoint tokens are designed to be compositional, serving as spatial primitives. This allows a planner agent to decompose a high-level user request into a structured sequence of primitives for the generator. By providing a simple, precise, and scalable building block for spatial control, MetaPoint unlocks more powerful compositional generative agents and enables intuitive, interactive editing systems.

ECCV 2026

Multi-scale Object-Aware Gaze Estimation via Geometric Reasoning

Jiajie Mi, Xinyu Liu, Mengke Song, Chenglizhao Chen

Gaze target estimation aims to predict the semantic object an observer fixates upon within an image, a task deeply rooted in the object-oriented nature of human gaze. Observers tend to select a specific semantic entity as the attentional target, rather than responding randomly across arbitrary regions of the image. However, existing methods typically model this task as a direct mapping from global features to gaze heatmaps, essentially treating it as a pixel-level regression problem. This approach fails to explicitly represent the gazed object as a distinct entity, making it difficult to produce stable and semantically consistent predictions in complex scenes. To address this, we propose a two-stage gaze estimation framework guided by object semantics, reformulating gaze target estimation as a hierarchical reasoning process. Our method incorporates object-level representations during feature encoding to align image features with discrete semantic entities, then introduces multi-scale feature fusion and geometric constraints from head pose and gaze direction for fine-grained localization and object-level discrimination. Extensive experiments on GazeFollow, VideoAttentionTarget, ChildPlay, and GOO-Real demonstrate that our method achieves AUC of 0.961, 0.948, 0.987, and 0.977 respectively, delivering strong performance across all benchmarks while maintaining a compact parameter size of 7.1M.

ECCV 2026

Spatial-TTT: Streaming Visual-based Spatial Intelligence with Test-Time Training

Fangfu Liu, Diankun Wu, Jiawei Chi, Yimo Cai, Yi-Hsin Hung, Xumin Yu, Hao Li, Han Hu, Yongming Rao, Yueqi Duan

Humans perceive and understand real-world spaces through a stream of visual observations. Therefore, the ability to streamingly maintain and update spatial evidence from potentially unbounded video streams is essential for spatial intelligence. The core challenge is not simply longer context windows but how spatial information is selected, organized, and retained over time. In this paper, we propose Spatial-TTT towards streaming visual-based spatial intelligence with test-time training (TTT), which adapts a subset of parameters (fast weights) to capture and organize spatial evidence over long-horizon scene videos. Specifically, we design a hybrid architecture and adopt large-chunk updates parallel with slidingwindow attention for efficient spatial video processing. To further promote spatial awareness, we introduce a spatial-predictive mechanism applied to TTT layers with 3D spatiotemporal convolution, which encourages the model to capture geometric correspondence and temporal continuity across frames. Beyond architecture design, we construct a dataset with dense 3D spatial descriptions, which guides the model to update its fast weights to memorize and organize global 3D spatial signals in a structured manner. Extensive experiments demonstrate that SpatialTTT improves long-horizon spatial understanding and achieves state-ofthe-art performance on video spatial benchmarks. Project page: https: //liuff19.github.io/Spatial-TTT.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.