paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

ECCV 2026

Learning from Primitive: Probing Visual Reasoning of LVLMs via Counting

Liwei Che, Zhiyu Xue, Yihao Quan, Benlin Liu, Zeru Shi, Ruixiang Tang, Ranjay Krishna, Vladimir Pavlovic

Counting serves as a simple but powerful test of a Large Vision-Language Model’s (LVLM) reasoning; it forces the model to identify each individual object and then add them all up. In this study, we investigate how LVLMs implement counting using controlled synthetic and real-world benchmarks, combined with mechanistic analyses. Our results show that LVLMs display a human-like counting behavior, with precise performance on small numerosities and noisy estimation for larger quantities. We introduce two novel interpretability methods, Visual Activation Patching and HeadLens, and use them to uncover a structured “counting circuit” that is largely shared across a variety of visual reasoning tasks. Building on these insights, we propose a lightweight intervention strategy that exploits simple and abundantly available synthetic images to fine-tune arbitrary pretrained LVLMs exclusively on counting. Despite the narrow scope of this fine-tuning, the intervention not only enhances counting accuracy on in-distribution synthetic data, but also yields an average improvement of +8.36% on out-of-distribution counting benchmarks and an average gain of +1.54% on complex, general visual reasoning tasks for Qwen2.5-VL. These findings highlight the central, influential role of counting in visual reasoning and suggest a potential pathway for improving overall visual reasoning capabilities through targeted enhancement of counting mechanisms.

ECCV 2026

KISS-GS: 3D Gaussian Splatting Compression Kept Simple

Wieland Morgenstern, Friedrich Elias Branschke, Florian Fleischmann, Adrian Szatmari, Paul Schlack, Florian Barthel, Anna Hilsmann, Peter Eisert

Scene reconstruction with 3D Gaussian Splatting (3DGS) has become common, however deployment remains painful as the uncompressed file sizes can be massive. Current 3DGS compression systems combine multiple strategies for file size reduction, which can obscure where gains come from and limit component reuse across training pipelines. To make the gains more transparent, we propose KISS-GS, a modular compression pipeline named after the principle of keeping things simple, designed to decouple compression entirely from training. Given a 3DGS scene reconstructed with vanilla 3DGS, we are able to reduce it through compaction by 15.7x using a combination of state-ofthe-art pruning schemes. Then we encode it into an image-based format designed for simple, ubiquitous decoding. With the SOG-XT format, we propose a novel extension to Self-Organizing Gaussians with two main contributions: (i) Self-organizing 2D Codebooks and (ii) Parallel Representative Assignment Smoothing (PRAS), which leverages the symmetry of quaternion and scale parameterizations to produce 2D attribute grids more amenable to encoding. This encoding reduces scene size by 6.6x. We show that optional encoding-aware fine-tuning yields a further 2.2x. Across standard 3DGS benchmarks, our simple and modular approach thus achieves a total of 85× to 319× reductions in the size of the scene over uncompressed vanilla 3DGS, setting new benchmarks for real-world scenes and surpassing tightly integrated methods in rate-distortion. Decoding relies solely on web-native image formats, and the modular design makes each stage easy to combine with future advances in reconstruction and compaction. Code and project page: https://fraunhoferhhi.github.io/KISS-GS/

ECCV 2026

IREU: Identity-Related Encoder-Only Unlearning for Customized Portrait Generation

Chaoyi Shi, Shanshan Zhang, Jian Yang

Customized Portrait Generation (CPG) technologies havebeen widely used to generate high-fidelity person images given an inputimage indicating the identity and a text prompt indicating the requirededits. Yet these methods pose significant privacy risks by spreading fakevisual information. Against such risks, each public generator should beable to suppress its generation ability for a particular person when re-quested. Therefore, in this work we investigate the identity unlearningproblem for CPG. Since there are no previous methods in this field, wepropose a simple baseline that updates the image encoder by minimizingidentity similarity between generated and input images for target iden-tities to be unlearned, while maximizing it for identities to be retained.However, we find such a global perturbation in the feature space harmsthe fidelity of generated images for other identities to be retained. Tosolve this problem, we propose a novel method IREU, which first locatesidentity-related features in an offline manner and then only performsfeature perturbations on them. The experimental results show that ourproposed method IREU achieves better identity unlearning performancefor target identities to be unlearned, and also keeps high fidelity for otheridentities to be retained. In addition, our unlearned image encoder isgeneralizable across different generators with the same encoder withoutfine-tuning, which is friendly for deployment in practice.

ECCV 2026

Interference-Aware Continual Vision–Language Learning via Instance-Level Expert Routing

Zhang Changyuan, Tianxiang Xu, Canran Xiao, Fei Shen

Continual vision–language learning is increasingly required in real deploymentswhere data, skills, and objectives drift, yet models still struggle to learn new tasks withouterasing old ones and to act reliably in task-agnostic settings with limited memory. We ad-dress this gap by turning cross-task conflict into an instance-level signal that drives how themodel adapts over time. We propose an interference-aware adapter routing framework thatestimates, for each sample, a metric-consistent projection energy to quantify potential inter-ference, routes to the least interfering experts with capacity balancing, enlarges inter-expertseparation via principal-angle packing, and grows LoRA rank only along the residual princi-pal direction when current experts are insufficient. Across classification, structured conceptmatching, generative VQA, and retrieval, our approach improves over strong baselines. Anal-yses show that the conflict signal reliably predicts downstream forgetting, packing widensprincipal angles over time, and rank growth is sparse yet beneficial—especially in mid–highlayers.

ECCV 2026

ICDepth: Taming Video Diffusion Models for Video Depth Estimation via In-Context Conditioning

Xuanhua He, Jiaxin Xie, Mingzhe Zheng, Qifeng Chen

Monocular video depth estimation requires temporal con-sistency, geometric accuracy, and generalization across diverse scenar-ios—yet existing methods struggle to achieve all three simultaneously.Discriminative models excel at per-frame accuracy but suffer from tem-poral drift due to limited context windows, while generative methodsimprove consistency and generalization at the cost of extensive train-ing data (10M+ samples) and lack of geometric precision. In responseto these issues, we introduce ICDepth, a framework that adapts pre-trained text-to-video diffusion transformers for video depth estimationvia In-Context Conditioning (ICC), leveraging their rich spatial-temporalpriors. To address key challenges in transferring ICC from generationto dense prediction, we propose: (1) SAND-Attention, which ensuresprecise spatial-temporal alignment via shared RoPE and enforces uni-directional attention to prevent noise contamination; (2) SRFM, whichinjects DINOv2 semantic and resolution priors to enhance geometric pre-cision. ICDepth achieves state-of-the-art results on multiple benchmarkswith remarkable data efficiency, trained on only 0.8M frames (6–13×less than competing generative methods), while demonstrating strongzero-shot generalization to diverse domains.

ECCV 2026

Holo-Captioning: A Comprehensive Textual View of 3D Scenes

Kun-Yu Lin, Chengke Bu, Zhenguo Li, Kai Han

This work introduces holo-captioning, a novel task that strivesto seek the text equivalent of 3D scenes. As the initial step, we formu-late holo-captioning as generating a structured textual description thatcomprehensively depicts all entities within a 3D scene—including theirsemantic tags, spatial locations, attributes, and inter-entity relations. Totackle this challenging task, we first develop an effective captioning en-gine to produce detailed descriptions of individual entity instances andinstance pairs, and contribute a large-scale benchmark comprising over15K scenes for training and evaluation. Building upon this foundation,we propose HoloScribe, a novel model that features an instance-awaredecoupled pipeline for generating structured holo-captions, and furtherincorporates anchor-aware instance linking to identify relational instancepairs. Additionally, we propose a comprehensive evaluation metric namedHoloScore, and provide a human-curated test set to ensure reliable modelassessment. Experimental results demonstrate that HoloScribe signifi-cantly outperforms state-of-the-art 3D dense captioners and 3D LLMgeneralists, underscoring the effectiveness of our approach. Project page:https://visual-ai.github.io/holocap/

ECCV 2026

GridFlow: Structured Latent Flow for Seamless City-Scale 3D Point Cloud Generation

Xinyu Wang, Muhammad Ibrahim, Atif Mansoor, Ajmal Mian

Generating realistic 3D city environments from remote sens-ing data is important for simulation, urban planning, and mixed reality,yet existing point cloud generation methods are limited to single ob-jects or bounded indoor scenes and cannot handle the scale, seamlesstiling, and partial observability challenges of city-scale generation. Wepresent GridFlow, a multi-stage framework that generates dense, col-ored point clouds (105 points per 150m×150m tile) at city scale, con-ditioned on satellite imagery, semantic segmentation maps, and digitalsurface models (DSM). A Grid-Aligned VAE encodes each tile into atopology-preserving latent grid where tokens correspond to fixed spatialregions, enabling spatially coherent multi-modal conditioning and com-pact latent-space edge consistency that implicitly aligns thousands ofboundary points for seamless cross-tile generation. A conditional rectifiedflow model synthesizes geometry latents from the fused multi-modal con-ditions, and an orientation-aware diffusion colorizer separately handlessatellite-visible horizontal surfaces and occluded vertical façades. To sup-port standardized evaluation, we build on public 3D data sources to in-troduce City3D-MultiGen, a benchmark of 163K densely annotated tilesfrom Melbourne and London with aligned point clouds, satellite images,semantic maps, and elevation data. Experiments show that GridFlowoutperforms adapted point cloud generation baselines across all geometrymetrics and produces visually coherent colored point clouds with seam-less boundaries over arbitrarily large urban extents. Our benchmark de-tails are available at https://huggingface.co/datasets/e32/City3D-MultiGen.

ECCV 2026

GenHOI: Generalized Hand-Object Pose Estimation with Occlusion Awareness

Hui Yang, Wei Sun, Jian Liu, Jian Xiao, Tao Xie, Hossein Rahmani, Ajmal Mian, Nicu Sebe, Gim Hee Lee

Generalized 3D hand-object pose estimation from a singleRGB image remains challenging due to the large variations in objectappearances and interaction patterns, especially under heavy occlusion.We propose GenHOI, a framework for generalized hand-object pose esti-mation with occlusion awareness. GenHOI integrates hierarchical seman-tic knowledge with hand priors to enhance model generalization underchallenging occlusion conditions. Specifically, we introduce a hierarchicalsemantic prompt that encodes object states, hand configurations, andinteraction patterns via textual descriptions. This enables the model tolearn abstract high-level representations of hand-object interactions forgeneralization to unseen objects and novel interactions while compen-sating for missing or ambiguous visual cues. To enable robust occlu-sion reasoning, we adopt a multi-modal masked modeling strategy overRGB images, predicted point clouds, and textual descriptions. Moreover,we leverage hand priors as stable spatial references to extract implicitinteraction constraints. This allows reliable pose inference even undersignificant variations in object shapes and interaction patterns. Exten-sive experiments on the challenging DexYCB and HO3Dv2 benchmarksdemonstrate that our method achieves state-of-the-art performance inhand-object pose estimation.

ECCV 2026

CLARITY: Medical World Model for Guiding Treatment Decisions by Simulating Context-Aware Disease Trajectories in Latent Space

Tianxingjian Ding, Yuanhao Zou, Chen Chen, Shah Mubarak, Yu Tian

Clinical decision-making in oncology requires forecasting howdisease evolves under treatment, yet most AI systems remain static pre-dictors that cannot model longitudinal, treatment-conditioned progres-sion. Although generative and world models have demonstrated strongcapabilities in general domains, their adaptation to medicine remainslimited and insufficient for capturing complex, treatment-induced phys-iological dynamics across temporal scales. To address these gaps, weintroduce CLARITY, a medical world model that enables counterfac-tual simulation of treatment-conditioned disease trajectories for clini-cal decision-making. By jointly encoding imaging-derived latent states,temporal intervals that capture irregular follow-ups, and patient-specificclinical context, CLARITY learns smooth and interpretable representa-tions of disease progression, allowing the model to simulate how alter-native treatments reshape future disease dynamics. Because treatmentoptimization is inherently sequential and uncertain, requiring evaluationof long-term outcomes across multiple possible interventions, we furtherpropose an entropy-regularized, computationally efficient long-horizonprediction-to-decision framework that plans treatment strategies overimagined disease trajectories and iteratively refines therapy proposalsthrough survival-aware feedback, forming a closed-loop simulation-to-decision framework for treatment planning. CLARITY achieves state-of-the-art performance in treatment planning and survival predictionacross three cancer datasets, including two brain tumor cohorts (MU-Glioma-Post and zero-shot on UCSF-ALPTDG) and one breast can-cer dataset (ISPY-2), demonstrating strong generalization across can-cer types while consistently outperforming prior generative methods andmedical-domain large language model baselines.

ECCV 2026

FreeGen: Feed-Forward Reconstruction–Generation Co-Training for Free-Viewpoint Driving Scene Synthesis

Shijie Chen, Peixi Peng

Closed-loop simulation for autonomous driving requires synthesizing free-viewpoint driving scenes. However, existing datasets and generative pipelines rarely provide consistent off-trajectory observations, limiting large-scale evaluation and training. While recent generative models demonstrate strong visual realism, they struggle to simultaneously achieve interpolation consistency and extrapolation realism without perscene optimization. To address this, we propose FreeGen, a feed-forward reconstruction-generation co-training framework for free-viewpoint driving scene synthesis. The reconstruction model provides stable geometric representations to ensure interpolation consistency, while the generation model performs geometry-aware enhancement to improve realism at unseen viewpoints. Through co-training, generative priors are distilled into the reconstruction model to improve off-trajectory rendering, and the refined geometry in turn offers stronger structural guidance for generation. Experiments demonstrate that FreeGen achieves state-of-the-art performance for free-viewpoint driving scene synthesis.

ECCV 2026

Exclusivity-Guided Mask Learning for Semi-Supervised Crowd Instance Segmentation and Counting

Jiyang Huang, Hongru Chen, Wei Lin, Jia Wan, Antoni Chan

Semi-supervised crowd analysis is a prominent area of re-search, as unlabeled data are typically abundant and inexpensive toobtain. However, traditional point-based annotations constrain perfor-mance because individual regions are inherently ambiguous, and con-sequently, learning fine-grained structural semantics from sparse anno-tations remains an unresolved challenge. In this paper, we first pro-pose an Exclusion-Constrained Dual-Prompt SAM (EDP-SAM), basedon our Nearest Neighbor Exclusion Circle (NNEC) constraint, to gener-ate mask supervision for current datasets. With the aim of segmentingindividuals in dense scenes, we then propose Exclusivity-Guided MaskLearning (XMask), which enforces spatial separation through a discrim-inative mask objective. Gaussian smoothing and a differentiable centersampling strategy are utilized to improve feature continuity and train-ing stability. Building on XMask, we present a semi-supervised crowdcounting framework that uses instance mask priors as pseudo-labels,which contain richer shape information than traditional point cues. Ex-tensive experiments on the ShanghaiTech A, UCF-QNRF, and JHU++datasets (using 5%, 10%, and 40% labeled data) verify that our end-to-end model achieves state-of-the-art semi-supervised segmentation andcounting performance, effectively bridging the gap between counting andinstance segmentation within a unified framework. Code can be found athttps://github.com/JoyceeH0127/ECCV2026XMask.

ECCV 2026

Dual-Generalization-aware Minimization for Continual Fine-Tuning of Vision-Language Models

Yanan Chen, Tieliang Gong, Yanle Lyu, yuanhong zhang, Weizhan Zhang

Continual fine-tuning of large vision–language models (VLMs)such as CLIP is crucial for adapting foundation models to evolving real-world distributions. However, such adaptation often disrupts pretrainedrepresentations and severely harms zero-shot generalization. Despite re-cent progress, existing methods overlook the underlying optimization ge-ometry, failing to anchor convergence within regions robust to distribu-tion shift. While recent studies suggest that flatter loss minima improvegeneralization, we empirically demonstrate that seeking local flatnesson a single task is insufficient to bridge the distributional gap betweenpre-trained and task-specific representations. Motivated by this obser-vation, we propose Dual-Generalization-aware Minimization (DGM), aplug-and-play optimization framework for continual adaptation of VLMs.DGM extends flatness-aware training by redefining the perturbation ob-jective to jointly promote pre-trained knowledge preservation and task-specific robustness. By incorporating these objectives into the inner max-imization step, DGM steers optimization toward solutions that remainstable under both zero-shot evaluation and task adaptation. Extensiveexperiments demonstrate that DGM effectively maintains zero-shot gen-eralization while achieving strong performance on sequential tasks.

ECCV 2026

Difficulty-Conditioned Attribute-Specific Restoration for Low-Light Image Enhancement

Mingzhu Zhang, Shuang Li, Jiaxu Leng, Long Sun, Can Zhang, Miaoqing Wang, Xinbo Gao

Low-light image enhancement (LLIE) aims to restore visi-bility and structural details from severely degraded images under insuf-ficient illumination. However, real-world low-light images often exhibitspatially heterogeneous degradations, where mildly under-exposed areasare easy to recover while severely dark, noisy, or color-unstable regionsremain challenging. Existing LLIE methods typically apply a globallyuniform enhancement strategy, which often leads to suboptimal trade-offs such as over-smoothing textures in hard regions or over-enhancementin easier areas. To address this, we propose a Difficulty-ConditionedAttribute-Specific Restoration (DCASR) framework that explicitly mod-els restoration difficulty and allocates enhancement capacity accordingly.DCASR incorporates a difficulty-aware enhancement network with twoprior-driven modulators: a Global Intensity Modulator (GIM) for co-herent illumination correction and a Structural Refinement Modulator(SRM) for detail recovery, together with an adaptive weighting schemethat emphasizes hard regions during optimization. Since residual-deriveddifficulty priors are unavailable at test time, we adopt a teacher–studentscheme in which a privileged teacher derives difficulty priors from resid-ual cues only during training, while a latent conditional diffusion studentgenerates aligned priors conditioned on a single low-light input at infer-ence. Experiments demonstrate that DCASR consistently outperformsstate-of-the-art methods on multiple benchmarks. Code will be madepublicly available at github.com/mingzhuzhang1/DCASR-LLIE.

ECCV 2026

DeCo: Zero-Shot Anomaly Generation through Decoupling and Recoupling

Shilei Zeng, Xurui Li, Yaohan Tang, Yu Zhou

Industrial anomaly inspection is severely hindered by thescarcity of real anomalous data. Zero-shot industrial anomaly genera-tion addresses this by generating anomalies on specific products withoutrequiring any of their real anomalous images. However, existing meth-ods suffer from two critical limitations, i.e., inaccurate anomaly informa-tion acquisition and uncontrolled anomaly-product fusion. To overcomethese challenges, we propose DeCo, which decouples the anomaly struc-ture from its source product, and explicitly recouples it with the nor-mal textures of the target product. During anomaly information acquisi-tion, Dual-Routing Flow (DR-Flow) binds the texture-invariant anomalystructure to an abnormal token, while a parallel constraint, Product-Invariant Flow (PI-Flow), prevents the abnormal token from binding thesource product. During anomaly-product fusion, we propose a hybrid in-jection to recouple the acquired anomaly structure with the target prod-uct, and Product Compatibility Correction (PCC) to compensate for theincompatibility between the acquired anomaly structure and the product.Extensive experiments demonstrate that DeCo establishes a new state-of-the-art. Training downstream detection models on our generated datayields massive pixel AP improvements of 5.1% on MVTec AD and 8.2%on VisA. Code is available at https://github.com/HUST-SLOW/DeCo.

ECCV 2026

DARL: Efficient Document-to-Markup Generation via Look-Ahead Diffusion Trajectory Sampling

Wentao Yang, Yongxin Shi, Rui Tang, Peirong Zhang, Shihang Wu, Huiguo He, Zheng Huang, Dezhi Peng, Minghui Liao, Lianwen Jin

While autoregressive (AR) Multimodal Large Language Mod-els (MLLMs) excel at complex document-to-markup generation, theirsequential decoding causes severe latency. Conversely, existing paralleland diffusion-based methods accelerate inference but often struggle tomaintain strict structural dependencies, resulting in significantly higherparsing errors. To bridge this gap, we propose DARL (Diffusion AutoRe-gression with Look-ahead), a novel hybrid decoding framework. DARLintroduces a Sliding Diffusion Block (SDB) that maintains a verifiedAR prefix for strict syntactic correctness while speculatively generat-ing a localized window of future tokens in parallel. To optimize thesediscrete diffusion trajectories, we introduce Online Monte Carlo Tra-jectory Generation (OMTG) and Diffusion Trajectory Preference Opti-mization (DTPO). OMTG dynamically samples candidate paths basedon the model’s real-time state, effectively mitigating the exposure biasinherent in static heuristics. DTPO integrates immediate and look-aheadrewards to optimize current accuracy and facilitate future correctness,ensuring stable and rapid convergence of the predictive trajectory space.Experiments on OmniDocBench-1.5 and olmOCR-Bench demonstratethat DARL establishes a new Pareto frontier in document parsing. Itachieves up to a 2.3× speedup while matching or exceeding the state-of-the-art accuracy of AR baselines. The code and model are available athttps://github.com/SCUT-DLVCLab/DARL.

ECCV 2026

CortexVideo: A Semantic-Spatial Dual-Anchor Framework for High-Fidelity fMRI-to-Video Reconstruction

Xiaoquan Shen, Jiaxuan Chen, Jiajun Li, Gang Pan

Recent advances in neural decoding have enabled the direct reconstruction of visual stimuli from non-invasive brain signals. However, reconstructing continuous video streams remains highly challenging due to the continuous spatial and semantic transformations inherent in dynamic scenes. Most methods rely on a "reconstruct-thendescribe" paradigm, where captions are generated from reconstructed video keyframes, which are highly prone to semantic drift. To overcome these challenges, we propose a novel decoding framework, CortexVideo, to reconstruct video from functional magnetic resonance imaging (fMRI). Inspired by the visual dual-stream hypothesis, CortexVideo introduces dual-guidance decoding strategy. Specifically, we leverage subject-adaptive semantic information to guide video reconstruction, while concurrently integrating keyframe perceptual weights. Experiments demonstrate that CortexVideo provides a cognitive enhancement to existing hierarchical models along two critical dimensions: video semantic understanding and spatial localization. The reconstructed videos show greater fidelity to the visual stimuli in both semantic and spatial aspects, achieving state-ofthe-art results.

ECCV 2026

CoIn: Comprehensive 2D-3D Inpainting with Gaussian Splatting Guidance

Hana Kim, Minje Kim, Tae-Kyun Kim

3D scene inpainting is essential for reconstructing areas cor-rupted by occlusions or limited viewpoints. While recent methods lever-age Gaussian Splatting (GS) for efficient 3D editing, they often de-pend on precise multi-view segmentation masks and are inherently con-strained to object removal tasks. We propose CoIn, a novel frameworkthat bridges 2D inpainting models and 3DGS through a multi-stage con-sistency pipeline. Our approach first generates initial inpainted imagesusing a diffusion model, enabling the use of arbitrary-shaped masks anddiverse tasks like object insertion. We then introduce Reference Adap-tive GS with Feature Attention to reconstruct a coarse 3D scene byadaptively weighing towards a reference view (2D → 3D). This 3D rep-resentation provides geometric guidance to the diffusion process via GS-based Reference Feature Warping, ensuring multi-view consistency (3D→ 2D). Finally, a Texture-Enhancing Discriminator refines the 3D sceneto achieve high photometric realism (2D → 3D). Experiments show thatCoIn, effectively leveraging bidirectional information flow, achieves state-of-the-art performance and effectively handles both object removal andobject insertion with flexible mask input.

ECCV 2026

CLIP-AUTT: Test-Time Personalization with Action Unit Prompting for Fine-Grained Video Emotion Recognition

Muhammad Osama Zeeshan, Masoumeh Sharafi, Benoît Savary, Alessandro Lameiras Koerich, Marco Pedersoli, Eric Granger

Personalization in emotion recognition (ER) is essential foran accurate interpretation of subtle and subject-specific expressive pat-terns. Recent advances in vision–language models (VLMs) such as CLIPdemonstrate strong potential for leveraging joint image–text represen-tations in ER. However, CLIP-based methods either depend on CLIP’scontrastive pretraining or on LLMs to generate descriptive text prompts,which are noisy, computationally expensive, and fail to capture fine-grained expressions, leading to degraded performance. In this work, weleverage Action Units (AUs) as structured textual prompts within CLIPto model fine-grained facial expressions. AUs encode the subtle muscleactivations underlying expressions, providing localized and interpretablesemantic cues for more robust ER. We introduce CLIP-AU, a lightweightAU–guided temporal learning method that integrates interpretable AUsemantics into CLIP. It learns generic, subject-agnostic representationsby aligning AU prompts with facial dynamics, enabling fine-grained ERwithout CLIP fine-tuning or LLM-generated text supervision. AlthoughCLIP-AU models fine-grained AU semantics, it does not adapt to subject-specific variability in subtle expressions. To address this limitation, wepropose CLIP-AUTT, a video-based test-time personalization method thatdynamically adapts AU prompts to videos from unseen subjects. By com-bining entropy-guided temporal window selection with prompt tuning,CLIP-AUTT enables subject-specific adaptation while preserving tempo-ral consistency. Our experiments on three challenging video-based subtleER datasets — BioVid, StressID, and BAH — indicate that CLIP-AUand CLIP-AUTT outperform state-of-the-art CLIP-based FER and TTAmethods. Code: https://github.com/osamazeeshan/CLIP-AUTT.

ECCV 2026

Causal Yet Future-Aware: Dual-Path Temporal Modeling for Online Action Segmentation

Zhichao Zheng, Peirong Ma, Ying Zhou, Li Kong, Junsheng Zhou

Temporal Action Segmentation (TAS) assigns action labelsto each frame by modeling the temporal structure of actions. While of-fline TAS leverages full temporal context, online TAS cannot directlyaccess future information during inference. Existing online TAS methodsmainly focus on modeling historical dependencies, but temporal ambigu-ity still arises due to the incomplete temporal dependencies, leading tofragmented predictions. To tackle this challenge, we propose Dual-PathTemporal Modeling (DPTM) for online TAS, a two-stage framework thatexplicitly models and transfers the future dependencies that are inacces-sible during online inference. In the first stage, we adopt an offline-styletraining scheme where bidirectional dependencies are decomposed intounidirectional historical and future components via a temporal decom-position strategy. In the second stage, we employ a predictive distillationstrategy that uses recent historical frames to predict the separated futuredependencies without violating causality. These designs enable DPTMto leverage more complete temporal context while remaining suitable foronline inference. Extensive experiments demonstrate that DPTM signifi-cantly improves temporal coherence in segmentation results and achievesstate-of-the-art performance, demonstrating the effectiveness of modelingdependencies beyond historical frames for reducing the temporal ambi-guity. The code is available at: https://github.com/zczheng777/DPTM/.

ECCV 2026

Benchmarking MLLMs on Mistake Recognition and Explanation in Single-Step Components of Cooking

Shun Takashige, Atsushi Hashimoto, Shin'ichi Satoh

While mistake understanding in procedural video is advanc-ing toward increasingly complex multi-step tasks and diverse mistaketypes, it remains unclear whether current MLLMs can even recognize andexplain mistakes at individual step levels. This study proposes CookingMistake Recognition and Explanation (Cook-MRE), a dataset for assess-ing MLLMs in-depth performance in understanding cooking mistakeswithin each single-step. The evaluation uses an LLM-based approachalong two aspects: recognition (mistake presence and type classification)and explanation (error correction). By focusing on mistakes in the formof deviations from instructions, Cook-MRE reuses an existing cookingvideo dataset and synthesizes mistake samples through modification oftext instructions. It covers four basic operation elements and includes dif-ficulty labels based on visual distinguishability. Evaluation of 13 MLLMsshowed that our comprehensive metric, Full Pass Accuracy, was sub-stantially lower than mistake presence accuracy alone, indicating thedifficulty of explaining mistakes in detail even for single steps. Analysis byutilizing our type and difficulty labels showed notably low performanceon visually challenging problems, and scaling model size yields limitedimprovement on them. The dataset and code are publicly available athttps://github.com/Shun-Takashige/cook-mre-benchmark.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.