paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

ECCV 2026

How Far Are Video Models from True Multimodal Reasoning?

Xiaotian Zhang, Jianhui Wei, Yuan Wang, Jie Tan, Yichen Li, Yan Zhang, Ziyi Chen, Daoan Zhang, Dezhi Yu, Wei Xu, Songtao Jiang, Zuozhu Liu

Despite remarkable progress toward general-purpose videomodels, a critical question remains unanswered: how far are these modelsfrom achieving true multimodal reasoning? Existing benchmarks fail toaddress this question rigorously, as they remain constrained by straight-forward task designs and fragmented evaluation metrics that neglectcomplex multimodal reasoning. To bridge this gap, we introduce CLVG-Bench, an evaluation framework designed to probe video models’ zero-shot reasoning capabilities via Context Learning in Video Generation.CLVG-Bench comprises more than 1,000 high-quality, manually anno-tated metadata across 6 categories and 47 subcategories, covering com-plex scenarios including physical simulation, logical reasoning, and inter-active contexts. To enable rigorous and scalable assessment, we furtherpropose an Adaptive Video Evaluator (AVE) that aligns with human ex-pert perception using minimal annotations, delivering interpretable tex-tual feedback across diverse video context tasks. Extensive experimentsreveal a striking answer to our central question: while state-of-the-art(SOTA) video models, such as Seedance 2.0, demonstrate competenceon certain understanding and reasoning subtasks, they fall substantiallyshort with logically grounded and interactive generation tasks (achiev-ing success rates < 25% and ∼ 0%, respectively), exposing multimodalreasoning and physical grounding as critical bottlenecks. By systemati-cally quantifying these limitations, the proposed method provides action-able feedbacks and a clear roadmap toward truly robust, general-purposevideo models. CLVG-Bench and code are released at here.

ECCV 2026

OBBSeg: Irregular Lesion Segmentation under Oriented Bounding Box Annotations

Jun Wei, Xinchang Liu, Yu Liu, Chuhua Yang, Shuhui Wang, Hui Huang

Pixel-level annotation remains a major bottleneck in medicalimage segmentation, making weak supervision an attractive yet under-constrained alternative. We propose OBBSeg, an intermediate super-vision paradigm guided by Oriented Bounding Boxes (OBBs) thatbridges the gap between full and weak supervision. By jointly encodingspatial extent and orientation, OBBs provide compact geometric super-vision that better aligns with elongated or anisotropic lesions, reducingthe ambiguity of coarse box annotations. To mitigate the inherent rect-angular bias of OBBs, we introduce a Mask-to-OBB loss, a differen-tiable formulation that enforces geometric consistency between predictedmasks and OBB regions. Furthermore, we incorporate prompt-drivensemantic guidance through two complementary modules—PAFE andDBFE—which enhance foreground representation and suppress back-ground interference. Extensive experiments on 13 datasets across 5imaging modalities show that OBBSeg not only outperforms existingweakly supervised methods but also achieves performance comparableto fully supervised approaches, demonstrating its potential for efficientand scalable medical image segmentation. The code is available athttps://github.com/StarLxc3/OBBSeg.

ECCV 2026

If It's Not Efficient, It's Not Usable: Real-Time OOD Detection with Latent De-Biasing and High-Quality Negative Samples

Yuchuan Li, Jae-Mo Kang, IL MIN KIM

On-device AI for resource-constrained systems, such as wear-ables and IoT sensors, requires models that are both highly efficientand reliable for mission-critical applications. Out-of-Distribution (OOD)detection is vital for ensuring this reliability, yet mainstream OOD re-search has focused on achieving state-of-the-art (SOTA) accuracy withoutregard for inference-time computational cost. This has led to powerful butcomplex methods (e.g., CLIP-based or supervised learning-based models)that are orders of magnitude too slow for real-world on-device deployment.We argue that the more meaningful benchmark for this domain is maxi-mizing accuracy within a practical, constrained inference computationalbudget. However, reducing inference complexity is typically achieved atthe expense of lower performance, exhibiting a trade-off between inferencespeed and accuracy. This paper breaks that trade-off. We introduce anovel framework that establishes an accuracy-SOTA under constrainedinference complexity. This is achieved through two key innovations: (i)a latent-space de-biasing technique that learns to remove pixel-intensitybias during training, drastically reducing inference complexity, and (ii) anintelligent negative sampling strategy that generates high-quality negativesamples to significantly boost performance. On both a real edge deviceand standard GPU infrastructure, extensive experiments reveal thatour method surpasses existing accuracy-SOTA OOD detection methodswithin a TinyML-appropriate inference budget.

ECCV 2026

ESTANet: Efficient Online Error Detection in Procedural Videos via Prediction Inconsistency

Shih-Po Lee, Reza Ghoddoosian, Faizan Siddiqui, Enna Sachdeva, Behzad Dariush

An efficient and accurate system for detecting errors in pro-cedural tasks is crucial for supporting human needs in daily life, asit can provide instant notifications and guide people to correct mis-takes. In this work, we study real-time online error detection in pro-cedural videos from a simple but overlooked perspective: the predictionbehavior of action detectors themselves. Instead of designing complexarchitectures or specialized supervision, we observe that action detec-tors naturally exhibit different prediction characteristics depending ontheir sensitivity to input dynamics and temporal context. We thereforepropose ESTANet (Error-Sensitive and Temporally-vArying Network),a lightweight framework that detects errors by exploiting inconsistenciesamong action predictions produced by a small set of action detectors.We construct standard and error-sensitive action detectors that behavesimilarly on correct executions but respond differently when errors occur.Meanwhile, detectors operating with different temporal contexts furtheramplify prediction inconsistencies when the procedure deviates from theintended sequence. During inference, we detect errors by aggregating mis-matches between standard and error-sensitive predictions through ma-jority voting to flag frames that contain errors. Extensive experimentson EgoPER, Assembly-101-O, and EPIC-Tent-O demonstrate that ES-TANet achieves state-of-the-art performance in online error detectionwhile maintaining real-time efficiency with a lightweight architecture.Our results highlight that leveraging the intrinsic properties of actiondetectors can yield a powerful and practical solution for online error de-tection without increasing architectural design complexity. Our code isavailable at: https://github.com/robert80203/ESTANet

ECCV 2026

AFFMAE: Scalable Vision Pre-Training for High-Resolution Microscopy Segmentation on Desktop Hardware

David Smerkous, ZiAn Wang, Behzad Najafian

Self-supervised pretraining has transformed computer visionby enabling data-efficient fine-tuning, yet high-resolution pretraining typ-ically requires server-scale infrastructure, limiting custom in-domain train-ing for many research laboratories. Masked Autoencoders (MAE) reducecomputation by encoding only visible tokens, but combining MAE withhierarchical downsampling architectures has remained structurally chal-lenging due to dense grid priors and mask-aware design compromises. Weintroduce AFFMAE, a masking-friendly hierarchical pretraining frame-work built on adaptive, off-grid token merging. AFFMAE removes dense-grid assumptions while preserving hierarchical scalability during pre-training and fine-tuning. To support this architecture, we developed nu-merically stable mixed-precision Triton kernels and a lightweight, point-based decoder that can be directly repurposed as a segmentation head.On high-resolution microscopy segmentation, AFFMAE matches MAEfinetuning performance on foot process width estimation with ViT back-bone at equal parameter counts while being 2x faster during pre-trainingand halving peak memory usage. Furthermore, AFFMAE achieves up to5x throughput speedups fine-tuning at the 1024px resolution, providinghigh-resolution model training on desktop hardware. Code available athttps://github.com/najafian-lab/affmae.

ECCV 2026

Adaptive Neural Dynamics for Robust Geometric LiDAR-Inertial State Estimation on UAVs

Sitian Peng, Rui Wang

This paper presents an enhanced LiDAR-inertial odometrysystem that addresses the challenge of spatiotemporal drift through anadaptive neural stochastic differential equation (SDE) framework. Tra-ditional IMU bias models based on random walk assumptions often failto capture the complex, high-order non-linearities inherent in low-costinertial sensors during high-dynamic UAV maneuvers. To bridge the gapbetween classical geometric constraints and deep learning, our approachintroduces three key innovations: an adaptive hybrid bias model that dy-namically integrates learned neural dynamics with linear motion priors;a neural SDE framework that explicitly models stochastic bias evolu-tion and heteroscedastic uncertainties; and a differentiable covariancepropagation mechanism that ensures consistency between learned repre-sentations and geometric state estimation within an invariant extendedKalman filter. Extensive evaluations on self-collected UAV datasets, val-idated by a Vicon motion capture system, demonstrate that our methodconsistently outperforms the baseline FAST-LIO2, providing superior ac-curacy in both position and orientation estimation. Furthermore, ourapproach offers reliable uncertainty estimates crucial for complex spatialperception and robust autonomous flight in challenging environments.

ECCV 2026

ZMIS-SAM: Segment Anything Model Enhanced With Wavelet Transform For Zooplankton Microscopy Image Instance Segmentation

Dekun Yuan, Zhongwei Li, Zheng Qiao, Jie Zhang

As primary consumers in the marine food chain, zooplanktonplay a crucial role in maintaining marine ecological balance. However, theSegment Anything Model (SAM) exhibits limited performance in micro-scopic image instance segmentation due to its lack of zooplankton-specificdomain knowledge. To address these challenges, we propose a novel in-stance segmentation model based on SAM and wavelet transform (ZMIS-SAM), effectively tackling issues such as inaccurate classification, dis-continuous segmentation of slender appendages, and incomplete bound-ary segmentation. Our framework incorporates three core innovations:ZM-ViT enhances SAM’s capability to model zooplankton morphologyand image intensity distributions through two lightweight adapters, theNeighboring Feature Aggregation Module (NFAM) improves continu-ous segmentation of semi-transparent slender appendages by integrat-ing general-purpose and domain-specific features, and the Wavelet-basedMulti-scale Multi-directional Feature Enhancement (WM2FE) moduleeffectively recovers high-frequency details to refine boundary segmenta-tion completeness. Extensive experiments demonstrate that ZMIS-SAMachieves state-of-the-art instance segmentation performance on the zoo-plankton dataset and exhibits strong generalization capability acrossmultiple public cross-domain datasets. Code: ZMIS-SAM.

ECCV 2026

StereoGS: Sparse-View 3D Gaussian Splatting via Stereo Priors

Wenhao Yuan, Yiyuan Ge, Deli Cai

3D Gaussian Splatting (3DGS) has achieved remarkable suc-cess in real-time novel view synthesis, yet it suffers from severe overfit-ting under sparse-view settings due to insufficient geometric constraints.While recent methods introduce monocular depth priors to mitigatethis, they inherently struggle with scale ambiguity and cross-view in-consistency, leading to defective geometry. In this paper, we proposeStereoGS, a novel sparse-view 3DGS framework that integrates stereopriors to establish reliable binocular consistency. Unlike scale-agnosticmonocular constraints, StereoGS introduces a Stereo Depth Regulariza-tion by constructing virtual stereo pairs during optimization and lever-aging a foundation stereo model to enforce absolute scale and binocular-consistent structures. To further suppress overfitting and eliminate re-dundant primitives, we design a Gradient-Aware Opacity Decay strat-egy that dynamically penalizes Gaussians based on their relative opacitygradient magnitudes. Combined with a Consistency-Aware Dense Ini-tialization using zero-shot multi-view depth estimation, StereoGS effec-tively anchors primitives to accurate scene surfaces. Extensive experi-ments on LLFF, DTU, Mip-NeRF360, and Blender datasets demonstratethat StereoGS achieves state-of-the-art performance in sparse-view set-tings without incurring any additional inference overhead. Project Page:https://stringerywh00.github.io/StereoGS_project_page/

ECCV 2026

STANCE: Controllable Video Generation for Structured Dynamics via Sparse-To-dense ANChored Encoding

ZhiFei Chen, Tianshuo Xu, Leyi Wu, Luozhou Wang, Dongyu Yan, Zihan You, Wenting Luo, Yingcong Chen

Video generation has recently made striking visual progress,but maintaining coherent object motion and interactions remains dif-ficult. We trace two practical bottlenecks: (i) human-provided motionhints (e.g., small 2D maps) often collapse to too few effective tokens afterencoding, weakening guidance; and (ii) Current architectures inevitablybias toward pixel-level texture reconstruction over complex structuralmotion, yielding videos that are photorealistic but physically incoherent.We present STANCE, an image-to-video framework that addresses bothissues with two simple components. First, we introduce Instance Cues,a pixel-aligned control signal that turns sparse, user-editable hints into adense 2.5D (camera-relative) motion field by averaging per-instance flowand augmenting with monocular depth over the instance mask. Thisreduces depth ambiguity compared to 2D drag/arrow inputs while re-maining easy to user. Second, we preserve the salience of these cuesin the latent token space with Dense RoPE, which tags a small setof initial-frame motion tokens with spatially-addressable rotary embed-dings. Finally, rather than treating appearance and structure as looselycoupled tasks, STANCE integrates an auxiliary stream directly withinthe same Diffusion Transformer (DiT). By sharing identical latent tokensand Dense RoPE, this auxiliary stream acts as a "geometry witness" thatheavily penalizes geometric drift.

ECCV 2026

SCORE: SubDistribution-aware Collaborative Knowledge Reinforcing for Cloth-Hybrid Lifelong Person Re-Identification

Kunlun Xu, Liangyu Ma, Jiangmeng Li, Xin Tong, Xiaode Liu, Yufei Guo, Jiahuan Zhou

Lifelong Person Re-Identification (LReID) aims to train aunified person retrieval model from a non-stationary data stream. Ex-isting LReID methods mainly focus on scenarios where the clothingof each person is consistent. Recently, the Cloth-Hybrid LReID (CH-LReID) where cloth-consistent and cloth-changing data alternately oc-cur, has emerged as a more practical and challenging scenario. Due to theconflict between clothing-relevant and clothing-irrelevant knowledge, thewell-known catastrophic forgetting problem is significantly exacerbatedin this task. To address this issue, we propose a SubDistribution-awareCOllaborative Knowledge REinforcing (SCORE) framework, where ourkey idea is explicitly modeling the intra-identity diversity to continu-ally consolidate distinct cloth-consistent and cloth-changing knowledge.Specifically, an Adaptive SubDistribution Modeling mechanism is devel-oped, where a set of distributional subprototypes is assigned to eachidentity to capture the intra-identity diversity, improving the compati-bility between cloth-consistent and cloth-changing knowledge. Then, aDistributional Knowledge Reinforcement scheme is introduced, wherethe knowledge of old distributional subprototypes is retained in the newones by a collaborative aligning mechanism. Extensive experiments showthat our SCORE achieves the state-of-the-art performance. Our code isavailable at https://github.com/zhoujiahuan1991/ECCV2026-SCORE.

ECCV 2026

Revisiting Avatar-As-Image: High-Fidelity Registration is All You Need

Margaret Kostyrko, Yuxuan Xue, Garvita Tiwari, Gerard Pons-Moll

The representation of 3D clothed humans as standardized 2DUV texture and displacement maps over an underlying body model haslong been studied. This compact representation is enticing as it enablespretrained image networks to process, generate, and edit 3D avatars, butis only useful if scans are accurately aligned and brought into correspon-dence via high-fidelity registration. This prerequisite has never been met,which we argue explains the limited quality of prior UV-based methodsfor clothed humans. Despite its significance, no public method produceshigh-fidelity SMPL(−X)+D registrations with UV texture from arbitraryclothed scans. We present AvaImg, a multi-stage optimization pipeline, toclose this gap: it enforces body-inside-clothing constraint via signed wind-ing numbers, made viable by a three-level efficiency cascade (∼10× run-time reduced, ∼95% storage saved), and recovers fine surface detail usingcoarse-to-fine displacement optimization. AvaImg outperforms all base-lines in body fitting, shape estimation, and surface registration acrosssix datasets, yielding textured registrations near-indistinguishable fromscans (PSNR=34.48dB). For validation of AvaImg’s Avatar-as-Image rep-resentation as imminently compatible with image foundation models, weauto-encode our UV maps via the frozen FLUX VAE. This achieves only0.76mm added Chamfer error relative to scan and shows that the result-ing maps lie within natural-image distributions, supporting the use of 2Dgenerative priors for 3D avatar generation. Code, data, and Singularitycontainers will be publicly released.

ECCV 2026

OmniMamba: Efficient and Unified Multimodal Understanding and Generation via State Space Models

Jialv Zou, Bencheng Liao, Qian Zhang, Wenyu Liu, Xinggang Wang

Recent advancements in unified multimodal understandingand visual generation (or multimodal generation) models have been hin-dered by their quadratic computational complexity and dependence onlarge-scale training data. We present OmniMamba, the first linear-architecture-based multimodal generation model that generates both text and imagesthrough a unified next-token prediction paradigm. The model fully lever-ages Mamba-2’s high computational and memory efficiency, extending itscapabilities from text generation to multimodal generation. To addressthe data inefficiency of existing unified models, we propose two key inno-vations: (1) decoupled vocabularies to guide modality-specific generation,and (2) task-specific LoRA for parameter-efficient adaptation. Further-more, we introduce a decoupled two-stage training strategy to mitigatedata imbalance between two tasks. Equipped with these techniques, Om-niMamba achieves competitive performance with JanusFlow while sur-passing Show-o across benchmarks, despite being trained on merely 2Mimage-text pairs, which is 1,000 times fewer than Show-o. Notably, Om-niMamba stands out with outstanding inference efficiency, achieving upto a 119.2× speedup and 63% GPU memory reduction for long-sequencegeneration compared to Transformer-based counterparts. Code and mod-els are released at https://github.com/hustvl/OmniMamba.

ECCV 2026

OmniHuman: A Large-scale Dataset and Benchmark for Human-Centric Video Generation

Lei Zhu, Xing Cai, Yingjie Chen, Li YiHeng, Binxin Yang, Hao Liu, Jie Chen, Chen Li, Jing Lyu

Recent advancements in audio-video joint generation mod-els have demonstrated impressive capabilities in content creation. How-ever, generating high-fidelity human-centric videos in complex, real-worldphysical scenes remains a significant challenge. We identify that the rootcause lies in the structural deficiencies of existing datasets across threedimensions: limited global scene and camera diversity, sparse interac-tion modeling, and insufficient individual attribute alignment. To bridgethese gaps, we present OmniHuman, a large-scale, multi-scene datasetdesigned for fine-grained human modeling. OmniHuman provides a hier-archical annotation covering video-level scenes, frame-level interactions,and individual-level attributes. To facilitate this, we develop a fully au-tomated pipeline for high-quality data collection and multi-modal an-notation. Complementary to the dataset, we establish the OmniHumanBenchmark (OHBench), a three-level evaluation system that providesa scientific diagnosis for human-centric audio-video synthesis. Crucially,OHBench introduces metrics that are highly consistent with human per-ception, filling the gaps in existing benchmarks by providing a compre-hensive diagnosis across global scenes, relational interactions, and in-dividual attributes. Experiments show that fine-tuning on only 20% ofOmniHuman significantly boosts performance, validating its effective-ness in advancing complex scenario modeling. The dataset is available athttps://huggingface.co/datasets/julia527/OmniHuman

ECCV 2026

Less is More: A Simple yet Effective Object-Centric Prompting Strategy for Vision-Language Reasoning in Autonomous Driving

Saiqian Peng, Duanfeng Chu, Liping Lu, Bing Shi

Large Vision-Language Models (VLMs) show immense po-tential in autonomous driving but struggle with precise spatial ground-ing, often hallucinating when forced to regress continuous image coordi-nates. To overcome this, we introduce a "Less is More" object-centricprompting strategy that fundamentally decouples spatial localizationfrom logical reasoning. By overlaying detected bounding boxes and dis-crete IDs directly onto multi-view images, the VLM only predicts dis-crete IDs, which are deterministically mapped back to precise coordi-nates via an offline lookup table. This model-agnostic approach trans-forms error-prone regression into reliable discrete matching without re-quiring structural modifications. Extensive evaluations on the DriveLM-nuScenes dataset demonstrate striking improvements in comprehensivedriving reasoning and spatial grounding. Notably, our method booststhe Accuracy and Match scores of the DriveLM-agent by absolute mar-gins of 8.26% and 10.39%, respectively. Furthermore, when applied tostronger models, it achieves impressive increases of 11.85% and 20.02%in Accuracy and Match for Qwen2.5-VL, as well as 8.75% and 19.88% forInternVL-3.5. These consistent gains definitively prove the strong plug-and-play generalizability of our paradigm across varying architectures.

ECCV 2026

DreamPartGen: Semantically Grounded Part-Level 3D Generation via Collaborative Latent Denoising

Tianjiao (Joey) Yu, Xinzhuo Li, Muntasir Wahed, Jerry Xiong, Yifan Shen, Ying Shen, Ismini Lourentzou

Understanding and generating 3D objects as compositionsof meaningful parts is fundamental to human perception and reasoning.However, most text-to-3D methods overlook the semantic and functionalstructure of parts. While recent part-aware approaches introduce de-composition, they remain largely geometry-focused, lacking semanticgrounding and failing to model how parts align with textual descriptionsor their inter-part relations. We propose DreamPartGen, a frameworkfor semantically grounded, part-aware text-to-3D generation. Dream-PartGen introduces Duplex Part Latents (DPLs) that jointly modeleach part’s geometry and appearance, and Relational Semantic Latents(RSLs) that capture inter-part dependencies derived from language. Asynchronized co-denoising process enforces mutual geometric and seman-tic consistency, enabling coherent, interpretable, and text-aligned 3Dsynthesis. Across multiple benchmarks, DreamPartGen delivers state-of-the-art performance in geometric fidelity (→60% Chamfer Distance) andtext–shape alignment (↑↓20% CLIP/ULIP), while producing composi-tionally consistent and controllable parts.PLAN Lab https://plan-lab.github.io/dreampartgen

ECCV 2026

When W4A4 Breaks Camouflaged Object Detection: Token-Group Dual-Constraint Activation Quantization

Tianqi Li, Wenyu Fang, Xin He, Xue Geng, Xu Cheng, Yun Liu

Camouflaged object detection (COD) segments objects thatintentionally blend with the background, so predictions depend on subtletexture and boundary cues. COD is often needed under tight on-devicememory and latency budgets, making low-bit inference highly desirable.However, COD is unusually hard to quantize aggressively. We studypost-training W4A4 quantization of Transformer-based COD and finda task-specific cliff: heavy-tailed background tokens dominate a sharedactivation range, inflating the step size and pushing weak-but-structuredboundary cues into the zero bin. This exposes a token-local bottleneck—remove cross-token range domination and bound the zero-bin mass under4-bit activations. To address this, we introduce COD-TDQ, a COD-aware Token-group Dual-constraint activation Quantization method.COD-TDQ addresses this token-local bottleneck with two coupled steps:Direct-Sum Token-Group (DSTG) assigns token-group scales to sup-press cross-token range domination, and Dual-Constraint Range Projection(DCRP) projects each token-group clip range to keep the step-to-dispersionratio and the zero-bin mass bounded. Across four COD benchmarksand two baseline models (CFRN and ESCNet), COD-TDQ consistentlyachieves an Sα score more than 0.12 higher than that of the state-of-the-art quantization method without retraining. The code is available athttps://github.com/MCG-NKU/nku-model-compre.

ECCV 2026

Parsimonious Flow Matching for Efficient Image Generation

Tianjiao Ding, Ziqing Xu, Benjamin Haeffele, Hongkang Li, Rene Vidal

Flow matching (FM) models generate data by learning a ve-locity field that transforms samples from a simple latent distribution,typically an isotropic Gaussian, to the data distribution. However, thegeometric mismatch between a unimodal, full-dimensional Gaussian anda multimodal, approximately low-dimensional data distribution leads toa complex velocity field that is costly to learn at training time andto integrate at inference time. In this paper, we propose ParsimoniousFlow Matching (PFM), which adopts a mixture of Gaussians (MoG)as the latent distribution whose geometry better aligns with the data.We identify key design choices that enable efficient FM, including anoptimal-transport data-latent coupling, MoG estimation via k-means,and eigenvalue regularization of the per-mode covariances. Theoretically,when both data and latent distributions are MoGs, we show that gra-dient descent for FM with affine velocity fields converges faster whenthe corresponding modes of the latent and data are more similar andthe per-mode covariances are well-conditioned. Further, for data thatfollows a separated MoG, we prove that replacing the isotropic Gaus-sian latent with a MoG in FM training accelerates gradient descentconvergence and lowers the initial loss. On CIFAR-10 and ImageNet(32 × 32), PFM achieves the same generation quality as the baselinein fewer training iterations, and produces higher-quality samples for thesame number of function evaluations at inference time. Code is available:https://github.com/tianjiaoding/pfm.

ECCV 2026

Towards Flexible, Natural, Efficient Interaction for Conversational Talking Face Generation

Baiqin Wang, Sen Chen, Jiankuo Zhao, Xiangyu Liu, Zhen Lei, Xiangyu Zhu

Conversational talking face generation has recently attractedincreasing attention, aiming to synthesize interactive talking videos wherecharacters speak, listen, and respond dynamically to each other. Thistask presents three core challenges: 1) Flexibility: enabling multi-rounddialogues with an arbitrary number of participants; 2) Naturalness: main-taining coherent motion and appropriate non-verbal feedback through-out the interaction; and 3) Efficiency: achieving real-time generation andlow computation overhead for long-term continuous online conversation.Despite recent advances, existing methods still fall short in balancingall three requirements. To bridge this gap, we introduce InterTalk, anovel and efficient framework designed for highly interactive conver-sational talking face generation. Built upon a motion-based architec-ture, InterTalk supports real-time conversation synthesis. Our methodachieves strong flexibility by explicitly modeling multi-round conversa-tional dynamics among each participant, eliminating constraints on theirnumbers. To enhance interactivity, we incorporate motion feedback frommultiple participants and introduce an iterative generation strategy formore natural behaviors. Besides, we disentangle motion into several fa-cial components, enabling targeted refinements for natural response suchas precise lip-sync and realistic eye-blinking. Finally, we construct a newmulti-person conversational dataset and enrich it with 3D face-baseddata augmentation. Extensive experiments demonstrate that InterTalkachieves superior interaction quality while maintaining real-time perfor-mance at 30 FPS.

ECCV 2026

Restoring Linguistic Grounding in VLA Models via Train-Free Attention Recalibration

Ninghao Zhang, Bin Zhu, Shijie Zhou, Jingjing Chen

Vision-Language-Action (VLA) models enable robots to per-form manipulation tasks directly from natural language instructions andare increasingly viewed as a foundation for generalist robotic policies.However, their reliability under Out-Of-Distribution (OOD) instructionsremains underexplored. In this paper, we reveal a critical failure modein which VLA policies continue executing visually plausible actions evenwhen the language instruction contradicts the scene. We refer to this phe-nomenon as linguistic blindness, where VLA policies prioritize visualpriors over instruction semantics during action generation. To systemati-cally analyze this issue, we introduce ICBench, a diagnostic benchmarkconstructed from the LIBERO dataset that probes language–action cou-pling by injecting controlled OOD instruction contradictions while keep-ing the visual environment unchanged. Evaluations on three represen-tative VLA architectures, including π0 , π0.5 , and OpenVLA-OFT, showthat these models frequently succeed at tasks despite logically impossi-ble instructions, revealing a strong visual bias in action generation. Tomitigate this issue, we propose Instruction-Guided Attention Re-calibration (IGAR), a train-free inference-time mechanism that rebal-ances attention distributions to restore the influence of language instruc-tions. IGAR operates without retraining or architectural modificationand can be directly applied to existing VLA models. Experiments across30 LIBERO tasks demonstrate that IGAR substantially reduces erro-neous execution under OOD contradictory instructions while preservingbaseline task performance. We additionally validate the approach on areal Franka robotic arm, where IGAR effectively prevents manipulationtriggered by inconsistent instructions.

ECCV 2026

StratoSplat: Taming Layered Regularities for Sparse Aerial 3D Gaussian Splatting

Zihan Gao, Lingling Li, Licheng Jiao, Fang Liu, wenping ma, Yuwei Guo, Shuyuan Yang

3D Gaussian Splatting has revolutionized novel view synthe-sis with real-time rendering and photorealistic quality. However, standard3DGS methods fail in aerial scenes due to sparse anisotropic UAV captures.Two critical issues emerge: incomplete triangulated points in texturelessregions that destabilizes Gaussian initialization and view-aligned degen-eracies where Gaussians overfit limited observations rather than respecttrue geometry. We present StratoSplat, a framework that exploits the in-herent layered structure of aerial scenes for robust sparse-view 3DGS. Ourkey insight is that buildings, terrain, and roads naturally stratify alonggravity, providing strong geometric priors to regularize this problem. Weaddress point sparsity through layered memory guided initialization thatenforces multi-view consistency. To prevent optimization degeneracies, weintroduce neural multi-plane Gaussians where virtual primitives anchor tolearned parallel planes for geometric regularization. Extensive experimentsdemonstrate state-of-the-art performance, surpassing the best baseline byover 3dB. Code available at https://github.com/keloee/StratoSplat.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.