paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

ECCV 2026

ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Driving

Zihao Sheng, Xin Ye, Jingru Luo, Sikai Chen, Liu Ren

End-to-end autonomous driving models based on Vision-Language-Action (VLA) architectures have shown promising results bylearning driving policies through behavior cloning on expert demonstra-tions. However, imitation learning inherently limits the model to repli-cating observed behaviors without exploring diverse driving strategies,leaving it brittle in novel or out-of-distribution scenarios. Reinforcementlearning (RL) offers a natural remedy by enabling policy explorationbeyond the expert distribution. Yet VLA models, typically trained onoffline datasets, lack directly observable state transitions, necessitating alearned world model to anticipate action consequences. In this work, wepropose a unified understanding-and-generation framework that lever-ages world modeling to simultaneously enable meaningful explorationand provide dense supervision. Specifically, we augment trajectory pre-diction with future RGB and depth image generation as dense worldmodeling objectives, requiring the model to learn fine-grained visual andgeometric representations that substantially enrich the planning back-bone. Beyond serving as a supervisory signal, the world model furtheracts as a source of intrinsic reward for policy exploration: its imageprediction uncertainty naturally measures a trajectory’s novelty rela-tive to the training distribution, where high uncertainty indicates out-of-distribution scenarios that, if safe, represent valuable learning op-portunities. We incorporate this exploration signal into a safety-gatedreward and optimize the policy via Group Relative Policy Optimiza-tion (GRPO). Experiments on the NAVSIM and nuScenes benchmarksdemonstrate the effectiveness of our approach, achieving a state-of-the-art PDMS score of 93.7 and an EPDMS of 88.8 on NAVSIM. The codeis available at https://zihaosheng.github.io/ExploreVLA/.

ECCV 2026

Spectral Prior for Reducing Exposure Bias in Diffusion Models

Yuya Kobayashi, Masato Ishii, Yuhta Takida, Takashi Shibuya, Yuki Mitsufuji

Diffusion models typically suffer from error accumulationduring iterative sampling, commonly referred to as exposure bias. Wereveal systematic frequency-dependent discrepancies between trainingand inference, which can be interpreted as frequency-dependent SNRerror. Crucially, the direction of this mismatch varies across modelsand timesteps, indicating that fixed correction rules do not generalize.We propose Spectral Alignment (SPA), a lightweight, guidance-basedmethod that calibrates the power spectrum of intermediate predictionsto a pre-computed prior. Our approach consists of two stages: (1) of-fline fitting of a parametric spectrum model from training data, and (2)inference-time guidance via efficient FFT-based gradient computation.SPA introduces minimal computational overhead (3-4%) and is com-plementary to Classifier-Free Guidance (CFG). We demonstrate consis-tent improvements across diverse architectures, from pixel-space mod-els (DDPM, ADM) to latent diffusion models (SD2.0, SDXL) and flow-matching models (SD3.5, FLUX).

ECCV 2026

D2PO: Optimizing Diffusion Samplers via Dynamic Preference

Jinkyu Kim, Jinyoung Choi, Bohyung Han

We propose D2PO (Dynamic Direct Preference Optimiza-tion), a principled framework for optimizing diffusion sampling policieswith respect to timestep schedules and classifier-free guidance (CFG)weights. Our work is motivated by a fundamental limitation of existingstudent-teacher regression frameworks; low-NFE student samplers aretrained to mimic high-NFE teachers, often sacrificing high-frequency tex-ture fidelity while preserving coarse global structures, thereby misalign-ing the sampler with perceptual quality. D2PO addresses this challengeby reformulating sampler optimization as a preference-based alignmentproblem, leveraging the Direct Preference Optimization (DPO) frame-work. To make DPO applicable to diffusion samplers, we model thesampling policy as an energy-based model (EBM), transforming pref-erence comparisons into tractable energy differences. We further intro-duce a novel energy formulation derived directly from the pretrainedscore network, enabling preference evaluation in perturbed spaces thatjointly capture structural consistency and fine-grained details. More-over, we introduce dynamic preferences, where the preferred samplesused for alignment progressively improve as the sampling policies arelearned. This self-improving mechanism replaces rigid static teacher su-pervision with an iterative, preference-guided refinement process, pro-viding progressively stronger alignment signals. Extensive experimentsdemonstrate that D2PO aligns diffusion samplers with perceptual qual-ity more faithfully, unlocking the full potential of high-quality teachersand consistently outperforming conventional regression-based schedulersunder low-NFE constraints.

ECCV 2026

HSFM: Hard-Set-Guided Feature-Space Meta-Learning for Robust Classification under Spurious Correlations

Aryan Yazdan Parast, Khawar Islam, Soyoun Won, Basim Azam, Naveed Akhtar

Deep neural networks often rely on spurious features to makepredictions, which makes them brittle under distribution shift and onsamples where the spurious correlation does not hold (e.g., minority-group examples). Recent studies have shown that, even in such settings,the feature extractor of an Empirical Risk Minimization (ERM)-trainedmodel can learn rich and informative representations, and that muchof the failure may be attributed to the classifier head. In particular,retraining a lightweight head while keeping the backbone frozen cansubstantially improve performance on shifted distributions and minor-ity groups. Motivated by this observation, we propose a bilevel meta-learning method that performs augmentation directly in feature space toimprove spurious correlation handling in the classifier head. Our methodlearns support-side feature edits such that, after a small number of inner-loop updates on the edited features, the classifier achieves lower loss onhard examples and improved worst-group performance. By operating atthe backbone output rather than in pixel space or through end-to-endoptimization, the method is highly efficient and stable, requiring only afew minutes of training on a single GPU. We further validate our methodwith CLIP-based visualizations, showing that the learned feature-spaceupdates induce semantically meaningful shifts aligned with spurious at-tributes. Our code is available at: https://github.com/ArianYp/HSFM.

ECCV 2026

DualDiff3D: Dual Structure-Appearance Diffusion Priors for Reliability-Enhanced 3D Gaussian Splatting

Qian Wang, Yu Wang, Weiqi Li, Xinhua Cheng, Xiandong Meng, Ronggang Wang, Jian Zhang

While 3D Gaussian Splatting (3DGS) has revolutionized 3Dreconstruction and novel-view synthesis, scenarios with limited inputviews often lead to poor reconstruction quality and artifacts in ren-dered novel views. Recent efforts attempt to utilize powerful diffusionpriors, yet they typically process rendered and reference views concate-nated along an additional dimension in a single network. These methodsoverlook an inherent nature that different views should maintain ap-pearance similarity but differ in structure due to view shifts, leadingto blur caused by conflicts between the two properties. In this paper,we propose DualDiff, a novel pipeline that leverages dual diffusion pri-ors with a Structure-Appearance Attention (SAA) module to introduce∗ Equal contributionB Corresponding author: zhangjian.sz@pku.edu.cnreference guidance for refining low-quality novel views rendered fromflawed 3D representations. Specifically, we retain one diffusion branchto focus on extracting structural information from the low-quality novelviews, while introducing another branch to ensure appearance consis-tency with reference views. Furthermore, we present a 3D reconstructionframework named DualDiff3D, which integrates a reliability-enhancedRender-Refine-Optimize (RRO) loop to progressively and robustly incor-porate the refined novel views, yielding more accurate 3DGS. Extensiveexperiments demonstrate that our approach outperforms state-of-the-art methods even in the inference-only setting, with further performancegains achievable through training. Our code and pre-trained weights willbe publicly released.

ECCV 2026

Distribution Matching Distillation Meets Reinforcement Learning

Dengyang Jiang, Dongyang Liu, Zanyi Wang, Qilong Wu, Liuzhuozheng Li, Heng-Zhuang Li, Xin Jin, Zhen Li, Changsheng Lu, Mengmeng Wang, Steven Hoi, Peng Gao, Harry Yang

Distribution Matching Distillation (DMD) facilitates efficientinference by distilling multi-step diffusion models into few-step variants.Concurrently, Reinforcement Learning (RL) has emerged as a vital toolfor aligning generative models with human preferences. While both repre-sent critical post-training stages for large-scale diffusion models, existingstudies typically treat them as independent, sequential processes, leav-ing a systematic framework for their unification largely unexplored. Inthis work, we demonstrate that jointly optimizing these two objectivesyields mutual benefits: RL enables more preference-aware and control-lable distillation rather than uniformly compressing the full data dis-tribution, while DMD serves as an effective regularizer to mitigate re-ward hacking during RL training. Building on these insights, we proposeDMDR, a unified framework that incorporates Reward-Tilted Distribu-tion Matching optimization alongside two dynamic distillation trainingstrategies in the initial stage, followed by the joint DMD and RL opti-mization in the second stage. Extensive experiments demonstrate thatDMDR achieves state-of-the-art visual quality and prompt adherenceamong few-step generation methods, even surpassing the performance ofits multi-step teacher model.

ECCV 2026

DeLux: Cross-Modal Local Artifact Restoration in Video Using Neuromorphic Data

Bartosz Stachowiak, Dariusz Brzezinski

Conventional RGB cameras suffer from lighting artifacts suchas flare, glare, flicker, and overexposure, leading to irrecoverable informa-tion loss that necessitates computational restoration. However, existingapproaches treat these problems in isolation, failing to recover structuraldetails completely obscured by complex spatially discrete image degrada-tions. In this paper, we propose a novel cross-modal restoration paradigmand present DeLux, a modular proof-of-concept pipeline that leveragesneuromorphic event streams as a structural prior to guide the targeteddetection and inpainting of lighting artifacts in RGB video. Validation onsynthetic benchmarks and real-world automotive footage demonstratesthat DeLux effectively suppresses local artifacts and restores affected re-gions. The proposed approach outperforms existing RGB-only baselinesand event-guided HDR models, achieving an average MS-SSIM of over0.99 across all artifact types and demonstrating up to an 88% reductionin artifact severity in real-world automotive footage. The synthetic arti-fact generation tools and curated real-world evaluation datasets are madepublicly available to foster future research on cross-modal restoration.

ECCV 2026

DeGuNet: Depth-Guided Ultra-Compact Backbones for Efficient LiDAR-Camera 3D Detection

Haifa Zhang, Yijing Wang, Peixi Peng, Zhiqiang Zuo

In autonomous driving perception, the fusion of LiDAR andcamera modalities has become the dominant paradigm for 3D object de-tection. However, current multi-modal frameworks heavily rely on mas-sive visual backbones pretrained on 2D semantic tasks. This relianceintroduces substantial parameter redundancy and a structural misalign-ment, as 2D priors are ill-equipped to handle the extreme sparsity of Li-DAR projections required for Bird’s-Eye-View geometry. To address this,we present DeGuNet, an ultra-compact and plug-and-play image back-bone explicitly designed for depth-guided representation learning. By in-corporating sparsity-aware feature extraction mechanisms, DeGuNet ef-fectively aligns multi-view images with unstructured LiDAR depth whilestrictly preventing invalid-region contamination. Extensive experimentson the nuScenes dataset demonstrate DeGuNet’s broad plug-and-playapplicability and superior efficiency. When integrated into establishedbaselines, it fundamentally eliminates architectural redundancy, reduc-ing GPU memory consumption by up to 66.5% and achieving a 1.16×inference speedup. Concurrently, DeGuNet delivers up to a 6.20 absolutemAP gain, establishing a new paradigm for parameter-efficient multi-modal 3D perception.

ECCV 2026

DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression

Bingzhou Li, Tao Huang

Omnimodal large language models (OmniLLMs) jointly process audio and visual streams, but the resulting long multimodal token sequences make inference prohibitively expensive. Existing compression methods typically rely on fixed window partitioning and attention-based pruning, which overlook the piecewise semantic structure of audio-visual signals and become fragile under aggressive token reduction. We propose Dynamic Audio-driven Semantic cHunking (DASH), a training-free framework that aligns token compression with semantic structure. DASH treats audio embeddings as a semantic anchor and detects boundary candidates via cosine-similarity discontinuities, inducing dynamic, variablelength segments that approximate the underlying piecewise-coherent organization of the sequence. These boundaries are projected onto video tokens as a soft temporally co-registered segmentation prior. Within each segment, token retention is determined by a tri-signal importance estimator that fuses structural boundary cues, representational distinctiveness, and attention-based salience, mitigating the sparsity bias of attentiononly selection. This structure-aware allocation preserves transition-critical tokens while reducing redundant regions. Extensive experiments on AVUT, VideoMME, and WorldSense demonstrate that DASH maintains competitive or superior accuracy while achieving higher compression ratios compared to prior methods. Code is available at: https://github.com/ laychou666/DASH.

ECCV 2026

ContextFlow: In-Context Flow Matching for Robot Manipulation

Jian Ding, Xianjie Dai, Roei Herzig, Nussair Hroub, Jinjie Mai, Dengxin Dai, Bernard Ghanem, Mohamed Elhoseiny

Although highly effective in vision and language domains, applying in-context learning to robotics remains challenging. Existing autoregressive in-context imitation methods discretize continuous actions and exacerbate the accumulation of early prediction errors through next-token prediction, limiting their generalization on unseen task configurations. Meanwhile, flow-matching policies have been explored for continuous robot control and can help mitigate compounding errors; however, in-context imitation learning within a flow-matching framework remains underexplored. To address these limitations, we introduce ContextFlow, a conditional flow-matching model that learns continuous action distributions for in-context imitation learning. ContextFlow conditions flow-based action prediction on demonstrations and observations, enabling robust generation from noisy action distributions. To better encode multimodal in-context demonstrations, we adapt perceiver-style multimodal context compressors that distill visual, proprioceptive, and action sequences into compact, task-relevant latent representations. On LIBERO, ContextFlow outperforms ICRT by 35 percentage points in average success rate on unseen task configurations, while matching the performance of the task-specific fine-tuned VLA model π with- 0 out any fine-tuning on unseen tasks. On real robots, it generalizes to unseen configurations of both single-arm and bimanual tasks, achieving 40% success on a new pen-uncapping configuration. Project Page: https://dingjiansw101.github.io/contextflow-page/.

ECCV 2026

Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMs

Huan Zheng, Yucheng Zhou, Tianyi Yan, Dubing Chen, Hongbo Lu, Wenlong Liao, Tao He, Pai Peng, Shen Jianbing

Multimodal Large Language Models (MLLMs) have demon-strated remarkable potential in medical image analysis. However, theirapplication in gastrointestinal endoscopy is currently hindered by twocritical limitations: the misalignment between general model reasoningand standardized clinical cognitive pathways, and the lack of causal as-sociation between visual features and diagnostic outcomes. In this paper,we propose a novel Clinical-Cognitive-Aligned (CogAlign) framework toaddress these challenges. First, we endow the model with rigorous clinicalanalytical capabilities by constructing the hierarchical clinical cognitiondataset and employing Supervised Fine-Tuning (SFT). Unlike conven-tional approaches, this strategy internalizes the hierarchical diagnosticlogic of experts, ranging from anatomical localization and morphologicalevaluation to microvascular analysis, directly into the model. Second, toeliminate visual bias, we provide a theoretical analysis demonstratingthat standard supervised tuning inevitably converges to spurious back-ground correlations. Guided by this insight, we propose a counterfactual-driven reinforcement learning strategy to enforce causal rectification. Bygenerating counterfactual normal samples via lesion masking and opti-mizing through clinical-cognition-centric rewards, we constrain the modelto strictly ground its diagnosis in causal lesion features. Extensive exper-iments demonstrate that our approach achieves State-of-the-Art (SoTA)performance across multiple benchmarks, significantly enhancing diag-nostic accuracy in complex clinical scenarios.

ECCV 2026

Beyond Language: Grounding Referring Expressions with Hand Pointing in Egocentric Vision

Ling Li, Bowen Liu, Zinuo Zhan, Peng Jie, Jianhui Zhong, Kenglun Chang, Zhidong Deng

Traditional Visual Grounding (VG) predominantly relies ontextual descriptions to localize objects, a paradigm that inherently strug-gles with linguistic ambiguity and often ignores non-verbal deictic cuesprevalent in real-world interactions. In natural egocentric engagements,hand-pointing combined with speech forms the most intuitive referringmechanism. To bridge this gap, we introduce EgoPoint-Ground, thefirst large-scale multimodal dataset dedicated to egocentric deictic visualgrounding. Comprising over 15k interactive samples in complex scenes,the dataset provides rich, multi-grained annotations including hand-targetbounding box pairs and dense semantic captions. We establish a com-prehensive benchmark for hand-pointing referring expression resolution,evaluating a wide spectrum of mainstream Multimodal Large LanguageModels (MLLMs) and state-of-the-art VG architectures. Furthermore, wepropose SV-CoT, a novel baseline framework that reformulates groundingas a structured inference process, synergizing gestural and linguistic cuesthrough a Visual Chain-of-Thought paradigm. Extensive experimentsdemonstrate that SV-CoT achieves an 11.7% absolute improvement overexisting methods, effectively mitigating semantic ambiguity and advanc-ing the capability of agents to comprehend multimodal physical intents.Code: https://github.com/lingli1724/Egopoint

ECCV 2026

SAND: Stage-Aware Noise Decomposition for Training-Free Diffusion Guidance

Daehyun Kim, Hyo-jun Lee, Hanul Kim, Yeong Jun Koh

Diffusion models have achieved state-of-the-art performanceacross 2D, video, and 3D generation, but their sampling quality remainsheavily dependent on guidance methods. Existing training-free guidancetechniques construct a weakened auxiliary branch through static per-turbations, yet these methods fundamentally disregard the stage-wisestructure of the denoising process: early timesteps govern magnitude-driven global formation, whereas later timesteps refine directional ge-ometry. We propose a principled training-free diffusion guidance withStage-Aware Noise Decomposition (SAND) that considers this temporalasymmetry by decomposing Gaussian noise into orthogonal magnitude-and direction-oriented components and modulating them with a time-adaptive weighting schedule aligned to diffusion dynamics. The decom-posed perturbations are injected directly into the self-attention queries,keys, and values, forming a dynamically degraded branch that requires noretraining while faithfully mirroring the evolving geometry of the sample.This representation-level perturbation provides a temporally coherentguidance signal that better balances global structure and fine detail. Ex-tensive experiments on Stable Diffusion XL (SDXL) and Stable Diffusion3 (SD3) demonstrate that our method consistently improves global co-herence, sharpens fine-grained texture, and yields superior sample qual-ity in both unconditional and conditional image generation. We furthershow that SAND can be extended to the guidance-distilled image gener-ation model FLUX.1 and the text-to-video model Wan2.2, highlightingits broad applicability.

ECCV 2026

Why and When Visual Token Pruning Fails? A Study on Relevant Visual Information Shift in MLLMs Decoding

Jiwan Kim, Kibum Kim, Wonjoong Kim, Byung-Kwan Lee, Chanyoung Park

Recently, visual token pruning has been studied to handlethe vast number of visual tokens in Multimodal Large Language Models.However, we observe that while existing pruning methods perform reliablyon simple visual understanding, they struggle to effectively generalizeto complex visual reasoning tasks, a critical gap underexplored in previ-ous studies. Through a systematic analysis, we identify Relevant VisualInformation Shift (RVIS) during decoding as the primary failure driver.To address this, we propose Decoding-stage Shift-aware Token Pruning(DSTP), a training-free add-on framework that enables existing pruningmethods to align visual tokens with shifting reasoning requirements duringthe decoding stage. Extensive experiments demonstrate that DSTP signifi-cantly mitigates performance degradation of pruning methods in complexreasoning tasks, while consistently yielding performance gains even acrossvisual understanding benchmarks. Furthermore, DSTP demonstrates ef-fectiveness across diverse state-of-the-art architectures, highlighting itsgeneralizability and efficiency with minimal computational overhead. Oursource code is available here.

ECCV 2026

VLA-Hijack: A Transferable Patch Attack against Vision-Language-Action Models via Visual Proprioception Hijacking

Jiyuan Fu, Kaixun Jiang, Jingkai Jia, Zhaoyu Chen, Xueyao Chen, Lingyi Hong, Shuyong Gao, Chenzhi Tan, Dingkang Yang, Wenqiang Zhang

While Vision-Language-Action (VLA) models have emergedas powerful generalist policies, their severe vulnerability to adversarialpatches significantly hinders their deployment in safety-critical domains.Moreover, existing patch attacks primarily focus on white-box settings,heavily overfitting to the specific action output space of the target model,which results in poor cross-architecture transferability. To overcome thislimitation, we propose VLA-Hijack, a unified adversarial framework thatbreaks the transferability bottleneck by exploiting a fundamental vulner-ability identified in this work: before planning any motion, a VLA modelmust first use visual information to locate its own robotic arm withinthe environment. Targeting this shared visual self-localization process,our approach concurrently optimizes Attention-Guided ProprioceptiveSuppression to inhibit the real robotic arm’s features, and MultimodalProprioceptive Injection to establish the patch as a surrogate "phantomembodiment". By alternating between semantic concept anchoring andvisual prototype projection, VLA-Hijack effectively severs the semanticrelationship between the agent’s true embodiment and its control pol-icy. Extensive experiments across diverse architectures (OpenVLA, Uni-VLA, and CronusVLA) demonstrate that VLA-Hijack achieves superioroptimization efficiency in white-box settings and sets a new SOTA forcross-architecture and cross-domain black-box transferability.

ECCV 2026

UniPart: Towards Zero-shot Language-Grounded 3D Part Segmentation for Embodied Interaction

XinQiang Yu, Zekun Qi, Jiawei He, Wenyao Zhang, Xuchuan Chen, Guocai Yao, Li Yi, Zhaoxiang Zhang, He Wang

Fine-grained robotic manipulation depends on understand-ing parts, not only whole objects. Existing 3D foundation models tendto be either generalized but object-aware, or part-aware but limited toclosed-set taxonomies, which weakens zero-shot transfer. We study text-conditioned 3D part segmentation, where a free-form phrase selects afunctional part on point cloud. We introduce UniPart, a feed-forwardcross-modal 3D Transformer that conditions CLIP text embedding. Toscale supervision, we build LangPart-1M with 160K+ Objaverse assetsand 8M text to part pairs using multi-view consistent part generation.We further manually label a high-quality subset, LangPart-4K, for fine-tuning and evaluation. UniPart achieves strong zero-shot results on open-vocabulary part benchmarks and transfers to language-conditioned partgrasping in real world.

ECCV 2026

Unified Backbone Refinement for Diffusion Models via Internal-Latent Analysis

HAKSU LIM, Myeongjin Lee, Wonjoon Chang, Jaesik Choi

Diffusion models have achieved remarkable success across di-verse domains, with performance closely related to the denoising back-bones that parameterize the score function. In this paper, we presenta systematic, phase-aware analysis of diffusion components and showthat abrupt, early-stage fluctuations in deep latents are strongly asso-ciated with artifacts. Guided by these findings, we introduce DUNE(Diffusion Unified Network refiNEr), a training-free refinement frame-work that detects abrupt deviations in deep low-noise internal latentsusing a shared EMA-based criterion, and applies backbone-specific sup-pression to the detector-selected entries. Although derived from U-Net,the same detect–suppress principle extends naturally to Transformer-based diffusion models by acting on the latents of deep self-attentionblocks. Extensive experiments across multiple backbones indicate thatDUNE improves fidelity while reducing hallucinations, offering new in-sight into where and when diffusion backbones should be controlled.

ECCV 2026

Nexels: Neurally-Textured Surfels for Real-Time Novel View Synthesis with Sparse Primitives

Victor Rong, Jan Held, Victor Chu, Daniel Rebain, Marc Van Droogenbroeck, Kyros Kutulakos, Andrea Tagliasacchi, David Lindell

Though Gaussian splatting has achieved impressive resultsin novel view synthesis, it requires millions of primitives to representhighly textured scenes — even when the underlying geometry is simple.We propose a representation that decouples geometry and appearanceto achieve a compact and computationally efficient representation. Ourmethod uses surfels for geometry and combines a global neural field withper-primitive colors for appearance. While existing approaches that tex-ture primitives incur significant overhead in either memory or renderingspeed, our method strikes a better balance by leveraging the neural fieldto implicitly compress texture information. We bound the computationsrequired in the rendering pass by querying the neural field for a fixednumber of primitives per pixel, enabling real-time rendering speeds. Ourrepresentation matches the perceptual quality of 3D Gaussian splattingwhile using 9.7× fewer primitives and 5.5× less memory on outdoorscenes, and 31× fewer primitives and 3.7× less memory on indoor scenes.

ECCV 2026

UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models

Ioannis Maniadis Metaxas, Adrian Bulat, Alberto Baldrati, Anestis Zaganidis, Yassine Ouali, Hyeonuk Kim, Georgios Tzimiropoulos

Large Vision-Language Models (LVLMs) remain bottleneckedby massive computational footprints, precluding their deployment onresource-constrained edge devices. While efforts to compress LVLMs fo-cus heavily on vision token reduction or smaller language models, thevision encoder is largely overlooked, typically deployed as a monolithic,computationally heavy feature extractor. Moreover, there is no previouseffort that designs a vision encoder for LVLMs directly optimized foron-device latency. In this paper, we present UltraViT, a vision encoderfor LVLMs, explicitly designed and optimized for on-device performance.Specifically, by taking into account real on-device latencies, we system-atically design a pyramidal architecture that strategically integrates andadapts heterogeneous spatial mixers at the macro-block level. Further-more, to pre-train UltraViT, we propose a novel two-stage generativepre-training strategy: cultivating rich spatial features via dense distil-lation, followed by direct generative supervision from a capacity-mixedfrozen LLM. Compared to standard contrastive and SSL, we show thatour pre-training is much more effective for achieving high-level seman-tic grounding for UltraViT needed for the subsequent generative multi-modal alignment of LVLM training. Extensive experiments demonstratethat our on-device latency-informed design combined with our tailoredtraining strategy establishes a new state-of-the-art for efficient LVLMencoding, significantly outperforming existing encoder-centric baselineswhile operating on-device at nearly 1.7× the speed.

ECCV 2026

SK-Adapter: Skeleton-Based Structural Control for Native 3D Generation

Anbang Wang, AO Yuzhuo, Elliott (Shangzhe) Wu, Chi-Keung Tang

Native 3D generative models have achieved remarkable _x001C_-delity and speed, yet they su_x001B_er from a critical limitation: inability toprescribe precise structural articulations, where precise structural con-trol within the native 3D space remains underexplored. This paper pro-poses SK-Adapter, a simple yet e_x001E_cient and e_x001B_ective framework thatunlocks precise skeletal manipulation for native 3D generation. Mov-ing beyond text or image prompts, which can be ambiguous for precisestructure, we treat the 3D skeleton as a _x001C_rst-class control signal. SK-Adapter is a lightweight structural adapter network that encodes jointcoordinates and topology into learnable tokens, which are injected intothe frozen 3D generation backbone via cross-attention. This design al-lows the model to not only e_x001B_ectively _x0010_attend_x0011_ to speci_x001C_c 3D structuralconstraints but also preserve its original generative priors. To bridgethe data gap, we contribute the Objaverse-TMS dataset, a large-scale⋆ „Equal contribution. Equal advising.dataset of 24k text-mesh-skeleton pairs. Extensive experiments con_x001C_rmthat our method achieves robust structural control while preserving thegeometry and texture quality of the foundation model, signi_x001C_cantly out-performing existing baselines. Furthermore, we extend this capability tolocal 3D editing, enabling region-speci_x001C_c editing of existing assets withskeletal guidance, which is unattainable by previous methods. Projectpage: https://sk-adapter.github.io/

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.