Vision-Language Pretraining papers, 2023–2026

Key findings

• Vision-Language Pretraining went from 65 papers in 2023 to 87 in 2026, roughly 1.3x in three years.
• NeurIPS publishes the most Vision-Language Pretraining work: 93 papers from 2023 to 2026, 35% of the four-venue total.
• The fastest-growing overlap is with Applications · computer vision: papers tagged with both rose from 0 in 2023 to 9 in 2026.
• Among the 5 topics under Open-Vocabulary Vision, it ranks #4 by 2026 paper count; the fastest grower in the group, Visual Grounding, is up 4.3x versus 1.3x here.

Representative papers

Spread across years and venues so every edition shows up

• NeurIPS 2026Benchmarking Optimizers for Large Language Model Pretraining
• ICML 2026How Does the Pretraining Distribution Shape In-Context Learning? A Fundamental Trade-Off
• ICLR 2026PonderLM: Pretraining Language Models to Ponder in Continuous Space
• CVPR 2026Large-scale Codec Avatars The Unreasonable Effectiveness of Large-scale Avatar Pretraining
• NeurIPS 2025ADPretrain: Advancing Industrial Anomaly Detection via Anomaly Representation Pretraining
• ICML 2025Soup-of-Experts: Pretraining Specialist Models via Parameters Averaging
• ICLR 2025Attributing Culture-Conditioned Generations to Pretraining Corpora
• CVPR 2025MAP: Unleashing Hybrid Mamba-Transformer Vision Backbone's Potential with Masked Autoregressive Pretraining
• NeurIPS 2024Data curation via joint example selection further accelerates multimodal learning
• ICML 2024Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining
• ICLR 2024Demystifying CLIP Data
• CVPR 2024ParameterNet: Parameters Are All You Need for Large-scale Visual Pretraining of Mobile Networks
• NeurIPS 2023Bootstrapping Vision-Language Learning with Decoupled Language Pre-training
• ICML 2023Contrastive Learning Meets Homophily: Two Birds with One Stone
• ICLR 2023Energy-Inspired Self-Supervised Pretraining for Vision Models
• CVPR 2023Active Finetuning: Exploiting Annotation Budget in the Pretraining-Finetuning Paradigm
• NeurIPS 2026C3P: Contrastive promoter-protein pretraining yields representations capturing bacterial gene regulation
• ICML 2026Multipole Semantic Attention: A Fast Approximation of Softmax Attention for Pretraining
• ICLR 2026MICLIP: Learning to Interpret Representation in Vision Models
• CVPR 2026The Surprising Effectiveness of Noise Pretraining for Implicit Neural Representations

All Vision-Language Pretraining papers

39 papers, page 1 of 1

1. ICML 2026A Theory of Contrastive Learning with Natural Images
2. ICML 2026Beyond Blind Noising: Disentangled Visual Rectification for Hallucination Mitigation in MLLMs
3. ICML 2026Beyond Logits: Coherent Hallucination Mitigation via Attention Contrastive Decoding
4. ICML 2026BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining
5. ICML 2026Data Augmentation of Contrastive Learning is Estimating Positive-incentive Noise
6. ICML 2026DEGAP: Dynamic Entropy-Guided Attention Perturbation for Contrastive Decoding in Large Vision-Language Models
7. ICML 2026Disease-Centric Vision-Language Pretraining with Hybrid Visual Encoding for 3D Computed Tomography
8. ICML 2026Facts in Stats: Impacts of Pretraining Diversity on Language Model Generalization
9. ICML 2026How Does the Pretraining Distribution Shape In-Context Learning? A Fundamental Trade-Off
10. ICML 2026Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAs
11. ICML 2026LLM-Guided Diagnostic Evidence Alignment for Medical Vision–Language Pretraining under Limited Pairing
12. ICML 2026Modular Pretraining Enables Access Control
13. ICML 2026Multipole Semantic Attention: A Fast Approximation of Softmax Attention for Pretraining
14. ICML 2026Neutral-Reference Prompting for Vision–Language Models
15. ICML 2026On Revisiting Entropy for Identifying Mislabeled Images
16. ICML 2026Optimal Splitting of Language Models from Mixtures to Specialized Domains
17. ICML 2026Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuning
18. ICML 2026Text-Conditional JEPA for Learning Semantically Rich Visual Representations
19. ICML 2026Unifying Heterogeneous Multi-Modal Remote Sensing Detection Via Language-Pivoted Pretraining
20. ICML 2025A Bayesian Model Selection Criterion for Selecting Pretraining Checkpoints
21. ICML 2025Learning Invariant Causal Mechanism from Vision-Language Models
22. ICML 2025Soup-of-Experts: Pretraining Specialist Models via Parameters Averaging
23. ICML 2024Amend to Alignment: Decoupled Prompt Tuning for Mitigating Spurious Correlation in Vision-Language Models
24. ICML 2024Modeling Caption Diversity in Contrastive Vision-Language Pretraining
25. ICML 2024OLLIE: Imitation Learning from Offline Pretraining to Online Finetuning
26. ICML 2024Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining
27. ICML 2024Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss
28. ICML 2024Provable Contrastive Continual Learning
29. ICML 2024Revealing Vision-Language Integration in the Brain with Multimodal Networks
30. ICML 2024SyCoCa: Symmetrizing Contrastive Captioners with Attentive Masking for Multimodal Alignment
31. ICML 2024Two Tales of Single-Phase Contrastive Hebbian Learning
32. ICML 2023Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining
33. ICML 2023Contrastive Learning Meets Homophily: Two Birds with One Stone
34. ICML 2023End-to-end Training of Deep Boltzmann Machines by Unbiased Contrastive Divergence with Local Mode Initialization
35. ICML 2023Lifelong Language Pretraining with Distribution-Specialized Experts
36. ICML 2023Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding
37. ICML 2023Pretraining Language Models with Human Preferences
38. ICML 2023Why Is Public Pretraining Necessary for Private Model Training?
39. ICML 2023XTab: Cross-table Pretraining for Tabular Transformers

FAQ

How many Vision-Language Pretraining papers were published at NeurIPS, ICLR, ICML and CVPR from 2023 to 2026?

263 in total: 65 in 2023, 55 in 2024, 56 in 2025, 87 in 2026.

How many Vision-Language Pretraining papers are at NeurIPS 2026?

NeurIPS 2026 has 27 Vision-Language Pretraining papers, versus 26 the year before.

How many Vision-Language Pretraining papers are at ICLR 2026?

ICLR 2026 has 14 Vision-Language Pretraining papers, versus 17 the year before.

How many Vision-Language Pretraining papers are at ICML 2026?

ICML 2026 has 19 Vision-Language Pretraining papers, versus 3 the year before.

How many Vision-Language Pretraining papers are at CVPR 2026?

CVPR 2026 has 27 Vision-Language Pretraining papers, versus 10 the year before.

Which conference publishes the most Vision-Language Pretraining papers?

NeurIPS, with 93 papers from 2023 to 2026.