Vision-Language Pretraining papers, 2023–2026

Key findings

• Vision-Language Pretraining went from 65 papers in 2023 to 87 in 2026, roughly 1.3x in three years.
• NeurIPS publishes the most Vision-Language Pretraining work: 93 papers from 2023 to 2026, 35% of the four-venue total.
• The fastest-growing overlap is with Applications · computer vision: papers tagged with both rose from 0 in 2023 to 9 in 2026.
• Among the 5 topics under Open-Vocabulary Vision, it ranks #4 by 2026 paper count; the fastest grower in the group, Visual Grounding, is up 4.3x versus 1.3x here.

Representative papers

Spread across years and venues so every edition shows up

• NeurIPS 2026Benchmarking Optimizers for Large Language Model Pretraining
• ICML 2026How Does the Pretraining Distribution Shape In-Context Learning? A Fundamental Trade-Off
• ICLR 2026PonderLM: Pretraining Language Models to Ponder in Continuous Space
• CVPR 2026Large-scale Codec Avatars The Unreasonable Effectiveness of Large-scale Avatar Pretraining
• NeurIPS 2025ADPretrain: Advancing Industrial Anomaly Detection via Anomaly Representation Pretraining
• ICML 2025Soup-of-Experts: Pretraining Specialist Models via Parameters Averaging
• ICLR 2025Attributing Culture-Conditioned Generations to Pretraining Corpora
• CVPR 2025MAP: Unleashing Hybrid Mamba-Transformer Vision Backbone's Potential with Masked Autoregressive Pretraining
• NeurIPS 2024Data curation via joint example selection further accelerates multimodal learning
• ICML 2024Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining
• ICLR 2024Demystifying CLIP Data
• CVPR 2024ParameterNet: Parameters Are All You Need for Large-scale Visual Pretraining of Mobile Networks
• NeurIPS 2023Bootstrapping Vision-Language Learning with Decoupled Language Pre-training
• ICML 2023Contrastive Learning Meets Homophily: Two Birds with One Stone
• ICLR 2023Energy-Inspired Self-Supervised Pretraining for Vision Models
• CVPR 2023Active Finetuning: Exploiting Annotation Budget in the Pretraining-Finetuning Paradigm
• NeurIPS 2026C3P: Contrastive promoter-protein pretraining yields representations capturing bacterial gene regulation
• ICML 2026Multipole Semantic Attention: A Fast Approximation of Softmax Attention for Pretraining
• ICLR 2026MICLIP: Learning to Interpret Representation in Vision Models
• CVPR 2026The Surprising Effectiveness of Noise Pretraining for Implicit Neural Representations

All Vision-Language Pretraining papers

56 papers, page 1 of 2

1. ICLR 2026CLIP Behaves like a Bag-of-Words Model Cross-modally but not Uni-modally
2. ICLR 2026CortiLife: A Unified Framework for Cortical Representation Learning across the Lifespan
3. ICLR 2026D2E: Scaling Vision-Action Pretraining on Desktop Data for Transfer to Embodied AI
4. ICLR 2026Genomic Foundationless Models: Pretraining Does Not Promise Performance
5. ICLR 2026GranViT: A Fine-Grained Vision Model With Autoregressive Perception For MLLMs
6. ICLR 2026Investigating Redundancy in Multimodal Large Language Models with Multiple Vision Encoders
7. ICLR 2026Learning Facts at Scale with Active Reading
8. ICLR 2026MICLIP: Learning to Interpret Representation in Vision Models
9. ICLR 2026PonderLM: Pretraining Language Models to Ponder in Continuous Space
10. ICLR 2026Reward Models Inherit Value Biases from Pretraining
11. ICLR 2026ScaleCap: Scalable Image Captioning via Dual-Modality Debiasing
12. ICLR 2026Self-Augmented Visual Contrastive Decoding
13. ICLR 2026Train Once, Answer All: Many Pretraining Experiments for the Cost of One
14. ICLR 2026When Priors Backfire: On the Vulnerability of Unlearnable Examples to Pretraining
15. ICLR 2025$\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs
16. ICLR 2025Attributing Culture-Conditioned Generations to Pretraining Corpora
17. ICLR 2025Autoregressive Pretraining with Mamba in Vision
18. ICLR 2025Beyond Random Augmentations: Pretraining with Hard Views
19. ICLR 2025Compositional Entailment Learning for Hyperbolic Vision-Language Models
20. ICLR 2025Detecting Backdoor Samples in Contrastive Language Image Pretraining
21. ICLR 2025Efficient stagewise pretraining via progressive subnetworks
22. ICLR 2025Fine-tuning can Help Detect Pretraining Data from Large Language Models
23. ICLR 2025Generalization v.s. Memorization: Tracing Language Models’ Capabilities Back to Pretraining Data
24. ICLR 2025Harnessing Diversity for Important Data Selection in Pretraining Large Language Models
25. ICLR 2025Holographic Node Representations: Pre-training Task-Agnostic Node Embeddings
26. ICLR 2025Improving Pretraining Data Using Perplexity Correlations
27. ICLR 2025Latent Action Pretraining from Videos
28. ICLR 2025Probabilistic Language-Image Pre-Training
29. ICLR 2025Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models
30. ICLR 2025TIPS: Text-Image Pretraining with Spatial awareness
31. ICLR 2025VTDexManip: A Dataset and Benchmark for Visual-tactile Pretraining and Dexterous Manipulation with Reinforcement Learning
32. ICLR 20243D Feature Prediction for Masked-AutoEncoder-Based Point Cloud Pretraining
33. ICLR 2024CLIP the Bias: How Useful is Balancing Data in Multimodal Learning?
34. ICLR 2024COSA: Concatenated Sample Pretrained Vision-Language Foundation Model
35. ICLR 2024Demystifying CLIP Data
36. ICLR 2024Detecting Pretraining Data from Large Language Models
37. ICLR 2024Headless Language Models: Learning without Predicting with Contrastive Weight Tying
38. ICLR 2024In-Context Pretraining: Language Modeling Beyond Document Boundaries
39. ICLR 2024LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment
40. ICLR 2024On the Provable Advantage of Unsupervised Pretraining
41. ICLR 2024Tag2Text: Guiding Vision-Language Model via Image Tagging
42. ICLR 2024Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization
43. ICLR 2024Unsupervised Pretraining for Fact Verification by Language Model Distillation
44. ICLR 2023A theoretical study of inductive biases in contrastive learning
45. ICLR 2023CodeBPE: Investigating Subtokenization Options for Large Language Model Pretraining on Source Code
46. ICLR 2023Contrastive Alignment of Vision to Language Through Parameter-Efficient Transfer Learning
47. ICLR 2023Energy-Inspired Self-Supervised Pretraining for Vision Models
48. ICLR 2023From $t$-SNE to UMAP with contrastive learning
49. ICLR 2023HiCLIP: Contrastive Language-Image Pretraining with Hierarchy-aware Attention
50. ICLR 2023Learning Language Representations with Logical Inductive Bias

FAQ

How many Vision-Language Pretraining papers were published at NeurIPS, ICLR, ICML and CVPR from 2023 to 2026?

263 in total: 65 in 2023, 55 in 2024, 56 in 2025, 87 in 2026.

How many Vision-Language Pretraining papers are at NeurIPS 2026?

NeurIPS 2026 has 27 Vision-Language Pretraining papers, versus 26 the year before.

How many Vision-Language Pretraining papers are at ICLR 2026?

ICLR 2026 has 14 Vision-Language Pretraining papers, versus 17 the year before.

How many Vision-Language Pretraining papers are at ICML 2026?

ICML 2026 has 19 Vision-Language Pretraining papers, versus 3 the year before.

How many Vision-Language Pretraining papers are at CVPR 2026?

CVPR 2026 has 27 Vision-Language Pretraining papers, versus 10 the year before.

Which conference publishes the most Vision-Language Pretraining papers?

NeurIPS, with 93 papers from 2023 to 2026.