Vision-Language Pretraining papers, 2023–2026

Key findings

• Vision-Language Pretraining went from 65 papers in 2023 to 87 in 2026, roughly 1.3x in three years.
• NeurIPS publishes the most Vision-Language Pretraining work: 93 papers from 2023 to 2026, 35% of the four-venue total.
• The fastest-growing overlap is with Applications · computer vision: papers tagged with both rose from 0 in 2023 to 9 in 2026.
• Among the 5 topics under Open-Vocabulary Vision, it ranks #4 by 2026 paper count; the fastest grower in the group, Visual Grounding, is up 4.3x versus 1.3x here.

Representative papers

Spread across years and venues so every edition shows up

• NeurIPS 2026Benchmarking Optimizers for Large Language Model Pretraining
• ICML 2026How Does the Pretraining Distribution Shape In-Context Learning? A Fundamental Trade-Off
• ICLR 2026PonderLM: Pretraining Language Models to Ponder in Continuous Space
• CVPR 2026Large-scale Codec Avatars The Unreasonable Effectiveness of Large-scale Avatar Pretraining
• NeurIPS 2025ADPretrain: Advancing Industrial Anomaly Detection via Anomaly Representation Pretraining
• ICML 2025Soup-of-Experts: Pretraining Specialist Models via Parameters Averaging
• ICLR 2025Attributing Culture-Conditioned Generations to Pretraining Corpora
• CVPR 2025MAP: Unleashing Hybrid Mamba-Transformer Vision Backbone's Potential with Masked Autoregressive Pretraining
• NeurIPS 2024Data curation via joint example selection further accelerates multimodal learning
• ICML 2024Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining
• ICLR 2024Demystifying CLIP Data
• CVPR 2024ParameterNet: Parameters Are All You Need for Large-scale Visual Pretraining of Mobile Networks
• NeurIPS 2023Bootstrapping Vision-Language Learning with Decoupled Language Pre-training
• ICML 2023Contrastive Learning Meets Homophily: Two Birds with One Stone
• ICLR 2023Energy-Inspired Self-Supervised Pretraining for Vision Models
• CVPR 2023Active Finetuning: Exploiting Annotation Budget in the Pretraining-Finetuning Paradigm
• NeurIPS 2026C3P: Contrastive promoter-protein pretraining yields representations capturing bacterial gene regulation
• ICML 2026Multipole Semantic Attention: A Fast Approximation of Softmax Attention for Pretraining
• ICLR 2026MICLIP: Learning to Interpret Representation in Vision Models
• CVPR 2026The Surprising Effectiveness of Noise Pretraining for Implicit Neural Representations

All Vision-Language Pretraining papers

263 papers, page 2 of 6

51. ICML 2026Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAs
52. ICML 2026LLM-Guided Diagnostic Evidence Alignment for Medical Vision–Language Pretraining under Limited Pairing
53. ICML 2026Modular Pretraining Enables Access Control
54. ICML 2026Multipole Semantic Attention: A Fast Approximation of Softmax Attention for Pretraining
55. ICML 2026Neutral-Reference Prompting for Vision–Language Models
56. ICML 2026On Revisiting Entropy for Identifying Mislabeled Images
57. ICML 2026Optimal Splitting of Language Models from Mixtures to Specialized Domains
58. ICML 2026Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuning
59. ICML 2026Text-Conditional JEPA for Learning Semantically Rich Visual Representations
60. ICML 2026Unifying Heterogeneous Multi-Modal Remote Sensing Detection Via Language-Pivoted Pretraining
61. NeurIPS 2026Benchmarking Optimizers for Large Language Model Pretraining
62. NeurIPS 2026C3P: Contrastive promoter-protein pretraining yields representations capturing bacterial gene regulation
63. NeurIPS 2026Contrastive Pretraining Scales Agentic Exploration
64. NeurIPS 2026CytoWave: Perturbation-Centric Pretraining for Single-Cell Response Prediction
65. NeurIPS 2026Does Cross-Panel Pretraining Transfer Under Marker Heterogeneity? A Large-Scale Empirical Study in Clinical Flow Cytometry
66. NeurIPS 2026Don't Pay Attention, PLANT It: Pretraining Attention via Learning-to-Rank
67. NeurIPS 2026EMO: Pretraining Mixture of Experts for Emergent Modularity
68. NeurIPS 2026Empowering Time Series Analysis with Large-Scale Multimodal Pretraining
69. NeurIPS 2026FinAl: Fine-grained Alignment for Detail-Preserving Medical Vision-Language Pretraining
70. NeurIPS 2026Improving Self-Supervised Vision Transformers with Cross Distillation
71. NeurIPS 2026Learning to Read Out: Unembedding Dynamics in Language Model Pretraining
72. NeurIPS 2026Learning What to Predict: Downstream-Guided Task Design for Continued Pretraining
73. NeurIPS 2026mmLIP: mmWave Radar-Language Interactive Pretraining via Point Confidence
74. NeurIPS 2026Neuron Populations Exhibit Divergent Selectivity with Scale
75. NeurIPS 2026Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima
76. NeurIPS 2026Pretraining Curricula Enable Selective Fine-tuning
77. NeurIPS 2026Pretraining Data Statistics Shape the Phases of Learning Entity Comparison in Language Models
78. NeurIPS 2026ProCLIP: Progressive Vision-Language Alignment via LLM-based Embedder
79. NeurIPS 2026Progressive Residual Warmup for Language Model Pretraining
80. NeurIPS 2026Rethinking Contrastive Targets in Radiology Language-Image Pretraining
81. NeurIPS 2026Scaling Neural Motor Decoding via Decoupled Behavioral Pretraining
82. NeurIPS 2026Shaping Useful Noise: Energy Distributions Predict Visual Pretraining Quality
83. NeurIPS 2026SimReg: Achieving Higher Performance in the Pretraining via Embedding Similarity Regularization
84. NeurIPS 2026Standing on the Shoulders of Giants: Rethinking EEG Foundation Model Pretraining via Multi-Teacher Distillation
85. NeurIPS 2026Strong Teacher Not Needed? On Distillation in LLM Pretraining
86. NeurIPS 2026Towards Scalable Data Diversification for Language Model Pretraining via Leverage Score Sampling
87. NeurIPS 2026TTVidT: Decoupling the Temporal Axis for Efficient Motion-Centric Video Pretraining
88. CVPR 20253D-MVP: 3D Multiview Pretraining for Manipulation
89. CVPR 2025Context-Aware Multimodal Pretraining
90. CVPR 2025MAP: Unleashing Hybrid Mamba-Transformer Vision Backbone's Potential with Masked Autoregressive Pretraining
91. CVPR 2025Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding
92. CVPR 2025Perceptual Inductive Bias Is What You Need Before Contrastive Learning
93. CVPR 2025Probing the Mid-level Vision Capabilities of Self-Supervised Learning
94. CVPR 2025Reproducible Vision-Language Models Meet Concepts Out of Pre-Training
95. CVPR 2025SAIST: Segment Any Infrared Small Target Model Guided by Contrastive Language-Image Pretraining
96. CVPR 2025SLADE: Shielding against Dual Exploits in Large Vision-Language Models
97. CVPR 2025Video Language Model Pretraining with Spatio-temporal Masking
98. ICLR 2025$\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs
99. ICLR 2025Attributing Culture-Conditioned Generations to Pretraining Corpora
100. ICLR 2025Autoregressive Pretraining with Mamba in Vision

FAQ

How many Vision-Language Pretraining papers were published at NeurIPS, ICLR, ICML and CVPR from 2023 to 2026?

263 in total: 65 in 2023, 55 in 2024, 56 in 2025, 87 in 2026.

How many Vision-Language Pretraining papers are at NeurIPS 2026?

NeurIPS 2026 has 27 Vision-Language Pretraining papers, versus 26 the year before.

How many Vision-Language Pretraining papers are at ICLR 2026?

ICLR 2026 has 14 Vision-Language Pretraining papers, versus 17 the year before.

How many Vision-Language Pretraining papers are at ICML 2026?

ICML 2026 has 19 Vision-Language Pretraining papers, versus 3 the year before.

How many Vision-Language Pretraining papers are at CVPR 2026?

CVPR 2026 has 27 Vision-Language Pretraining papers, versus 10 the year before.

Which conference publishes the most Vision-Language Pretraining papers?

NeurIPS, with 93 papers from 2023 to 2026.