视觉语言预训练论文趋势 2023–2026

主要结论

• 视觉语言预训练论文从 2023 年的 65 篇增长到 2026 年的 87 篇，三年间扩大到约 1.3 倍。
• NeurIPS 是发视觉语言预训练论文最多的会议，2023–2026 年共 93 篇，占四个会议总量的 35%。
• 交叉方向里涨得最快的是应用 · computer vision：同时属于两个主题的论文从 2023 年的 0 篇增至 2026 年的 9 篇。
• 在「开放词汇视觉」的 5 个主题里，它的 2026 年论文数排第 4；同组增长最快的视觉定位是 4.3 倍，视觉语言预训练是 1.3 倍。

代表论文

按年份和会议均匀挑选，方便快速看到每一届的样貌

• NeurIPS 2026Benchmarking Optimizers for Large Language Model Pretraining
• ICML 2026How Does the Pretraining Distribution Shape In-Context Learning? A Fundamental Trade-Off
• ICLR 2026PonderLM: Pretraining Language Models to Ponder in Continuous Space
• CVPR 2026Large-scale Codec Avatars The Unreasonable Effectiveness of Large-scale Avatar Pretraining
• NeurIPS 2025ADPretrain: Advancing Industrial Anomaly Detection via Anomaly Representation Pretraining
• ICML 2025Soup-of-Experts: Pretraining Specialist Models via Parameters Averaging
• ICLR 2025Attributing Culture-Conditioned Generations to Pretraining Corpora
• CVPR 2025MAP: Unleashing Hybrid Mamba-Transformer Vision Backbone's Potential with Masked Autoregressive Pretraining
• NeurIPS 2024Data curation via joint example selection further accelerates multimodal learning
• ICML 2024Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining
• ICLR 2024Demystifying CLIP Data
• CVPR 2024ParameterNet: Parameters Are All You Need for Large-scale Visual Pretraining of Mobile Networks
• NeurIPS 2023Bootstrapping Vision-Language Learning with Decoupled Language Pre-training
• ICML 2023Contrastive Learning Meets Homophily: Two Birds with One Stone
• ICLR 2023Energy-Inspired Self-Supervised Pretraining for Vision Models
• CVPR 2023Active Finetuning: Exploiting Annotation Budget in the Pretraining-Finetuning Paradigm
• NeurIPS 2026C3P: Contrastive promoter-protein pretraining yields representations capturing bacterial gene regulation
• ICML 2026Multipole Semantic Attention: A Fast Approximation of Softmax Attention for Pretraining
• ICLR 2026MICLIP: Learning to Interpret Representation in Vision Models
• CVPR 2026The Surprising Effectiveness of Noise Pretraining for Implicit Neural Representations

全部视觉语言预训练论文

共 65 篇，第 1 / 2 页

1. CVPR 2023Accelerating Vision-Language Pretraining With Free Language Modeling
2. CVPR 2023Active Finetuning: Exploiting Annotation Budget in the Pretraining-Finetuning Paradigm
3. CVPR 2023CLIP2: Contrastive Language-Image-Point Pretraining From Real-World Point Cloud Data
4. CVPR 2023CREPE: Can Vision-Language Foundation Models Reason Compositionally?
5. CVPR 2023DegAE: A New Pretraining Paradigm for Low-Level Vision
6. CVPR 2023Finetune Like You Pretrain: Improved Finetuning of Zero-Shot Vision Models
7. CVPR 2023FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation
8. CVPR 2023HyperMatch: Noise-Tolerant Semi-Supervised Learning via Relaxed Contrastive Constraint
9. CVPR 2023Image as a Foreign Language: BEiT Pretraining for Vision and Vision-Language Tasks
10. CVPR 2023Is BERT Blind? Exploring the Effect of Vision-and-Language Pretraining on Visual Language Understanding
11. CVPR 2023Learning Imbalanced Data With Vision Transformers
12. CVPR 2023Learning Instance-Level Representation for Large-Scale Multi-Modal Pretraining in E-Commerce
13. CVPR 2023MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretraining
14. CVPR 2023MixMAE: Mixed and Masked Autoencoder for Efficient Pretraining of Hierarchical Vision Transformers
15. CVPR 2023OmniMAE: Single Model Masked Pretraining on Images and Videos
16. CVPR 2023Open-Set Fine-Grained Retrieval via Prompting Vision-Language Evaluator
17. CVPR 2023Q: How To Specialize Large Vision-Language Models to Data-Scarce VQA Tasks? A: Self-Train on Unlabeled Images!
18. CVPR 2023Region-Aware Pretraining for Open-Vocabulary Object Detection With Vision Transformers
19. CVPR 2023Training Debiased Subnetworks With Contrastive Weight Pruning
20. CVPR 2023VILA: Learning Image Aesthetics From User Comments With Vision-Language Pretraining
21. ICLR 2023A theoretical study of inductive biases in contrastive learning
22. ICLR 2023CodeBPE: Investigating Subtokenization Options for Large Language Model Pretraining on Source Code
23. ICLR 2023Contrastive Alignment of Vision to Language Through Parameter-Efficient Transfer Learning
24. ICLR 2023Energy-Inspired Self-Supervised Pretraining for Vision Models
25. ICLR 2023From $t$-SNE to UMAP with contrastive learning
26. ICLR 2023HiCLIP: Contrastive Language-Image Pretraining with Hierarchy-aware Attention
27. ICLR 2023Learning Language Representations with Logical Inductive Bias
28. ICLR 2023Proposal-Contrastive Pretraining for Object Detection from Fewer Data
29. ICLR 2023Protein Representation Learning by Geometric Structure Pretraining
30. ICLR 2023Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints
31. ICLR 2023UL2: Unifying Language Learning Paradigms
32. ICLR 2023What Do Self-Supervised Vision Transformers Learn?
33. ICLR 2023When and Why Vision-Language Models Behave like Bags-Of-Words, and What to Do About It?
34. ICML 2023Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining
35. ICML 2023Contrastive Learning Meets Homophily: Two Birds with One Stone
36. ICML 2023End-to-end Training of Deep Boltzmann Machines by Unbiased Contrastive Divergence with Local Mode Initialization
37. ICML 2023Lifelong Language Pretraining with Distribution-Specialized Experts
38. ICML 2023Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding
39. ICML 2023Pretraining Language Models with Human Preferences
40. ICML 2023Why Is Public Pretraining Necessary for Private Model Training?
41. ICML 2023XTab: Cross-table Pretraining for Tabular Transformers
42. NeurIPS 2023Bootstrapping Vision-Language Learning with Decoupled Language Pre-training
43. NeurIPS 2023Brain encoding models based on multimodal transformers can transfer across language and vision
44. NeurIPS 2023CoDet: Co-occurrence Guided Region-Word Alignment for Open-Vocabulary Object Detection
45. NeurIPS 2023Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP
46. NeurIPS 2023D4: Improving LLM Pretraining via Document De-Duplication and Diversification
47. NeurIPS 2023DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physics
48. NeurIPS 2023Don’t Stop Pretraining? Make Prompt-based Fine-tuning Powerful Learner
49. NeurIPS 2023DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
50. NeurIPS 2023Enhancing Adversarial Contrastive Learning via Adversarial Invariant Regularization

常见问题

2023–2026 年四大会议一共有多少篇视觉语言预训练论文？

共 263 篇，按年份分别是 2023 年 65 篇、2024 年 55 篇、2025 年 56 篇、2026 年 87 篇。

NeurIPS 2026 有多少篇视觉语言预训练论文？

NeurIPS 2026 有 27 篇视觉语言预训练论文，上一届是 26 篇。

ICLR 2026 有多少篇视觉语言预训练论文？

ICLR 2026 有 14 篇视觉语言预训练论文，上一届是 17 篇。

ICML 2026 有多少篇视觉语言预训练论文？

ICML 2026 有 19 篇视觉语言预训练论文，上一届是 3 篇。

CVPR 2026 有多少篇视觉语言预训练论文？

CVPR 2026 有 27 篇视觉语言预训练论文，上一届是 10 篇。

哪个会议的视觉语言预训练论文最多？

NeurIPS，2023–2026 年共 93 篇。