分布式训练论文趋势 2023–2026

主要结论

• 分布式训练论文从 2023 年的 95 篇增长到 2026 年的 98 篇，三年间扩大到约 1.0 倍。
• NeurIPS 是发分布式训练论文最多的会议，2023–2026 年共 208 篇，占四个会议总量的 53%。
• 交叉方向里涨得最快的是多模态大模型：同时属于两个主题的论文从 2023 年的 1 篇增至 2026 年的 10 篇。
• 在「效率与系统」的 5 个主题里，它的 2026 年论文数排第 4；同组增长最快的硬件协同是 4.4 倍，分布式训练是 1.0 倍。

代表论文

按年份和会议均匀挑选，方便快速看到每一届的样貌

• NeurIPS 2026AgentWeave: Efficient Distributed Agent Serving via Flow Decomposition
• ICML 2026Breaking the Capacity Bottleneck in Model-Heterogeneous Federated Learning via Gradual Model Restoration
• ICLR 2026Cascadia: An Efficient Cascade Serving System for Large Language Models
• CVPR 2026Finding Distributed Object-Centric Properties in Self-Supervised Transformers
• NeurIPS 2025A Fair Federated Learning Method for Handling Client Participation Probability Inconsistencies in Heterogeneous Environments
• ICML 2025BSemiFL: Semi-supervised Federated Learning via a Bayesian Approach
• ICLR 2025Addressing Label Shift in Distributed Learning via Entropy Regularization
• CVPR 2025Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPs
• NeurIPS 2024Communication Efficient Distributed Training with Distributed Lion
• ICML 2024Embarrassingly Parallel GFlowNets
• ICLR 2024Benchmarking Algorithms for Federated Domain Generalization
• CVPR 2024Deciphering 'What' and 'Where' Visual Pathways from Spectral Clustering of Layer-Distributed Neural Representations
• NeurIPS 20232Direction: Theoretically Faster Distributed Training with Bidirectional Communication Compression
• ICML 2023Block Subsampled Randomized Hadamard Transform for Nyström Approximation on Distributed Architectures
• ICLR 2023Distributed Extra-gradient with Optimal Complexity and Communication Guarantees
• CVPR 2023DisCo-CLIP: A Distributed Contrastive Loss for Memory Efficient CLIP Training
• NeurIPS 2026Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods
• ICML 2026Clipping Makes Distributed and Federated Asynchronous SGD Robust to Stragglers
• ICLR 2026DualMap: Enabling Both Cache Affinity and Load Balancing for Distributed LLM Serving
• CVPR 2026Parallel Rigidity Matters for Bundle Adjustment

全部分布式训练论文

共 54 篇，第 1 / 2 页

1. NeurIPS 20232Direction: Theoretically Faster Distributed Training with Bidirectional Communication Compression
2. NeurIPS 2023A Computation and Communication Efficient Method for Distributed Nonconvex Problems in the Partial Participation Setting
3. NeurIPS 2023A Fast and Accurate Estimator for Large Scale Linear Model via Data Averaging
4. NeurIPS 2023A Hierarchical Training Paradigm for Antibody Structure-sequence Co-design
5. NeurIPS 2023AD-PT: Autonomous Driving Pre-Training with Large-scale Point Cloud Dataset
6. NeurIPS 2023Adversarial Training from Mean Field Perspective
7. NeurIPS 2023Adversarially Robust Distributed Count Tracking via Partial Differential Privacy
8. NeurIPS 2023American Stories: A Large-Scale Structured Text Dataset of Historical U.S. Newspapers
9. NeurIPS 2023An Inverse Scaling Law for CLIP Training
10. NeurIPS 2023Best Arm Identification with Fixed Budget: A Large Deviation Perspective
11. NeurIPS 2023Bullying10K: A Large-Scale Neuromorphic Dataset towards Privacy-Preserving Bullying Recognition
12. NeurIPS 2023Byzantine-Tolerant Methods for Distributed Variational Inequalities
13. NeurIPS 2023Clustering the Sketch: Dynamic Compression for Embedding Tables
14. NeurIPS 2023Connecting Certified and Adversarial Training
15. NeurIPS 2023Coordinating Distributed Example Orders for Provably Accelerated Training
16. NeurIPS 2023Correlation Aware Sparsified Mean Estimation Using Random Projection
17. NeurIPS 2023Defending against Data-Free Model Extraction by Distributionally Robust Defensive Training
18. NeurIPS 2023Delayed Algorithms for Distributed Stochastic Weakly Convex Optimization
19. NeurIPS 2023Direct Training of SNN using Local Zeroth Order Method
20. NeurIPS 2023Distributionally Robust Ensemble of Lottery Tickets Towards Calibrated Sparse Network Training
21. NeurIPS 2023DropCompute: simple and more robust distributed synchronous training via compute variance reduction
22. NeurIPS 2023Dynamic Sparsity Is Channel-Level Sparsity Learner
23. NeurIPS 2023Eliminating Catastrophic Overfitting Via Abnormal Adversarial Examples Regularization
24. NeurIPS 2023Exact Optimality of Communication-Privacy-Utility Tradeoffs in Distributed Mean Estimation
25. NeurIPS 2023Expanding Small-Scale Datasets with Guided Imagination
26. NeurIPS 2023H3T: Efficient Integration of Memory Optimization and Parallelism for Large-scale Transformer Training
27. NeurIPS 2023Langevin Quasi-Monte Carlo
28. NeurIPS 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias
29. NeurIPS 2023Large-Scale Distributed Learning via Private On-Device LSH
30. NeurIPS 2023Layer-Neighbor Sampling --- Defusing Neighborhood Explosion in GNNs
31. NeurIPS 2023Lockdown: Backdoor Defense for Federated Learning with Isolated Subspace Training
32. NeurIPS 2023Neural Combinatorial Optimization with Heavy Decoder: Toward Large Scale Generalization
33. NeurIPS 2023Objaverse-XL: A Universe of 10M+ 3D Objects
34. NeurIPS 2023OpenProteinSet: Training data for structural biology at scale
35. NeurIPS 2023Privacy Amplification via Compression: Achieving the Optimal Privacy-Accuracy-Communication Trade-off in Distributed Mean Estimation
36. NeurIPS 2023Privacy Auditing with One (1) Training Run
37. NeurIPS 2023Probabilistic Weight Fixing: Large-scale training of neural network weight uncertainties for quantisation.
38. NeurIPS 2023Pruning vs Quantization: Which is Better?
39. NeurIPS 2023Revisit the Power of Vanilla Knowledge Distillation: from Small Scale to Large Scale
40. NeurIPS 2023Robust Distributed Learning: Tight Error Bounds and Breakdown Point under Data Heterogeneity
41. NeurIPS 2023Scale-teaching: Robust Multi-scale Training for Time Series Classification with Noisy Labels
42. NeurIPS 2023Similarity, Compression and Local Steps: Three Pillars of Efficient Communications for Distributed Variational Inequalities
43. NeurIPS 2023SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling
44. NeurIPS 2023Stable Nonconvex-Nonconcave Training via Linear Interpolation
45. NeurIPS 2023Stochastic Distributed Optimization under Average Second-order Similarity: Algorithms and Analysis
46. NeurIPS 2023Structure-free Graph Condensation: From Large-scale Graphs to Condensed Graph-free Data
47. NeurIPS 2023TMT-VIS: Taxonomy-aware Multi-dataset Joint Training for Video Instance Segmentation
48. NeurIPS 2023Tools for Verifying Neural Models' Training Data
49. NeurIPS 2023Towards Accelerated Model Training via Bayesian Data Selection
50. NeurIPS 2023Training Transformers with 4-bit Integers

常见问题

2023–2026 年四大会议一共有多少篇分布式训练论文？

共 396 篇，按年份分别是 2023 年 95 篇、2024 年 74 篇、2025 年 129 篇、2026 年 98 篇。

NeurIPS 2026 有多少篇分布式训练论文？

NeurIPS 2026 有 36 篇分布式训练论文，上一届是 69 篇。

ICLR 2026 有多少篇分布式训练论文？

ICLR 2026 有 18 篇分布式训练论文，上一届是 18 篇。

ICML 2026 有多少篇分布式训练论文？

ICML 2026 有 42 篇分布式训练论文，上一届是 40 篇。

CVPR 2026 有多少篇分布式训练论文？

CVPR 2026 有 2 篇分布式训练论文，上一届是 2 篇。

哪个会议的分布式训练论文最多？

NeurIPS，2023–2026 年共 208 篇。