分布式训练论文趋势 2023–2026

主要结论

• 分布式训练论文从 2023 年的 95 篇增长到 2026 年的 98 篇，三年间扩大到约 1.0 倍。
• NeurIPS 是发分布式训练论文最多的会议，2023–2026 年共 208 篇，占四个会议总量的 53%。
• 交叉方向里涨得最快的是多模态大模型：同时属于两个主题的论文从 2023 年的 1 篇增至 2026 年的 10 篇。
• 在「效率与系统」的 5 个主题里，它的 2026 年论文数排第 4；同组增长最快的硬件协同是 4.4 倍，分布式训练是 1.0 倍。

代表论文

按年份和会议均匀挑选，方便快速看到每一届的样貌

• NeurIPS 2026AgentWeave: Efficient Distributed Agent Serving via Flow Decomposition
• ICML 2026Breaking the Capacity Bottleneck in Model-Heterogeneous Federated Learning via Gradual Model Restoration
• ICLR 2026Cascadia: An Efficient Cascade Serving System for Large Language Models
• CVPR 2026Finding Distributed Object-Centric Properties in Self-Supervised Transformers
• NeurIPS 2025A Fair Federated Learning Method for Handling Client Participation Probability Inconsistencies in Heterogeneous Environments
• ICML 2025BSemiFL: Semi-supervised Federated Learning via a Bayesian Approach
• ICLR 2025Addressing Label Shift in Distributed Learning via Entropy Regularization
• CVPR 2025Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPs
• NeurIPS 2024Communication Efficient Distributed Training with Distributed Lion
• ICML 2024Embarrassingly Parallel GFlowNets
• ICLR 2024Benchmarking Algorithms for Federated Domain Generalization
• CVPR 2024Deciphering 'What' and 'Where' Visual Pathways from Spectral Clustering of Layer-Distributed Neural Representations
• NeurIPS 20232Direction: Theoretically Faster Distributed Training with Bidirectional Communication Compression
• ICML 2023Block Subsampled Randomized Hadamard Transform for Nyström Approximation on Distributed Architectures
• ICLR 2023Distributed Extra-gradient with Optimal Complexity and Communication Guarantees
• CVPR 2023DisCo-CLIP: A Distributed Contrastive Loss for Memory Efficient CLIP Training
• NeurIPS 2026Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods
• ICML 2026Clipping Makes Distributed and Federated Asynchronous SGD Robust to Stragglers
• ICLR 2026DualMap: Enabling Both Cache Affinity and Load Balancing for Distributed LLM Serving
• CVPR 2026Parallel Rigidity Matters for Bundle Adjustment

全部分布式训练论文

共 40 篇，第 1 / 1 页

1. ICML 2025Achieving Linear Speedup and Near-Optimal Complexity for Decentralized Optimization over Row-stochastic Networks
2. ICML 2025Aequa: Fair Model Rewards in Collaborative Learning via Slimmable Networks
3. ICML 2025ATA: Adaptive Task Allocation for Efficient Resource Management in Distributed Machine Learning
4. ICML 2025Beyond Communication Overhead: A Multilevel Monte Carlo Approach for Mitigating Compression Bias in Distributed Learning
5. ICML 2025BSemiFL: Semi-supervised Federated Learning via a Bayesian Approach
6. ICML 2025Causality Inspired Federated Learning for OOD Generalization
7. ICML 2025Deep Streaming View Clustering
8. ICML 2025Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs
9. ICML 2025Distributed Conformal Prediction via Message Passing
10. ICML 2025Distributed Event-Based Learning via ADMM
11. ICML 2025Distributed Nonparametric Estimation: from Sparse to Dense Samples per Terminal
12. ICML 2025Distributed Parallel Gradient Stacking(DPGS): Solving Whole Slide Image Stacking Challenge in Multi-Instance Learning
13. ICML 2025Distributed Retraction-Free and Communication-Efficient Optimization on the Stiefel Manifold
14. ICML 2025DSP: Dynamic Sequence Parallelism for Multi-Dimensional Transformers
15. ICML 2025DTZO: Distributed Trilevel Zeroth Order Learning with Provable Non-Asymptotic Convergence
16. ICML 2025Efficient Distributed Optimization under Heavy-Tailed Noise
17. ICML 2025Efficiently Serving Large Multimodal Models Using EPD Disaggregation
18. ICML 2025Exploiting Similarity for Computation and Communication-Efficient Decentralized Optimization
19. ICML 2025FedOne: Query-Efficient Federated Learning for Black-box Discrete Prompt Learning
20. ICML 2025FSL-SAGE: Accelerating Federated Split Learning via Smashed Activation Gradient Estimation
21. ICML 2025HALoS: Hierarchical Asynchronous Local SGD over Slow Networks for Geo-Distributed Large Language Model Training
22. ICML 2025How Distributed Collaboration Influences the Diffusion Model Training? A Theoretical Perspective
23. ICML 2025Hybrid Batch Normalisation: Resolving the Dilemma of Batch Normalisation in Federated Learning
24. ICML 2025Improving Generalization in Federated Learning with Highly Heterogeneous Data via Momentum-Based Stochastic Controlled Weight Averaging
25. ICML 2025Layer-wise Quantization for Quantized Optimistic Dual Averaging
26. ICML 2025Learning curves theory for hierarchically compositional data with power-law distributed features
27. ICML 2025LV-XAttn: Distributed Cross-Attention for Long Visual Inputs in Multimodal Large Language Models
28. ICML 2025Nesterov Method for Asynchronous Pipeline Parallel Optimization
29. ICML 2025On Fine-Grained Distinct Element Estimation
30. ICML 2025On the Tension between Byzantine Robustness and No-Attack Accuracy in Distributed Learning
31. ICML 2025Provably Near-Optimal Federated Ensemble Distillation with Negligible Overhead
32. ICML 2025Riemannian Diffusion Adaptation for Distributed Optimization on Manifolds
33. ICML 2025Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity
34. ICML 2025Safe-EF: Error Feedback for Non-smooth Constrained Optimization
35. ICML 2025Scalable First-order Method for Certifying Optimal k-Sparse GLMs
36. ICML 2025Shortcut-connected Expert Parallelism for Accelerating Mixture of Experts
37. ICML 2025SPD: Sync-Point Drop for Efficient Tensor Parallelism of Large Language Models
38. ICML 2025The Panaceas for Improving Low-Rank Decomposition in Communication-Efficient Federated Learning
39. ICML 2025Unified Breakdown Analysis for Byzantine Robust Gossip
40. ICML 2025When, Where and Why to Average Weights?

常见问题

2023–2026 年四大会议一共有多少篇分布式训练论文？

共 396 篇，按年份分别是 2023 年 95 篇、2024 年 74 篇、2025 年 129 篇、2026 年 98 篇。

NeurIPS 2026 有多少篇分布式训练论文？

NeurIPS 2026 有 36 篇分布式训练论文，上一届是 69 篇。

ICLR 2026 有多少篇分布式训练论文？

ICLR 2026 有 18 篇分布式训练论文，上一届是 18 篇。

ICML 2026 有多少篇分布式训练论文？

ICML 2026 有 42 篇分布式训练论文，上一届是 40 篇。

CVPR 2026 有多少篇分布式训练论文？

CVPR 2026 有 2 篇分布式训练论文，上一届是 2 篇。

哪个会议的分布式训练论文最多？

NeurIPS，2023–2026 年共 208 篇。