分布式训练论文趋势 2023–2026

主要结论

• 分布式训练论文从 2023 年的 95 篇增长到 2026 年的 98 篇，三年间扩大到约 1.0 倍。
• NeurIPS 是发分布式训练论文最多的会议，2023–2026 年共 208 篇，占四个会议总量的 53%。
• 交叉方向里涨得最快的是多模态大模型：同时属于两个主题的论文从 2023 年的 1 篇增至 2026 年的 10 篇。
• 在「效率与系统」的 5 个主题里，它的 2026 年论文数排第 4；同组增长最快的硬件协同是 4.4 倍，分布式训练是 1.0 倍。

代表论文

按年份和会议均匀挑选，方便快速看到每一届的样貌

• NeurIPS 2026AgentWeave: Efficient Distributed Agent Serving via Flow Decomposition
• ICML 2026Breaking the Capacity Bottleneck in Model-Heterogeneous Federated Learning via Gradual Model Restoration
• ICLR 2026Cascadia: An Efficient Cascade Serving System for Large Language Models
• CVPR 2026Finding Distributed Object-Centric Properties in Self-Supervised Transformers
• NeurIPS 2025A Fair Federated Learning Method for Handling Client Participation Probability Inconsistencies in Heterogeneous Environments
• ICML 2025BSemiFL: Semi-supervised Federated Learning via a Bayesian Approach
• ICLR 2025Addressing Label Shift in Distributed Learning via Entropy Regularization
• CVPR 2025Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPs
• NeurIPS 2024Communication Efficient Distributed Training with Distributed Lion
• ICML 2024Embarrassingly Parallel GFlowNets
• ICLR 2024Benchmarking Algorithms for Federated Domain Generalization
• CVPR 2024Deciphering 'What' and 'Where' Visual Pathways from Spectral Clustering of Layer-Distributed Neural Representations
• NeurIPS 20232Direction: Theoretically Faster Distributed Training with Bidirectional Communication Compression
• ICML 2023Block Subsampled Randomized Hadamard Transform for Nyström Approximation on Distributed Architectures
• ICLR 2023Distributed Extra-gradient with Optimal Complexity and Communication Guarantees
• CVPR 2023DisCo-CLIP: A Distributed Contrastive Loss for Memory Efficient CLIP Training
• NeurIPS 2026Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods
• ICML 2026Clipping Makes Distributed and Federated Asynchronous SGD Robust to Stragglers
• ICLR 2026DualMap: Enabling Both Cache Affinity and Load Balancing for Distributed LLM Serving
• CVPR 2026Parallel Rigidity Matters for Bundle Adjustment

全部分布式训练论文

共 36 篇，第 1 / 1 页

1. NeurIPS 2026Adversary-Robust Learning from Fully Asynchronous Directional Derivative Estimates
2. NeurIPS 2026AgentWeave: Efficient Distributed Agent Serving via Flow Decomposition
3. NeurIPS 2026BReD: Block Replay Dithering for Stable Low-Bit EMA Optimizer States
4. NeurIPS 2026Can Model Merging Improve Aggregation in DiLoCo?
5. NeurIPS 2026Clapping: Removing Per-sample Storage for Pipeline Parallel Learning with Communication Compression
6. NeurIPS 2026CRAFT: Conflict-Resolved Aggregation for Federated Training
7. NeurIPS 2026DDBench: A Benchmark for Agentic Debugging on Distributed Systems
8. NeurIPS 2026Delightful Distributed Policy Gradient
9. NeurIPS 2026Dependency-Guided Parallel Decoding in Discrete Diffusion Language Models
10. NeurIPS 2026DISTMOE: Rehearsal-free Routing in Mixture-of-Experts for Distributed Instruction Tuning
11. NeurIPS 2026Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods
12. NeurIPS 2026ECHO-2: A Large-Scale Distributed Rollout Framework for Cost-Efficient Reinforcement Learning
13. NeurIPS 2026Efficient Algorithms for Distributed Saddle Problems
14. NeurIPS 2026Federated Concept-Based Models: Interpretable models with distributed supervision
15. NeurIPS 2026FedPeel: Peeling Stabilized Layers for Robust Heterogeneous Federated Learning
16. NeurIPS 2026FlashMask-3: Efficient and Expressive Mask-Aware Distributed Attention
17. NeurIPS 2026Focus on Where You Aggregate: Restricted SAM for Non-IID Federated Learning
18. NeurIPS 2026FRESCO: A Novel Consistency Control for Asynchronous Pipeline Parallel Training
19. NeurIPS 2026Fully Distributed Tâtonnement for Chores Markets
20. NeurIPS 2026GPU-Accelerated Synthesis of Mixed-Boolean Arithmetic: Beyond Caching
21. NeurIPS 2026HetCCL: Efficient LLM Training on Heterogeneous Vendor GPUs
22. NeurIPS 2026In STeP: Speculative Tensor Parallelism for Concurrent Heterogeneous Inference of LLMs
23. NeurIPS 2026LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging
24. NeurIPS 2026Measuring Layer-wise Intrinsic Dimensionality of FFNs in LLMs via PCA
25. NeurIPS 2026Mitigating Data Heterogeneity Effect in Client-Reshuffling-Based Federated Learning
26. NeurIPS 2026nnTrace: Detecting and Localizing Silent Bugs in Distributed Training
27. NeurIPS 2026On the Parallel Optimality of Exponentiated Gradient Descent
28. NeurIPS 2026P-EAGLE: Parallel-Drafting EAGLE with Scalable Training
29. NeurIPS 2026Principled Federated Random Forests for Heterogeneous Data
30. NeurIPS 2026Privacy Meets Hierarchy: Differentially Private Distributed Trilevel Learning
31. NeurIPS 2026ReCoVer: Resilient LLM Pre-Training System via Fault-Tolerant Collective and Versatile Workload
32. NeurIPS 2026Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity
33. NeurIPS 2026Scalable Distributed Stochastic Optimization via Bidirectional Compression: Beyond Pessimistic Limits
34. NeurIPS 2026SMMBench: A Benchmark for Source-Distributed Multimodal Agent Memory
35. NeurIPS 2026vExpert: Virtualizing Expert Storage for Adaptive Load Balancing in Distributed MoE Inference
36. NeurIPS 2026When Must AI Training Stage Checkpoints? A Distributional Model of Durability Boundaries at Scale

常见问题

2023–2026 年四大会议一共有多少篇分布式训练论文？

共 396 篇，按年份分别是 2023 年 95 篇、2024 年 74 篇、2025 年 129 篇、2026 年 98 篇。

NeurIPS 2026 有多少篇分布式训练论文？

NeurIPS 2026 有 36 篇分布式训练论文，上一届是 69 篇。

ICLR 2026 有多少篇分布式训练论文？

ICLR 2026 有 18 篇分布式训练论文，上一届是 18 篇。

ICML 2026 有多少篇分布式训练论文？

ICML 2026 有 42 篇分布式训练论文，上一届是 40 篇。

CVPR 2026 有多少篇分布式训练论文？

CVPR 2026 有 2 篇分布式训练论文，上一届是 2 篇。

哪个会议的分布式训练论文最多？

NeurIPS，2023–2026 年共 208 篇。