分布式训练论文趋势 2023–2026

主要结论

• 分布式训练论文从 2023 年的 95 篇增长到 2026 年的 98 篇，三年间扩大到约 1.0 倍。
• NeurIPS 是发分布式训练论文最多的会议，2023–2026 年共 208 篇，占四个会议总量的 53%。
• 交叉方向里涨得最快的是多模态大模型：同时属于两个主题的论文从 2023 年的 1 篇增至 2026 年的 10 篇。
• 在「效率与系统」的 5 个主题里，它的 2026 年论文数排第 4；同组增长最快的硬件协同是 4.4 倍，分布式训练是 1.0 倍。

代表论文

按年份和会议均匀挑选，方便快速看到每一届的样貌

• NeurIPS 2026AgentWeave: Efficient Distributed Agent Serving via Flow Decomposition
• ICML 2026Breaking the Capacity Bottleneck in Model-Heterogeneous Federated Learning via Gradual Model Restoration
• ICLR 2026Cascadia: An Efficient Cascade Serving System for Large Language Models
• CVPR 2026Finding Distributed Object-Centric Properties in Self-Supervised Transformers
• NeurIPS 2025A Fair Federated Learning Method for Handling Client Participation Probability Inconsistencies in Heterogeneous Environments
• ICML 2025BSemiFL: Semi-supervised Federated Learning via a Bayesian Approach
• ICLR 2025Addressing Label Shift in Distributed Learning via Entropy Regularization
• CVPR 2025Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPs
• NeurIPS 2024Communication Efficient Distributed Training with Distributed Lion
• ICML 2024Embarrassingly Parallel GFlowNets
• ICLR 2024Benchmarking Algorithms for Federated Domain Generalization
• CVPR 2024Deciphering 'What' and 'Where' Visual Pathways from Spectral Clustering of Layer-Distributed Neural Representations
• NeurIPS 20232Direction: Theoretically Faster Distributed Training with Bidirectional Communication Compression
• ICML 2023Block Subsampled Randomized Hadamard Transform for Nyström Approximation on Distributed Architectures
• ICLR 2023Distributed Extra-gradient with Optimal Complexity and Communication Guarantees
• CVPR 2023DisCo-CLIP: A Distributed Contrastive Loss for Memory Efficient CLIP Training
• NeurIPS 2026Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods
• ICML 2026Clipping Makes Distributed and Federated Asynchronous SGD Robust to Stragglers
• ICLR 2026DualMap: Enabling Both Cache Affinity and Load Balancing for Distributed LLM Serving
• CVPR 2026Parallel Rigidity Matters for Bundle Adjustment

全部分布式训练论文

共 74 篇，第 2 / 2 页

51. NeurIPS 2024FM-Delta: Lossless Compression for Storing Massive Fine-tuned Foundation Models
52. NeurIPS 2024Fully Distributed, Flexible Compositional Visual Representations via Soft Tensor Products
53. NeurIPS 2024HyperPrism: An Adaptive Non-linear Aggregation Framework for Distributed Machine Learning over Non-IID Data and Time-varying Communication Links
54. NeurIPS 2024Improving the Worst-Case Bidirectional Communication Complexity for Nonconvex Distributed Optimization under Function Similarity
55. NeurIPS 2024Leveraging partial stragglers within gradient coding
56. NeurIPS 2024Loki: Low-rank Keys for Efficient Sparse Attention
57. NeurIPS 2024LoRANN: Low-Rank Matrix Factorization for Approximate Nearest Neighbor Search
58. NeurIPS 2024LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing
59. NeurIPS 2024Mini-Sequence Transformers: Optimizing Intermediate Memory for Long Sequences Training
60. NeurIPS 2024Near-Optimal Distributed Minimax Optimization under the Second-Order Similarity
61. NeurIPS 2024Optimal Private and Communication Constraint Distributed Goodness-of-Fit Testing for Discrete Distributions in the Large Sample Regime
62. NeurIPS 2024OxonFair: A Flexible Toolkit for Algorithmic Fairness
63. NeurIPS 2024pFedClub: Controllable Heterogeneous Model Aggregation for Personalized Federated Learning
64. NeurIPS 2024Pipeline Parallelism with Controllable Memory
65. NeurIPS 2024RandNet-Parareal: a time-parallel PDE solver using Random Neural Networks
66. NeurIPS 2024Rethinking Memory and Communication Costs for Efficient Data Parallel Training of Large Language Models
67. NeurIPS 2024SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM Training
68. NeurIPS 2024SGLang: Efficient Execution of Structured Language Model Programs
69. NeurIPS 2024Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
70. NeurIPS 2024Sketching for Distributed Deep Learning: A Sharper Analysis
71. NeurIPS 2024SpeedLoader: An I/O efficient scheme for heterogeneous and distributed LLM operation
72. NeurIPS 2024Temporal-Difference Learning Using Distributed Error Signals
73. NeurIPS 2024TinyTTA: Efficient Test-time Adaptation via Early-exit Ensembles on Edge Devices
74. NeurIPS 2024Toward Efficient Inference for Mixture of Experts

常见问题

2023–2026 年四大会议一共有多少篇分布式训练论文？

共 396 篇，按年份分别是 2023 年 95 篇、2024 年 74 篇、2025 年 129 篇、2026 年 98 篇。

NeurIPS 2026 有多少篇分布式训练论文？

NeurIPS 2026 有 36 篇分布式训练论文，上一届是 69 篇。

ICLR 2026 有多少篇分布式训练论文？

ICLR 2026 有 18 篇分布式训练论文，上一届是 18 篇。

ICML 2026 有多少篇分布式训练论文？

ICML 2026 有 42 篇分布式训练论文，上一届是 40 篇。

CVPR 2026 有多少篇分布式训练论文？

CVPR 2026 有 2 篇分布式训练论文，上一届是 2 篇。

哪个会议的分布式训练论文最多？

NeurIPS，2023–2026 年共 208 篇。