分布式训练论文趋势 2023–2026

主要结论

• 分布式训练论文从 2023 年的 95 篇增长到 2026 年的 98 篇，三年间扩大到约 1.0 倍。
• NeurIPS 是发分布式训练论文最多的会议，2023–2026 年共 208 篇，占四个会议总量的 53%。
• 交叉方向里涨得最快的是多模态大模型：同时属于两个主题的论文从 2023 年的 1 篇增至 2026 年的 10 篇。
• 在「效率与系统」的 5 个主题里，它的 2026 年论文数排第 4；同组增长最快的硬件协同是 4.4 倍，分布式训练是 1.0 倍。

代表论文

按年份和会议均匀挑选，方便快速看到每一届的样貌

• NeurIPS 2026AgentWeave: Efficient Distributed Agent Serving via Flow Decomposition
• ICML 2026Breaking the Capacity Bottleneck in Model-Heterogeneous Federated Learning via Gradual Model Restoration
• ICLR 2026Cascadia: An Efficient Cascade Serving System for Large Language Models
• CVPR 2026Finding Distributed Object-Centric Properties in Self-Supervised Transformers
• NeurIPS 2025A Fair Federated Learning Method for Handling Client Participation Probability Inconsistencies in Heterogeneous Environments
• ICML 2025BSemiFL: Semi-supervised Federated Learning via a Bayesian Approach
• ICLR 2025Addressing Label Shift in Distributed Learning via Entropy Regularization
• CVPR 2025Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPs
• NeurIPS 2024Communication Efficient Distributed Training with Distributed Lion
• ICML 2024Embarrassingly Parallel GFlowNets
• ICLR 2024Benchmarking Algorithms for Federated Domain Generalization
• CVPR 2024Deciphering 'What' and 'Where' Visual Pathways from Spectral Clustering of Layer-Distributed Neural Representations
• NeurIPS 20232Direction: Theoretically Faster Distributed Training with Bidirectional Communication Compression
• ICML 2023Block Subsampled Randomized Hadamard Transform for Nyström Approximation on Distributed Architectures
• ICLR 2023Distributed Extra-gradient with Optimal Complexity and Communication Guarantees
• CVPR 2023DisCo-CLIP: A Distributed Contrastive Loss for Memory Efficient CLIP Training
• NeurIPS 2026Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods
• ICML 2026Clipping Makes Distributed and Federated Asynchronous SGD Robust to Stragglers
• ICLR 2026DualMap: Enabling Both Cache Affinity and Load Balancing for Distributed LLM Serving
• CVPR 2026Parallel Rigidity Matters for Bundle Adjustment

全部分布式训练论文

共 69 篇，第 1 / 2 页

1. NeurIPS 2025A Fair Federated Learning Method for Handling Client Participation Probability Inconsistencies in Heterogeneous Environments
2. NeurIPS 2025Accelerating Parallel Diffusion Model Serving with Residual Compression
3. NeurIPS 2025ACCO: Accumulate While You Communicate for Communication-Overlapped Sharded LLM Training
4. NeurIPS 2025Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agents
5. NeurIPS 2025Approximate Gradient Coding for Distributed Learning with Heterogeneous Stragglers
6. NeurIPS 2025AREAL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning
7. NeurIPS 2025ASDSV: Multimodal Generation Made Efficient with Approximate Speculative Diffusion and Speculative Verification
8. NeurIPS 2025Block-Diagonal LoRA for Eliminating Communication Overhead in Tensor Parallel LoRA Serving
9. NeurIPS 2025Channel Simulation and Distributed Compression with Ensemble Rejection Sampling
10. NeurIPS 2025ClusterFusion: Expanding Operator Fusion Scope for LLM Inference via Cluster-Level Collective Primitive
11. NeurIPS 2025Communication-Efficient Diffusion Denoising Parallelization via Reuse-then-Predict Mechanism
12. NeurIPS 2025Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo
13. NeurIPS 2025Contextual Tokenization for Graph Inverted Indices
14. NeurIPS 2025Cost-Efficient LLM Training with Lifetime-Aware Tensor Offloading via GPUDirect Storage
15. NeurIPS 2025Distance Adaptive Beam Search for Provably Accurate Graph-Based Nearest Neighbor Search
16. NeurIPS 2025Distributed mediation analysis with communication efficiency
17. NeurIPS 2025Distributed Multi-Agent Bandits Over Erdős-Rényi Random Networks
18. NeurIPS 2025DUO: No Compromise to Accuracy Degradation
19. NeurIPS 2025DynaPipe: Dynamic Layer Redistribution for Efficient Serving of LLMs with Pipeline Parallelism
20. NeurIPS 2025Efficient Pre-Training of LLMs via Topology-Aware Communication Alignment on More Than 9600 GPUs
21. NeurIPS 2025Efficient Training-Free Online Routing for High-Volume Multi-LLM Serving
22. NeurIPS 2025ElasticMM: Efficient Multimodal LLMs Serving with Elastic Multimodal Parallelism
23. NeurIPS 2025EnCompass: Enhancing Agent Programming with Search Over Program Execution Paths
24. NeurIPS 2025Exploiting Dynamic Sparsity in Einsum
25. NeurIPS 2025FedLPA: Local Prior Alignment for Heterogeneous Federated Generalized Category Discovery
26. NeurIPS 2025First Attentions Last: Better Exploiting First Attentions for Efficient Parallel Training
27. NeurIPS 2025FlashMoE: Fast Distributed MoE in a Single Kernel
28. NeurIPS 2025FlowMoE: A Scalable Pipeline Scheduling Framework for Distributed Mixture-of-Experts Training
29. NeurIPS 2025Foresight: Adaptive Layer Reuse for Accelerated and High-Quality Text-to-Video Generation
30. NeurIPS 2025Gatekeeper: Improving Model Cascades Through Confidence Tuning
31. NeurIPS 2025Generative Caching for Structurally Similar Prompts and Responses
32. NeurIPS 2025HyGen: Efficient LLM Serving via Elastic Online-Offline Request Co-location
33. NeurIPS 2025InfantAgent-Next: A Multimodal Generalist Agent for Automated Computer Interaction
34. NeurIPS 2025KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows
35. NeurIPS 2025Loquetier: A Virtualized Multi-LoRA Framework for Unified LLM Fine-tuning and Serving
36. NeurIPS 2025MeCeFO: Enhancing LLM Training Robustness via Fault-Tolerant Optimization
37. NeurIPS 2025Mixtures of Subspaces for Bandwidth Efficient Context Parallel Training
38. NeurIPS 2025Mozart: Modularized and Efficient MoE Training on 3.5D Wafer-Scale Chiplet Architectures
39. NeurIPS 2025msf-CNN: Patch-based Multi-Stage Fusion with Convolutional Neural Networks for TinyML
40. NeurIPS 2025MUSTAFAR: Promoting Unstructured Sparsity for KV Cache Pruning in LLM Inference
41. NeurIPS 2025NestedFP: High-Performance, Memory-Efficient Dual-Precision Floating Point Support for LLMs
42. NeurIPS 2025PipeFusion: Patch-level Pipeline Parallelism for Diffusion Transformers Inference
43. NeurIPS 2025Progressive Data Dropout: An Embarrassingly Simple Approach to Train Faster
44. NeurIPS 2025PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecture
45. NeurIPS 2025Quartet: Native FP4 Training Can Be Optimal for Large Language Models
46. NeurIPS 2025REASONING COMPILER: LLM-Guided Optimizations for Efficient Model Serving
47. NeurIPS 2025Refinement Methods for Distributed Distribution Estimation under $\ell^p$-Losses
48. NeurIPS 2025Repo2Run: Automated Building Executable Environment for Code Repository at Scale
49. NeurIPS 2025Robust Distributed Estimation: Extending Gossip Algorithms to Ranking and Trimmed Means
50. NeurIPS 2025SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training

常见问题

2023–2026 年四大会议一共有多少篇分布式训练论文？

共 396 篇，按年份分别是 2023 年 95 篇、2024 年 74 篇、2025 年 129 篇、2026 年 98 篇。

NeurIPS 2026 有多少篇分布式训练论文？

NeurIPS 2026 有 36 篇分布式训练论文，上一届是 69 篇。

ICLR 2026 有多少篇分布式训练论文？

ICLR 2026 有 18 篇分布式训练论文，上一届是 18 篇。

ICML 2026 有多少篇分布式训练论文？

ICML 2026 有 42 篇分布式训练论文，上一届是 40 篇。

CVPR 2026 有多少篇分布式训练论文？

CVPR 2026 有 2 篇分布式训练论文，上一届是 2 篇。

哪个会议的分布式训练论文最多？

NeurIPS，2023–2026 年共 208 篇。