Distributed Training papers, 2023–2026

Key findings

• Distributed Training went from 95 papers in 2023 to 98 in 2026, roughly 1.0x in three years.
• NeurIPS publishes the most Distributed Training work: 208 papers from 2023 to 2026, 53% of the four-venue total.
• The fastest-growing overlap is with Multimodal LLMs: papers tagged with both rose from 1 in 2023 to 10 in 2026.
• Among the 5 topics under Efficiency and Systems, it ranks #4 by 2026 paper count; the fastest grower in the group, Hardware-Aware ML, is up 4.4x versus 1.0x here.

Representative papers

Spread across years and venues so every edition shows up

• NeurIPS 2026AgentWeave: Efficient Distributed Agent Serving via Flow Decomposition
• ICML 2026Breaking the Capacity Bottleneck in Model-Heterogeneous Federated Learning via Gradual Model Restoration
• ICLR 2026Cascadia: An Efficient Cascade Serving System for Large Language Models
• CVPR 2026Finding Distributed Object-Centric Properties in Self-Supervised Transformers
• NeurIPS 2025A Fair Federated Learning Method for Handling Client Participation Probability Inconsistencies in Heterogeneous Environments
• ICML 2025BSemiFL: Semi-supervised Federated Learning via a Bayesian Approach
• ICLR 2025Addressing Label Shift in Distributed Learning via Entropy Regularization
• CVPR 2025Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPs
• NeurIPS 2024Communication Efficient Distributed Training with Distributed Lion
• ICML 2024Embarrassingly Parallel GFlowNets
• ICLR 2024Benchmarking Algorithms for Federated Domain Generalization
• CVPR 2024Deciphering 'What' and 'Where' Visual Pathways from Spectral Clustering of Layer-Distributed Neural Representations
• NeurIPS 20232Direction: Theoretically Faster Distributed Training with Bidirectional Communication Compression
• ICML 2023Block Subsampled Randomized Hadamard Transform for Nyström Approximation on Distributed Architectures
• ICLR 2023Distributed Extra-gradient with Optimal Complexity and Communication Guarantees
• CVPR 2023DisCo-CLIP: A Distributed Contrastive Loss for Memory Efficient CLIP Training
• NeurIPS 2026Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods
• ICML 2026Clipping Makes Distributed and Federated Asynchronous SGD Robust to Stragglers
• ICLR 2026DualMap: Enabling Both Cache Affinity and Load Balancing for Distributed LLM Serving
• CVPR 2026Parallel Rigidity Matters for Bundle Adjustment

All Distributed Training papers

54 papers, page 1 of 2

1. NeurIPS 20232Direction: Theoretically Faster Distributed Training with Bidirectional Communication Compression
2. NeurIPS 2023A Computation and Communication Efficient Method for Distributed Nonconvex Problems in the Partial Participation Setting
3. NeurIPS 2023A Fast and Accurate Estimator for Large Scale Linear Model via Data Averaging
4. NeurIPS 2023A Hierarchical Training Paradigm for Antibody Structure-sequence Co-design
5. NeurIPS 2023AD-PT: Autonomous Driving Pre-Training with Large-scale Point Cloud Dataset
6. NeurIPS 2023Adversarial Training from Mean Field Perspective
7. NeurIPS 2023Adversarially Robust Distributed Count Tracking via Partial Differential Privacy
8. NeurIPS 2023American Stories: A Large-Scale Structured Text Dataset of Historical U.S. Newspapers
9. NeurIPS 2023An Inverse Scaling Law for CLIP Training
10. NeurIPS 2023Best Arm Identification with Fixed Budget: A Large Deviation Perspective
11. NeurIPS 2023Bullying10K: A Large-Scale Neuromorphic Dataset towards Privacy-Preserving Bullying Recognition
12. NeurIPS 2023Byzantine-Tolerant Methods for Distributed Variational Inequalities
13. NeurIPS 2023Clustering the Sketch: Dynamic Compression for Embedding Tables
14. NeurIPS 2023Connecting Certified and Adversarial Training
15. NeurIPS 2023Coordinating Distributed Example Orders for Provably Accelerated Training
16. NeurIPS 2023Correlation Aware Sparsified Mean Estimation Using Random Projection
17. NeurIPS 2023Defending against Data-Free Model Extraction by Distributionally Robust Defensive Training
18. NeurIPS 2023Delayed Algorithms for Distributed Stochastic Weakly Convex Optimization
19. NeurIPS 2023Direct Training of SNN using Local Zeroth Order Method
20. NeurIPS 2023Distributionally Robust Ensemble of Lottery Tickets Towards Calibrated Sparse Network Training
21. NeurIPS 2023DropCompute: simple and more robust distributed synchronous training via compute variance reduction
22. NeurIPS 2023Dynamic Sparsity Is Channel-Level Sparsity Learner
23. NeurIPS 2023Eliminating Catastrophic Overfitting Via Abnormal Adversarial Examples Regularization
24. NeurIPS 2023Exact Optimality of Communication-Privacy-Utility Tradeoffs in Distributed Mean Estimation
25. NeurIPS 2023Expanding Small-Scale Datasets with Guided Imagination
26. NeurIPS 2023H3T: Efficient Integration of Memory Optimization and Parallelism for Large-scale Transformer Training
27. NeurIPS 2023Langevin Quasi-Monte Carlo
28. NeurIPS 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias
29. NeurIPS 2023Large-Scale Distributed Learning via Private On-Device LSH
30. NeurIPS 2023Layer-Neighbor Sampling --- Defusing Neighborhood Explosion in GNNs
31. NeurIPS 2023Lockdown: Backdoor Defense for Federated Learning with Isolated Subspace Training
32. NeurIPS 2023Neural Combinatorial Optimization with Heavy Decoder: Toward Large Scale Generalization
33. NeurIPS 2023Objaverse-XL: A Universe of 10M+ 3D Objects
34. NeurIPS 2023OpenProteinSet: Training data for structural biology at scale
35. NeurIPS 2023Privacy Amplification via Compression: Achieving the Optimal Privacy-Accuracy-Communication Trade-off in Distributed Mean Estimation
36. NeurIPS 2023Privacy Auditing with One (1) Training Run
37. NeurIPS 2023Probabilistic Weight Fixing: Large-scale training of neural network weight uncertainties for quantisation.
38. NeurIPS 2023Pruning vs Quantization: Which is Better?
39. NeurIPS 2023Revisit the Power of Vanilla Knowledge Distillation: from Small Scale to Large Scale
40. NeurIPS 2023Robust Distributed Learning: Tight Error Bounds and Breakdown Point under Data Heterogeneity
41. NeurIPS 2023Scale-teaching: Robust Multi-scale Training for Time Series Classification with Noisy Labels
42. NeurIPS 2023Similarity, Compression and Local Steps: Three Pillars of Efficient Communications for Distributed Variational Inequalities
43. NeurIPS 2023SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling
44. NeurIPS 2023Stable Nonconvex-Nonconcave Training via Linear Interpolation
45. NeurIPS 2023Stochastic Distributed Optimization under Average Second-order Similarity: Algorithms and Analysis
46. NeurIPS 2023Structure-free Graph Condensation: From Large-scale Graphs to Condensed Graph-free Data
47. NeurIPS 2023TMT-VIS: Taxonomy-aware Multi-dataset Joint Training for Video Instance Segmentation
48. NeurIPS 2023Tools for Verifying Neural Models' Training Data
49. NeurIPS 2023Towards Accelerated Model Training via Bayesian Data Selection
50. NeurIPS 2023Training Transformers with 4-bit Integers

FAQ

How many Distributed Training papers were published at NeurIPS, ICLR, ICML and CVPR from 2023 to 2026?

396 in total: 95 in 2023, 74 in 2024, 129 in 2025, 98 in 2026.

How many Distributed Training papers are at NeurIPS 2026?

NeurIPS 2026 has 36 Distributed Training papers, versus 69 the year before.

How many Distributed Training papers are at ICLR 2026?

ICLR 2026 has 18 Distributed Training papers, versus 18 the year before.

How many Distributed Training papers are at ICML 2026?

ICML 2026 has 42 Distributed Training papers, versus 40 the year before.

How many Distributed Training papers are at CVPR 2026?

CVPR 2026 has 2 Distributed Training papers, versus 2 the year before.

Which conference publishes the most Distributed Training papers?

NeurIPS, with 208 papers from 2023 to 2026.