Deep learning · optimization papers, 2023–2026

Key findings

• Deep learning · optimization fell from 256 papers in 2023 to 240 in 2026, so interest is cooling.
• NeurIPS publishes the most Deep learning · optimization work: 500 papers from 2023 to 2026, 52% of the four-venue total.
• The fastest-growing overlap is with Alignment and Safety: papers tagged with both rose from 5 in 2023 to 23 in 2026.
• Among the 11 topics under Deep Learning, it ranks #6 by 2026 paper count; the fastest grower in the group, Diffusion Models, is up 4.1x versus 0.9x here.

Representative papers

Spread across years and venues so every edition shows up

• NeurIPS 2026Achieving Directional-Stationarity from a Single Random Direction Step
• ICML 2026Bottleneck Communication Delay Minimization for Communication-Efficient Decentralized Learning
• ICLR 2026From Sorting Algorithms to Scalable Kernels: Bayesian Optimization in High-Dimensional Permutation Spaces
• CVPR 2026Coverage Optimization for Camera View Selection
• NeurIPS 2025High-Performance Arithmetic Circuit Optimization via Differentiable Architecture Search
• ICML 2025Feature-Mapping Topology Optimization with Neural Heaviside Signed Distance Functions
• ICLR 2025Boosting Neural Combinatorial Optimization for Large-Scale Vehicle Routing Problems
• CVPR 2025HotSpot: Signed Distance Function Optimization with an Asymptotically Sufficient Condition
• NeurIPS 2024$\boldsymbol{\mu}\mathbf{P^2}$: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling
• ICML 2024Barrier Algorithms for Constrained Non-Convex Optimization
• ICLR 2024Bayesian Coreset Optimization for Personalized Federated Learning
• CVPR 2024Multi-Session SLAM with Differentiable Wide-Baseline Pose Optimization
• NeurIPS 2023$\textbf{A}^2\textbf{CiD}^2$: Accelerating Asynchronous Communication in Decentralized Deep Learning
• ICML 2023Buying Information for Stochastic Optimization
• ICLR 2023Alternating Differentiation for Optimization Layers
• CVPR 2023Visual-Language Prompt Tuning With Knowledge-Guided Context Optimization
• NeurIPS 2026Adaptive Conditional Gradient Sliding: Projection-Free and Line-Search-Free Acceleration
• ICML 2026Riemannian Optimization for Fair Spectral Clustering
• ICLR 2026Gradient-Based Diversity Optimization with Differentiable Top-$k$ Objective
• CVPR 2026Its Never Too Late Noise Optimization for Collapse Recovery in

All Deep learning · optimization papers

191 papers, page 3 of 4

101. NeurIPS 2024LoRA-GA: Low-Rank Adaptation with Gradient Approximation
102. NeurIPS 2024Loss Landscape Characterization of Neural Networks without Over-Parametrization
103. NeurIPS 2024Low Precision Local Training is Enough for Federated Learning
104. NeurIPS 2024Make Continual Learning Stronger via C-Flat
105. NeurIPS 2024Matching the Statistical Query Lower Bound for $k$-Sparse Parity Problems with Sign Stochastic Gradient Descent
106. NeurIPS 2024Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimization
107. NeurIPS 2024Memory-Efficient LLM Training with Online Subspace Descent
108. NeurIPS 2024MG-Net: Learn to Customize QAOA with Circuit Depth Awareness
109. NeurIPS 2024MicroAdam: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence
110. NeurIPS 2024Model Fusion through Bayesian Optimization in Language Model Fine-Tuning
111. NeurIPS 2024MOTE-NAS: Multi-Objective Training-based Estimate for Efficient Neural Architecture Search
112. NeurIPS 2024Navigating Extremes: Dynamic Sparsity in Large Output Spaces
113. NeurIPS 2024Nearly Minimax Optimal Submodular Maximization with Bandit Feedback
114. NeurIPS 2024Neur2BiLO: Neural Bilevel Optimization
115. NeurIPS 2024Neural Characteristic Activation Analysis and Geometric Parameterization for ReLU Networks
116. NeurIPS 2024Neural collapse vs. low-rank bias: Is deep neural collapse really optimal?
117. NeurIPS 2024Newton Losses: Using Curvature Information for Learning with Differentiable Algorithms
118. NeurIPS 2024Non-geodesically-convex optimization in the Wasserstein space
119. NeurIPS 2024Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset
120. NeurIPS 2024Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers
121. NeurIPS 2024Nuclear Norm Regularization for Deep Learning
122. NeurIPS 2024Offline Behavior Distillation
123. NeurIPS 2024On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions
124. NeurIPS 2024On the Inductive Bias of Stacking Towards Improving Reasoning
125. NeurIPS 2024Optimization-based Causal Estimation from Heterogeneous Environments
126. NeurIPS 2024Outlier-Robust Distributionally Robust Optimization via Unbalanced Optimal Transport
127. NeurIPS 2024PACE: Marrying generalization in PArameter-efficient fine-tuning with Consistency rEgularization
128. NeurIPS 2024Penalty-based Methods for Simple Bilevel Optimization under Hölderian Error Bounds
129. NeurIPS 2024Personalized Federated Learning via Feature Distribution Adaptation
130. NeurIPS 2024PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models
131. NeurIPS 2024Pre-trained Gaussian Processes for Bayesian Optimization
132. NeurIPS 2024Quantum Algorithms for Non-smooth Non-convex Optimization
133. NeurIPS 2024Recurrent neural networks: vanishing and exploding gradients are not the end of the story
134. NeurIPS 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution
135. NeurIPS 2024Regularized Adaptive Momentum Dual Averaging with an Efficient Inexact Subproblem Solver for Training Structured Neural Network
136. NeurIPS 2024ReLIZO: Sample Reusable Linear Interpolation-based Zeroth-order Optimization
137. NeurIPS 2024Reshuffling Resampling Splits Can Improve Generalization of Hyperparameter Optimization
138. NeurIPS 2024Resolving Discrepancies in Compute-Optimal Scaling of Language Models
139. NeurIPS 2024Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models
140. NeurIPS 2024Revive Re-weighting in Imbalanced Learning by Density Ratio Estimation
141. NeurIPS 2024S-SOS: Stochastic Sum-Of-Squares for Parametric Polynomial Optimization
142. NeurIPS 2024S-STE: Continuous Pruning Function for Efficient 2:4 Sparse Pre-training
143. NeurIPS 2024SAMPa: Sharpness-aware Minimization Parallelized
144. NeurIPS 2024Scalable Kernel Inverse Optimization
145. NeurIPS 2024Scalable Optimization in the Modular Norm
146. NeurIPS 2024ScaleKD: Strong Vision Transformers Could Be Excellent Teachers
147. NeurIPS 2024Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations
148. NeurIPS 2024Semidefinite Relaxations of the Gromov-Wasserstein Distance
149. NeurIPS 2024SequentialAttention++ for Block Sparsification: Differentiable Pruning Meets Combinatorial Optimization
150. NeurIPS 2024SGD vs GD: Rank Deficiency in Linear Networks

FAQ

How many Deep learning · optimization papers were published at NeurIPS, ICLR, ICML and CVPR from 2023 to 2026?

953 in total: 256 in 2023, 294 in 2024, 163 in 2025, 240 in 2026.

How many Deep learning · optimization papers are at NeurIPS 2026?

NeurIPS 2026 has 103 Deep learning · optimization papers, versus 62 the year before.

How many Deep learning · optimization papers are at ICLR 2026?

ICLR 2026 has 44 Deep learning · optimization papers, versus 51 the year before.

How many Deep learning · optimization papers are at ICML 2026?

ICML 2026 has 86 Deep learning · optimization papers, versus 44 the year before.

How many Deep learning · optimization papers are at CVPR 2026?

CVPR 2026 has 7 Deep learning · optimization papers, versus 6 the year before.

Which conference publishes the most Deep learning · optimization papers?

NeurIPS, with 500 papers from 2023 to 2026.