Deep learning · optimization papers, 2023–2026

Key findings

• Deep learning · optimization fell from 256 papers in 2023 to 240 in 2026, so interest is cooling.
• NeurIPS publishes the most Deep learning · optimization work: 500 papers from 2023 to 2026, 52% of the four-venue total.
• The fastest-growing overlap is with Alignment and Safety: papers tagged with both rose from 5 in 2023 to 23 in 2026.
• Among the 11 topics under Deep Learning, it ranks #6 by 2026 paper count; the fastest grower in the group, Diffusion Models, is up 4.1x versus 0.9x here.

Representative papers

Spread across years and venues so every edition shows up

• NeurIPS 2026Achieving Directional-Stationarity from a Single Random Direction Step
• ICML 2026Bottleneck Communication Delay Minimization for Communication-Efficient Decentralized Learning
• ICLR 2026From Sorting Algorithms to Scalable Kernels: Bayesian Optimization in High-Dimensional Permutation Spaces
• CVPR 2026Coverage Optimization for Camera View Selection
• NeurIPS 2025High-Performance Arithmetic Circuit Optimization via Differentiable Architecture Search
• ICML 2025Feature-Mapping Topology Optimization with Neural Heaviside Signed Distance Functions
• ICLR 2025Boosting Neural Combinatorial Optimization for Large-Scale Vehicle Routing Problems
• CVPR 2025HotSpot: Signed Distance Function Optimization with an Asymptotically Sufficient Condition
• NeurIPS 2024$\boldsymbol{\mu}\mathbf{P^2}$: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling
• ICML 2024Barrier Algorithms for Constrained Non-Convex Optimization
• ICLR 2024Bayesian Coreset Optimization for Personalized Federated Learning
• CVPR 2024Multi-Session SLAM with Differentiable Wide-Baseline Pose Optimization
• NeurIPS 2023$\textbf{A}^2\textbf{CiD}^2$: Accelerating Asynchronous Communication in Decentralized Deep Learning
• ICML 2023Buying Information for Stochastic Optimization
• ICLR 2023Alternating Differentiation for Optimization Layers
• CVPR 2023Visual-Language Prompt Tuning With Knowledge-Guided Context Optimization
• NeurIPS 2026Adaptive Conditional Gradient Sliding: Projection-Free and Line-Search-Free Acceleration
• ICML 2026Riemannian Optimization for Fair Spectral Clustering
• ICLR 2026Gradient-Based Diversity Optimization with Differentiable Top-$k$ Objective
• CVPR 2026Its Never Too Late Noise Optimization for Collapse Recovery in

All Deep learning · optimization papers

500 papers, page 7 of 10

301. NeurIPS 2024ReLIZO: Sample Reusable Linear Interpolation-based Zeroth-order Optimization
302. NeurIPS 2024Reshuffling Resampling Splits Can Improve Generalization of Hyperparameter Optimization
303. NeurIPS 2024Resolving Discrepancies in Compute-Optimal Scaling of Language Models
304. NeurIPS 2024Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models
305. NeurIPS 2024Revive Re-weighting in Imbalanced Learning by Density Ratio Estimation
306. NeurIPS 2024S-SOS: Stochastic Sum-Of-Squares for Parametric Polynomial Optimization
307. NeurIPS 2024S-STE: Continuous Pruning Function for Efficient 2:4 Sparse Pre-training
308. NeurIPS 2024SAMPa: Sharpness-aware Minimization Parallelized
309. NeurIPS 2024Scalable Kernel Inverse Optimization
310. NeurIPS 2024Scalable Optimization in the Modular Norm
311. NeurIPS 2024ScaleKD: Strong Vision Transformers Could Be Excellent Teachers
312. NeurIPS 2024Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations
313. NeurIPS 2024Semidefinite Relaxations of the Gromov-Wasserstein Distance
314. NeurIPS 2024SequentialAttention++ for Block Sparsification: Differentiable Pruning Meets Combinatorial Optimization
315. NeurIPS 2024SGD vs GD: Rank Deficiency in Linear Networks
316. NeurIPS 2024Sharpness-Aware Minimization Activates the Interactive Teaching's Understanding and Optimization
317. NeurIPS 2024Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance
318. NeurIPS 2024Shuffling Gradient-Based Methods for Nonconvex-Concave Minimax Optimization
319. NeurIPS 2024SIRIUS : Contexual Sparisty with Correction for Efficient LLMs
320. NeurIPS 2024Sketching for Distributed Deep Learning: A Sharper Analysis
321. NeurIPS 2024SLowcalSGD : Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
322. NeurIPS 2024SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models
323. NeurIPS 2024Sparse maximal update parameterization: A holistic approach to sparse training dynamics
324. NeurIPS 2024Stabilized Proximal-Point Methods for Federated Optimization
325. NeurIPS 2024Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes
326. NeurIPS 2024Stepping Forward on the Last Mile
327. NeurIPS 2024Stepping on the Edge: Curvature Aware Learning Rate Tuners
328. NeurIPS 2024Stochastic Newton Proximal Extragradient Method
329. NeurIPS 2024Super Consistency of Neural Network Landscapes and Learning Rate Transfer
330. NeurIPS 2024Taming Cross-Domain Representation Variance in Federated Prototype Learning with Heterogeneous Data Domains
331. NeurIPS 2024The Closeness of In-Context Learning and Weight Shifting for Softmax Regression
332. NeurIPS 2024The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains
333. NeurIPS 2024The Feature Speed Formula: a flexible approach to scale hyper-parameters of deep neural networks
334. NeurIPS 2024The Implicit Bias of Adam on Separable Data
335. NeurIPS 2024The Prevalence of Neural Collapse in Neural Multivariate Regression
336. NeurIPS 2024The Road Less Scheduled
337. NeurIPS 2024Theoretical Characterisation of the Gauss Newton Conditioning in Neural Networks
338. NeurIPS 2024There is No Silver Bullet: Benchmarking Methods in Predictive Combinatorial Optimization
339. NeurIPS 2024Thinking Forward: Memory-Efficient Federated Finetuning of Language Models
340. NeurIPS 2024This Too Shall Pass: Removing Stale Observations in Dynamic Bayesian Optimization
341. NeurIPS 2024Topological obstruction to the training of shallow ReLU neural networks
342. NeurIPS 2024TorchOpt: An Efficient Library for Differentiable Optimization
343. NeurIPS 2024Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMs
344. NeurIPS 2024Training Binary Neural Networks via Gaussian Variational Inference and Low-Rank Semidefinite Programming
345. NeurIPS 2024TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks
346. NeurIPS 2024Universal Neural Functionals
347. NeurIPS 2024Unraveling the Gradient Descent Dynamics of Transformers
348. NeurIPS 2024Unveiling LoRA Intrinsic Ranks via Salience Analysis
349. NeurIPS 2024UQ-Guided Hyperparameter Optimization for Iterative Learners
350. NeurIPS 2024VeLoRA: Memory Efficient Training using Rank-1 Sub-Token Projections

FAQ

How many Deep learning · optimization papers were published at NeurIPS, ICLR, ICML and CVPR from 2023 to 2026?

953 in total: 256 in 2023, 294 in 2024, 163 in 2025, 240 in 2026.

How many Deep learning · optimization papers are at NeurIPS 2026?

NeurIPS 2026 has 103 Deep learning · optimization papers, versus 62 the year before.

How many Deep learning · optimization papers are at ICLR 2026?

ICLR 2026 has 44 Deep learning · optimization papers, versus 51 the year before.

How many Deep learning · optimization papers are at ICML 2026?

ICML 2026 has 86 Deep learning · optimization papers, versus 44 the year before.

How many Deep learning · optimization papers are at CVPR 2026?

CVPR 2026 has 7 Deep learning · optimization papers, versus 6 the year before.

Which conference publishes the most Deep learning · optimization papers?

NeurIPS, with 500 papers from 2023 to 2026.