Robustness Benchmarks papers, 2023–2026

Key findings

• Robustness Benchmarks went from 64 papers in 2023 to 69 in 2026, roughly 1.1x in three years.
• NeurIPS publishes the most Robustness Benchmarks work: 80 papers from 2023 to 2026, 30% of the four-venue total.
• The fastest-growing overlap is with General ML · evaluation: papers tagged with both rose from 0 in 2023 to 8 in 2026.
• Among the 5 topics under Evaluation and Benchmarks, it ranks #4 by 2026 paper count; the fastest grower in the group, LLM Benchmarks, is up 154x versus 1.1x here.

Representative papers

Spread across years and venues so every edition shows up

• NeurIPS 2026Brittlebench: Quantifying LLM robustness via prompt sensitivity
• ICML 2026Geometrically Constrained Outlier Synthesis
• ICLR 2026Pitfalls in Evaluating Language Model Forecasters
• CVPR 2026FoV-Net Rotation-Invariant CAD B-rep Learning via Field-of-View Ray Casting
• NeurIPS 2025Aggregation Hides Out-of-Distribution Generalization Failures from Spurious Correlations
• ICML 2025Diverse Prototypical Ensembles Improve Robustness to Subpopulation Shift
• ICLR 2025DRoP: Distributionally Robust Data Pruning
• CVPR 2025FRAMES-VQA: Benchmarking Fine-Tuning Robustness across Multi-Modal Shifts in Visual Question Answering
• NeurIPS 2024An Analysis of Robustness of Non-Lipschitz Networks
• ICML 2024Controllable Prompt Tuning For Balancing Group Distributional Robustness
• ICLR 2024Realistic Evaluation of Semi-supervised Learning Algorithms in Open Environments
• CVPR 2024A Bayesian Approach to OOD Robustness in Image Classification
• NeurIPS 2023Revisiting Out-of-distribution Robustness in NLP: Benchmarks, Analysis, and LLMs Evaluations
• ICML 2023A Law of Robustness beyond Isoperimetry
• ICLR 2023A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification
• CVPR 2023Improving Robustness of Vision Transformers by Reducing Sensitivity To Patch Corruptions
• NeurIPS 2026Stress-Testing Neural Network Verifiers with Provably Robust Instances
• ICML 2026Optimal Attention Temperature Improves the Robustness of In-Context Learning under Distribution Shift in High Dimensions
• ICLR 2026CLARC: C/C++ Benchmark for Robust Code Search
• CVPR 2026LIBERO-Plus A Progressive Robustness Benchmark for Visual-Language-Action Models

All Robustness Benchmarks papers

264 papers, page 4 of 6

151. ICLR 2024A Benchmark Study on Calibration
152. ICLR 2024A Recipe for Improved Certifiable Robustness
153. ICLR 2024Annealing Self-Distillation Rectification Improves Adversarial Training
154. ICLR 2024Boosting the Adversarial Robustness of Graph Neural Networks: An OOD Perspective
155. ICLR 2024Does CLIP’s generalization performance mainly stem from high train-test similarity?
156. ICLR 2024DRSM: De-Randomized Smoothing on Malware Classifier Providing Certified Robustness
157. ICLR 2024Evaluating Large Language Models at Evaluating Instruction Following
158. ICLR 2024Evaluating the Zero-shot Robustness of Instruction-tuned Language Models
159. ICLR 2024Expressive Losses for Verified Robustness via Convex Combinations
160. ICLR 2024Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
161. ICLR 2024ImageNet-OOD: Deciphering Modern Out-of-Distribution Detection Algorithms
162. ICLR 2024Large Language Models Are Not Robust Multiple Choice Selectors
163. ICLR 2024MetaPhysiCa: Improving OOD Robustness in Physics-informed Machine Learning
164. ICLR 2024Mitigating the Curse of Dimensionality for Certified Robustness via Dual Randomized Smoothing
165. ICLR 2024Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting
166. ICLR 2024Realistic Evaluation of Semi-supervised Learning Algorithms in Open Environments
167. ICLR 2024Towards Robust Out-of-Distribution Generalization Bounds via Sharpness
168. ICLR 2024Understanding Domain Generalization: A Noise Robustness Perspective
169. ICML 2024Can Implicit Bias Imply Adversarial Robustness?
170. ICML 2024Connect Later: Improving Fine-tuning for Robustness with Targeted Augmentations
171. ICML 2024Controllable Prompt Tuning For Balancing Group Distributional Robustness
172. ICML 2024Density Ratio Estimation with Doubly Strong Robustness
173. ICML 2024Enhancing Adversarial Robustness in SNNs with Sparse Gradients
174. ICML 2024Et Tu Certifications: Robustness Certificates Yield Better Adversarial Examples
175. ICML 2024Improving Accuracy-robustness Trade-off via Pixel Reweighted Adversarial Training
176. ICML 2024Improving Group Robustness on Spurious Correlation Requires Preciser Group Inference
177. ICML 2024Multigroup Robustness
178. ICML 2024OODRobustBench: a Benchmark and Large-Scale Analysis of Adversarial Robustness under Distribution Shift
179. ICML 2024Position: Exploring the Robustness of Pipeline-Parallelism-Based Decentralized Training
180. ICML 2024Revealing the Dark Secrets of Extremely Large Kernel ConvNets on Robustness
181. ICML 2024Robustness of Deep Learning for Accelerated MRI: Benefits of Diverse Training Data
182. ICML 2024Robustness of Nonlinear Representation Learning
183. ICML 2024WAVES: Benchmarking the Robustness of Image Watermarks
184. ICML 2024What Would Gauss Say About Representations? Probing Pretrained Image Models using Synthetic Gaussian Benchmarks
185. NeurIPS 2024A Sober Look at the Robustness of CLIPs to Spurious Features
186. NeurIPS 2024A StrongREJECT for Empty Jailbreaks
187. NeurIPS 2024Adaptive Randomized Smoothing: Certified Adversarial Robustness for Multi-Step Defences
188. NeurIPS 2024An Analysis of Robustness of Non-Lipschitz Networks
189. NeurIPS 2024BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices
190. NeurIPS 2024FedLLM-Bench: Realistic Benchmarks for Federated Learning of Large Language Models
191. NeurIPS 2024GREAT Score: Global Robustness Evaluation of Adversarial Perturbation using Generative Models
192. NeurIPS 2024Improving robustness to corruptions with multiplicative weight perturbations
193. NeurIPS 2024Intrinsic Robustness of Prophet Inequality to Strategic Reward Signaling
194. NeurIPS 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
195. NeurIPS 2024Mitigating Spurious Correlations via Disagreement Probability
196. NeurIPS 2024On the Worst Prompt Performance of Large Language Models
197. NeurIPS 2024Predicting the Performance of Foundation Models via Agreement-on-the-Line
198. NeurIPS 2024Provable Editing of Deep Neural Networks using Parametric Linear Relaxation
199. NeurIPS 2024TabularBench: Benchmarking Adversarial Robustness for Tabular Deep Learning in Real-world Use-cases
200. NeurIPS 2024The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlations

FAQ

How many Robustness Benchmarks papers were published at NeurIPS, ICLR, ICML and CVPR from 2023 to 2026?

264 in total: 64 in 2023, 59 in 2024, 72 in 2025, 69 in 2026.

How many Robustness Benchmarks papers are at NeurIPS 2026?

NeurIPS 2026 has 18 Robustness Benchmarks papers, versus 34 the year before.

How many Robustness Benchmarks papers are at ICLR 2026?

ICLR 2026 has 15 Robustness Benchmarks papers, versus 10 the year before.

How many Robustness Benchmarks papers are at ICML 2026?

ICML 2026 has 24 Robustness Benchmarks papers, versus 20 the year before.

How many Robustness Benchmarks papers are at CVPR 2026?

CVPR 2026 has 12 Robustness Benchmarks papers, versus 8 the year before.

Which conference publishes the most Robustness Benchmarks papers?

NeurIPS, with 80 papers from 2023 to 2026.