Robustness Benchmarks papers, 2023–2026

Key findings

• Robustness Benchmarks went from 64 papers in 2023 to 69 in 2026, roughly 1.1x in three years.
• NeurIPS publishes the most Robustness Benchmarks work: 80 papers from 2023 to 2026, 30% of the four-venue total.
• The fastest-growing overlap is with General ML · evaluation: papers tagged with both rose from 0 in 2023 to 8 in 2026.
• Among the 5 topics under Evaluation and Benchmarks, it ranks #4 by 2026 paper count; the fastest grower in the group, LLM Benchmarks, is up 154x versus 1.1x here.

Representative papers

Spread across years and venues so every edition shows up

• NeurIPS 2026Brittlebench: Quantifying LLM robustness via prompt sensitivity
• ICML 2026Geometrically Constrained Outlier Synthesis
• ICLR 2026Pitfalls in Evaluating Language Model Forecasters
• CVPR 2026FoV-Net Rotation-Invariant CAD B-rep Learning via Field-of-View Ray Casting
• NeurIPS 2025Aggregation Hides Out-of-Distribution Generalization Failures from Spurious Correlations
• ICML 2025Diverse Prototypical Ensembles Improve Robustness to Subpopulation Shift
• ICLR 2025DRoP: Distributionally Robust Data Pruning
• CVPR 2025FRAMES-VQA: Benchmarking Fine-Tuning Robustness across Multi-Modal Shifts in Visual Question Answering
• NeurIPS 2024An Analysis of Robustness of Non-Lipschitz Networks
• ICML 2024Controllable Prompt Tuning For Balancing Group Distributional Robustness
• ICLR 2024Realistic Evaluation of Semi-supervised Learning Algorithms in Open Environments
• CVPR 2024A Bayesian Approach to OOD Robustness in Image Classification
• NeurIPS 2023Revisiting Out-of-distribution Robustness in NLP: Benchmarks, Analysis, and LLMs Evaluations
• ICML 2023A Law of Robustness beyond Isoperimetry
• ICLR 2023A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification
• CVPR 2023Improving Robustness of Vision Transformers by Reducing Sensitivity To Patch Corruptions
• NeurIPS 2026Stress-Testing Neural Network Verifiers with Provably Robust Instances
• ICML 2026Optimal Attention Temperature Improves the Robustness of In-Context Learning under Distribution Shift in High Dimensions
• ICLR 2026CLARC: C/C++ Benchmark for Robust Code Search
• CVPR 2026LIBERO-Plus A Progressive Robustness Benchmark for Visual-Language-Action Models

All Robustness Benchmarks papers

264 papers, page 3 of 6

101. ICML 2025Position: Medical Large Language Model Benchmarks Should Prioritize Construct Validity
102. ICML 2025Position: Societal Impacts Research Requires Benchmarks for Creative Composition Tasks
103. ICML 2025Probably Approximately Global Robustness Certification
104. ICML 2025Quadratic Upper Bound for Boosting Robustness
105. ICML 2025Statistical Hypothesis Testing for Auditing Robustness in Language Models
106. ICML 2025Towards Robust Influence Functions with Flat Validation Minima
107. ICML 2025When and How Does CLIP Enable Domain and Compositional Generalization?
108. NeurIPS 2025A Technical Report on “Erasing the Invisible”: The 2024 NeurIPS Competition on Stress Testing Image Watermarks
109. NeurIPS 2025A Unified Approach to Submodular Maximization Under Noise
110. NeurIPS 2025Absence Bench: Language Models Can’t See What’s Missing
111. NeurIPS 2025Adaptive Distraction: Probing LLM Contextual Robustness with Automated Tree Search
112. NeurIPS 2025Adversarial Robustness of Nonparametric Regression
113. NeurIPS 2025Aggregation Hides Out-of-Distribution Generalization Failures from Spurious Correlations
114. NeurIPS 2025Augmenting Biological Fitness Prediction Benchmarks with Landscapes Features from GraphFLA
115. NeurIPS 2025Automated Detection of Visual Attribute Reliance with a Self-Reflective Agent
116. NeurIPS 2025BenchmarkCards: Standardized Documentation for Large Language Model Benchmarks
117. NeurIPS 2025Bridging Crypto with ML-based Solvers: the SAT Formulation and Benchmarks
118. NeurIPS 2025Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations
119. NeurIPS 2025ColorBench: Can VLMs See and Understand the Colorful World? A Comprehensive Benchmark for Color Perception, Reasoning, and Robustness
120. NeurIPS 2025Data Selection Matters: Towards Robust Instruction Tuning of Large Multimodal Models
121. NeurIPS 2025Embracing Contradiction: Theoretical Inconsistency Will Not Impede the Road of Building Responsible AI Systems
122. NeurIPS 2025Enhancing Graph Classification Robustness with Singular Pooling
123. NeurIPS 2025Evaluating Robustness of Monocular Depth Estimation with Procedural Scene Perturbations
124. NeurIPS 2025Fantastic Bugs and Where to Find Them in AI Benchmarks
125. NeurIPS 2025HiMoLE: Towards OOD-Robust LoRA via Hierarchical Mixture of Experts
126. NeurIPS 2025Measuring what Matters: Construct Validity in Large Language Model Benchmarks
127. NeurIPS 2025Metritocracy: Representative Metrics for Lite Benchmarks
128. NeurIPS 2025Multimodal Negative Learning
129. NeurIPS 2025Mysteries of the Deep: Role of Intermediate Representations in Out of Distribution Detection
130. NeurIPS 2025On the Optimality of the Median-of-Means Estimator under Adversarial Contamination
131. NeurIPS 2025REOBench: Benchmarking Robustness of Earth Observation Foundation Models
132. NeurIPS 2025Rethinking Evaluation of Infrared Small Target Detection
133. NeurIPS 2025Robust Equilibria in Continuous Games: From Strategic to Dynamic Robustness
134. NeurIPS 2025Robust learning of halfspaces under log-concave marginals
135. NeurIPS 2025Robustifying Learning-Augmented Caching Efficiently without Compromising 1-Consistency
136. NeurIPS 2025Scalable Neural Network Geometric Robustness Validation via Hölder Optimisation
137. NeurIPS 2025Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation
138. NeurIPS 2025The Cost of Robustness: Tighter Bounds on Parameter Complexity for Robust Memorization in ReLU Nets
139. NeurIPS 2025Trans-EnV: A Framework for Evaluating the Linguistic Robustness of LLMs Against English Varieties
140. NeurIPS 2025Vulnerable Data-Aware Adversarial Training
141. NeurIPS 2025XIFBench: Evaluating Large Language Models on Multilingual Instruction Following
142. CVPR 2024A Bayesian Approach to OOD Robustness in Image Classification
143. CVPR 2024Benchmarking the Robustness of Temporal Action Detection Models Against Temporal Corruptions
144. CVPR 2024Bilateral Adaptation for Human-Object Interaction Detection with Occlusion-Robustness
145. CVPR 2024Calibrating Multi-modal Representations: A Pursuit of Group Robustness without Annotations
146. CVPR 2024ImageNet-D: Benchmarking Neural Network Robustness on Diffusion Synthetic Object
147. CVPR 2024Model Inversion Robustness: Can Transfer Learning Help?
148. CVPR 2024Reconstructing Hands in 3D with Transformers
149. CVPR 2024RoDLA: Benchmarking the Robustness of Document Layout Analysis Models
150. CVPR 2024Soften to Defend: Towards Adversarial Robustness via Self-Guided Label Refinement

FAQ

How many Robustness Benchmarks papers were published at NeurIPS, ICLR, ICML and CVPR from 2023 to 2026?

264 in total: 64 in 2023, 59 in 2024, 72 in 2025, 69 in 2026.

How many Robustness Benchmarks papers are at NeurIPS 2026?

NeurIPS 2026 has 18 Robustness Benchmarks papers, versus 34 the year before.

How many Robustness Benchmarks papers are at ICLR 2026?

ICLR 2026 has 15 Robustness Benchmarks papers, versus 10 the year before.

How many Robustness Benchmarks papers are at ICML 2026?

ICML 2026 has 24 Robustness Benchmarks papers, versus 20 the year before.

How many Robustness Benchmarks papers are at CVPR 2026?

CVPR 2026 has 12 Robustness Benchmarks papers, versus 8 the year before.

Which conference publishes the most Robustness Benchmarks papers?

NeurIPS, with 80 papers from 2023 to 2026.