Human Evaluation papers, 2023–2026

Key findings

• Human Evaluation went from 33 papers in 2023 to 108 in 2026, roughly 3.3x in three years.
• NeurIPS publishes the most Human Evaluation work: 90 papers from 2023 to 2026, 33% of the four-venue total.
• The fastest-growing overlap is with Datasets and Benchmarks: papers tagged with both rose from 3 in 2023 to 31 in 2026.
• Among the 5 topics under Evaluation and Benchmarks, it ranks #3 by 2026 paper count; the fastest grower in the group, LLM Benchmarks, is up 154x versus 3.3x here.

Representative papers

Spread across years and venues so every edition shows up

• NeurIPS 2026Ad-Hoc Teamwork from Human Demonstrations
• ICML 2026SWE-IF: Aligning Code Evaluation with Human Preference
• ICLR 2026Human-LLM Collaborative Feature Engineering for Tabular Data
• CVPR 2026Bridging Human Evaluation to Infrared and Visible Image Fusion
• NeurIPS 2025Deep Edge Filter: Return of the Human-Crafted Layer in Deep Learning
• ICML 2025Position: When Incentives Backfire, Data Stops Being Human
• ICLR 2025Eliciting Human Preferences with Language Models
• CVPR 2025DynPose: Largely Improving the Efficiency of Human Pose Estimation by a Simple Dynamic Framework
• NeurIPS 2024DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA
• ICML 2024A Fine-grained Analysis of Fitted Q-evaluation: Beyond Parametric Models
• ICLR 2024Human Feedback is not Gold Standard
• CVPR 2024LocLLM: Exploiting Generalizable Human Keypoint Localization via Large Language Model
• NeurIPS 2023Privacy Assessment on Reconstructed Images: Are Existing Evaluation Metrics Faithful to Human Perception?
• ICML 2023Regions of Reliability in the Evaluation of Multivariate Probabilistic Forecasts
• ICLR 2023Beyond calibration: estimating the grouping loss of modern neural networks
• CVPR 20233D Human Keypoints Estimation From Point Clouds in the Wild Without Human Labels
• NeurIPS 2026AI Construct Lexis: An Ontology of the Hidden Assumptions in AI Evaluation
• ICML 2026Rethinking Evaluation Paradigms in IBP-based Certified Training
• ICLR 2026Multi-turn Evaluation of Anthropomorphic Behaviours in Large Language Models
• CVPR 2026Ground Reaction Inertial Poser Physics-based Human Motion Capture from Sparse

All Human Evaluation papers

49 papers, page 1 of 1

1. ICML 2026A Computational Framework for Evaluating Human-likeness in LLMs' Open-ended Human Behaviors
2. ICML 2026AppWorld-UL: Benchmarking Diverse Agent-User Interactions for Tool-Use
3. ICML 2026Benchmarking World-Model Learning with Environment-Level Queries
4. ICML 2026Certifying Capabilities from Finite Tests: When Is It Possible?
5. ICML 2026Conversation for Non-verifiable Learning: Self-Evolving Large Language Models through Meta-Evaluation
6. ICML 2026Efficient Inference for Noisy LLM-as-a-Judge Evaluation
7. ICML 2026Expectation Alignment of Language Models for Real-World User Expectations
8. ICML 2026Explanations are a Means to an End: Decision Theoretic Explanation Evaluation
9. ICML 2026FiRE: Fine-grained Ranking Evaluation for Machine Translation
10. ICML 2026GICDM: Mitigating Hubness for Reliable Distance-Based Generative Model Evaluation
11. ICML 2026Hair-Trigger Alignment: Black-Box Evaluation Cannot Guarantee Post-Update Alignment
12. ICML 2026LiveOIBench: Can Large Language Models Outperform Human Contestants in Informatics Olympiads?
13. ICML 2026Moving Out: Physically-grounded Human-AI Collaboration
14. ICML 2026Nonparametric LLM Evaluation from Preference Data
15. ICML 2026PerceptionRubrics: Calibrating Multimodal Evaluation to Human Perception
16. ICML 2026QEDBench: Quantifying the Alignment Gap in Automated Evaluation of University-Level Mathematical Proofs
17. ICML 2026Query-efficient model evaluation using cached responses
18. ICML 2026Rethinking Evaluation Paradigms in IBP-based Certified Training
19. ICML 2026Reward Modeling from Natural Language Human Feedback
20. ICML 2026STABLEVAL: Disagreement-Aware and Stable Evaluation of AI Systems
21. ICML 2026SWE-IF: Aligning Code Evaluation with Human Preference
22. ICML 2025Accelerating Unbiased LLM Evaluation via Synthetic Feedback
23. ICML 2025AutoEval Done Right: Using Synthetic Data for Model Evaluation
24. ICML 2025CogMath: Assessing LLMs' Authentic Mathematical Ability from a Human Cognitive Perspective
25. ICML 2025GRADEO: Towards Human-Like Evaluation for Text-to-Video Generation via Multi-Step Reasoning
26. ICML 2025MapEval: A Map-Based Evaluation of Geo-Spatial Reasoning in Foundation Models
27. ICML 2025MCU: An Evaluation Framework for Open-Ended Game Agents
28. ICML 2025Position: AI Competitions Provide the Gold Standard for Empirical Rigor in GenAI Evaluation
29. ICML 2025Position: AI Evaluation Should Learn from How We Test Humans
30. ICML 2025Position: AI Should Not Be An Imitation Game: Centaur Evaluations
31. ICML 2025Position: Democratic AI is Possible. The Democracy Levels Framework Shows How It Might Work.
32. ICML 2025Position: Human Baselines in Model Evaluations Need Rigor and Transparency (With Recommendations & Reporting Checklist)
33. ICML 2025Position: Stop treating `AGI' as the north-star goal of AI research
34. ICML 2025Position: When Incentives Backfire, Data Stops Being Human
35. ICML 2025Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing
36. ICML 2024A Fine-grained Analysis of Fitted Q-evaluation: Beyond Parametric Models
37. ICML 2024AutoOS: Make Your OS More Powerful by Exploiting Large Language Models
38. ICML 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
39. ICML 2024Evaluation of Trajectory Distribution Predictions with Energy Score
40. ICML 2024Human-like Category Learning by Injecting Ecological Priors from Large Language Models into Neural Networks
41. ICML 2024Kernel-Based Evaluation of Conditional Biological Sequence Models
42. ICML 2024ODIN: Disentangled Reward Mitigates Hacking in RLHF
43. ICML 2024On the Calibration of Human Pose Estimation
44. ICML 2024Open-Domain Text Evaluation via Contrastive Distribution Methods
45. ICML 2024PerceptAnon: Exploring the Human Perception of Image Anonymization Beyond Pseudonymization for GDPR
46. ICML 2024Rethinking Generative Large Language Model Evaluation for Semantic Comprehension
47. ICML 2024Stability Evaluation through Distributional Perturbation Analysis
48. ICML 2024Using AI Uncertainty Quantification to Improve Human Decision-Making
49. ICML 2023Regions of Reliability in the Evaluation of Multivariate Probabilistic Forecasts

FAQ

How many Human Evaluation papers were published at NeurIPS, ICLR, ICML and CVPR from 2023 to 2026?

271 in total: 33 in 2023, 70 in 2024, 60 in 2025, 108 in 2026.

How many Human Evaluation papers are at NeurIPS 2026?

NeurIPS 2026 has 39 Human Evaluation papers, versus 17 the year before.

How many Human Evaluation papers are at ICLR 2026?

ICLR 2026 has 29 Human Evaluation papers, versus 16 the year before.

How many Human Evaluation papers are at ICML 2026?

ICML 2026 has 21 Human Evaluation papers, versus 14 the year before.

How many Human Evaluation papers are at CVPR 2026?

CVPR 2026 has 19 Human Evaluation papers, versus 13 the year before.

Which conference publishes the most Human Evaluation papers?

NeurIPS, with 90 papers from 2023 to 2026.