Human Evaluation papers, 2023–2026

Key findings

• Human Evaluation went from 33 papers in 2023 to 108 in 2026, roughly 3.3x in three years.
• NeurIPS publishes the most Human Evaluation work: 90 papers from 2023 to 2026, 33% of the four-venue total.
• The fastest-growing overlap is with Datasets and Benchmarks: papers tagged with both rose from 3 in 2023 to 31 in 2026.
• Among the 5 topics under Evaluation and Benchmarks, it ranks #3 by 2026 paper count; the fastest grower in the group, LLM Benchmarks, is up 154x versus 3.3x here.

Representative papers

Spread across years and venues so every edition shows up

• NeurIPS 2026Ad-Hoc Teamwork from Human Demonstrations
• ICML 2026SWE-IF: Aligning Code Evaluation with Human Preference
• ICLR 2026Human-LLM Collaborative Feature Engineering for Tabular Data
• CVPR 2026Bridging Human Evaluation to Infrared and Visible Image Fusion
• NeurIPS 2025Deep Edge Filter: Return of the Human-Crafted Layer in Deep Learning
• ICML 2025Position: When Incentives Backfire, Data Stops Being Human
• ICLR 2025Eliciting Human Preferences with Language Models
• CVPR 2025DynPose: Largely Improving the Efficiency of Human Pose Estimation by a Simple Dynamic Framework
• NeurIPS 2024DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA
• ICML 2024A Fine-grained Analysis of Fitted Q-evaluation: Beyond Parametric Models
• ICLR 2024Human Feedback is not Gold Standard
• CVPR 2024LocLLM: Exploiting Generalizable Human Keypoint Localization via Large Language Model
• NeurIPS 2023Privacy Assessment on Reconstructed Images: Are Existing Evaluation Metrics Faithful to Human Perception?
• ICML 2023Regions of Reliability in the Evaluation of Multivariate Probabilistic Forecasts
• ICLR 2023Beyond calibration: estimating the grouping loss of modern neural networks
• CVPR 20233D Human Keypoints Estimation From Point Clouds in the Wild Without Human Labels
• NeurIPS 2026AI Construct Lexis: An Ontology of the Hidden Assumptions in AI Evaluation
• ICML 2026Rethinking Evaluation Paradigms in IBP-based Certified Training
• ICLR 2026Multi-turn Evaluation of Anthropomorphic Behaviours in Large Language Models
• CVPR 2026Ground Reaction Inertial Poser Physics-based Human Motion Capture from Sparse

All Human Evaluation papers

90 papers, page 1 of 2

1. NeurIPS 2026Ad-Hoc Teamwork from Human Demonstrations
2. NeurIPS 2026AI Construct Lexis: An Ontology of the Hidden Assumptions in AI Evaluation
3. NeurIPS 2026AIRA 2: Overcoming Bottlenecks in AI Research Agents
4. NeurIPS 2026AMUSE: Anytime Muon with Stable Gradient Evaluation
5. NeurIPS 2026An Enigma of Artificial Reason: Investigating the Production-Evaluation Gap in Large Reasoning Models
6. NeurIPS 2026BayesJudge: Uncertainty-Aware Bayesian Meta-Evaluation of Human and LLM Judgments
7. NeurIPS 2026BenchRep-T: A Systematic Evaluation of T-Cell Repertoire-Based Disease Diagnostics
8. NeurIPS 2026Beyond Proxy Metrics: MLLM-Based Human Surrogate Evaluation for Explainable AI
9. NeurIPS 2026Correct, Route, Calibrate: Efficient Preference Optimization from Noisy, Heterogeneous Human Feedback
10. NeurIPS 2026Cosine is Human: The Reproducibility Ceiling of Perceptual Similarity
11. NeurIPS 2026Evaluating an Evaluation: Membership Inference Attacks as Machine Unlearning Diagnostics
12. NeurIPS 2026Evaluation of Visual Processing Should Be Human-Centered, Not Metric-Centered
13. NeurIPS 2026From Uncertain Judgments to Calibrated Rankings: Conformal Elo Estimation for LLM Evaluation
14. NeurIPS 2026General Agent Evaluation
15. NeurIPS 2026GT-Free OCR Metrics: A Reference-Free Evaluation Framework for Document OCR Systems
16. NeurIPS 2026Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scale
17. NeurIPS 2026HearSayBench: Can LLMs Navigate from Abstract Human Rights to Lived Lives?
18. NeurIPS 2026HERO: Improving the Reliability and Sensitivity of Generative Model Evaluation Using Historical Data
19. NeurIPS 2026ImageNet FID is a Pass Check, Not a Finish Line
20. NeurIPS 2026Intra-Option Fitted Q-Evaluation: Evaluating Hierarchical Policies from Non-Hierarchical Data
21. NeurIPS 2026Leak-CURBER: A Leakage-Controlled Multimodal Evaluation Benchmark for Enzymatic Reaction Tasks
22. NeurIPS 2026OdysSim: Building Foundation Models for Human Behavior Simulation
23. NeurIPS 2026Off-Policy Evaluation of Large Language Models via Learned Semantic Bottleneck Embeddings
24. NeurIPS 2026OmniMechanism Design for Human-AI Collaboration with Impressionable Minds
25. NeurIPS 2026OTROPE: Optimal Transport-based Robust Off-policy Evaluation for Large Language Models
26. NeurIPS 2026PhyProbe: Rethinking Physical Consistency Evaluation in Generated Videos
27. NeurIPS 2026Post-Selection Distributional Model Evaluation
28. NeurIPS 2026pyCD: A Unified Benchmark for Reliable Evaluation of Cognitive Diagnosis Models
29. NeurIPS 2026Reporting Practice Matters: The Impact of Reference Choice on Chest X-ray Report Evaluation
30. NeurIPS 2026Rethinking XAI Evaluation: A Human-Centered Audit of Shapley Benchmarks in High-Stakes Settings
31. NeurIPS 2026SALART-VQA: Diagnosing Whether VLMs Understand Salient Artifacts in Generated Images
32. NeurIPS 2026Select Smarter, Not More? Prompt-Aware Evaluation Scheduling with Submodular Guarantees
33. NeurIPS 2026SR-Prominence: A Crowdsourced Protocol and Dataset Suite for Perceptually-Weighted Super-Resolution Artifact Evaluation
34. NeurIPS 2026The Deadline Effect: Identifying and Correcting Temporal Bias in Human Evaluation
35. NeurIPS 2026Verifier Choice is a Benchmark Design Variable: Auditing Structural Counting Evaluation in Text-to-Image Models
36. NeurIPS 2026VIPBench: A Human-Aligned Benchmark for Voice Identity Perception in the Age of Voice Cloning
37. NeurIPS 2026VTBench: Disentangled and Human-Aligned Evaluation for Image-Based Virtual Try-on
38. NeurIPS 2026When Are Predictions Enough? An Evaluation Protocol for Frozen Expert Composition
39. NeurIPS 2026WorldReasonBench: Human-Aligned Stress Testing of Video Generators as Future World-State Predictors
40. NeurIPS 2025A Principled Path to Fitted Distributional Evaluation
41. NeurIPS 2025AGENTIF: Benchmarking Large Language Models Instruction Following Ability in Agentic Scenarios
42. NeurIPS 2025AI Debate Aids Assessment of Controversial Claims
43. NeurIPS 2025Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representations
44. NeurIPS 2025Can Large Language Models Master Complex Card Games?
45. NeurIPS 2025Can LLMs Outshine Conventional Recommenders? A Comparative Evaluation
46. NeurIPS 2025Clean First, Align Later: Benchmarking Preference Data Cleaning for Reliable LLM Alignment
47. NeurIPS 2025Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning
48. NeurIPS 2025Deep Edge Filter: Return of the Human-Crafted Layer in Deep Learning
49. NeurIPS 2025Don’t call it privacy-preserving or human-centric pose estimation if you don’t measure privacy
50. NeurIPS 2025Handling Missing Responses under Cluster Dependence with Applications to Language Model Evaluation

FAQ

How many Human Evaluation papers were published at NeurIPS, ICLR, ICML and CVPR from 2023 to 2026?

271 in total: 33 in 2023, 70 in 2024, 60 in 2025, 108 in 2026.

How many Human Evaluation papers are at NeurIPS 2026?

NeurIPS 2026 has 39 Human Evaluation papers, versus 17 the year before.

How many Human Evaluation papers are at ICLR 2026?

ICLR 2026 has 29 Human Evaluation papers, versus 16 the year before.

How many Human Evaluation papers are at ICML 2026?

ICML 2026 has 21 Human Evaluation papers, versus 14 the year before.

How many Human Evaluation papers are at CVPR 2026?

CVPR 2026 has 19 Human Evaluation papers, versus 13 the year before.

Which conference publishes the most Human Evaluation papers?

NeurIPS, with 90 papers from 2023 to 2026.