AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

arXiv

RADD: Retrieval-Augmented Discrete Diffusion for Multi-Modal Knowledge Graph Completion

Guanglin Niu, Bo Li

Most multi-modal knowledge graph completion (MMKGC) models use one embedding scorer to conduct both retrieval over the full entity set and final link prediction. We argue that this coupling is a core bottleneck: global high-recall search and local fine-grained disambiguation require different inductive biases. Therefore, we propose a Retrieval-Augmented Discrete Diffusion (RADD) framework to decouple retrieval and reranking for MMKGC. A relation-aware multimodal knowledge graph embedding (KGE) retriever serves as both global retriever and distillation teacher, while a conditional discrete denoiser performs shortlist-level entity-identity generation for reranking. Training combines KGE supervision, denoising cross-entropy, and temperature-scaled distillation from the retriever to the denoiser. At inference, the designed Diff-Rerank first forms a top-K shortlist with the retriever and then reranks it with the denoiser, ensuring that recall is a strict requirement for precision. Experiments on three MMKGC benchmarks show that RADD achieves the best performance and consistent gains over strong unimodal, multimodal, and LLM-based baselines, while ablations further verify each component's contribution.

arXiv

"I understand your perspective": LLM Persuasion through the Lens of Communicative Action Theory

Esra Dönmez, Agnieszka Falenska

Large Language Models (LLMs) can generate high-quality arguments, yet their ability to engage in nuanced and persuasive communicative actions remains largely unexplored. This work explores the persuasive potential of LLMs through the framework of Jürgen Habermas' Theory of Communicative Action. It examines whether LLMs express illocutionary intent (i.e., pragmatic functions of language such as conveying knowledge, building trust, or signaling similarity) in ways that are comparable to human communication. We simulate online discussions between opinion holders and LLMs using conversations from the persuasive subreddit ChangeMyView. We then compare the likelihood of illocutionary intents in human-written and LLM-generated counter-arguments, specifically those that successfully changed the original poster's view. We find that all three LLMs effectively convey illocutionary intent -- often more so than humans -- potentially increasing their anthropomorphism. Further, LLMs craft sycophantic responses that closely align with the opinion holder's intent, a strategy strongly associated with opinion change. Finally, crowd-sourced workers find LLM-generated counter-arguments more agreeable and consistently prefer them over human-written ones. These findings suggest that LLMs' persuasive power extends beyond merely generating high-quality arguments. On the contrary, training LLMs with human preferences effectively tunes them to mirror human communication patterns, particularly nuanced communicative actions, potentially increasing individuals' susceptibility to their influence.

arXiv

Bigger Is Safer: Provable Robustness in In-Context Learning Scales with Capacity

Di Zhang, Ningxu Zhang, Zimeng Liu

In-context learning (ICL) allows large language models to adapt to new tasks from a few examples without updating their parameters. Existing theories explain ICL by assuming the test task distribution matches pretraining -- an assumption that breaks down under adversarial distribution shifts. We introduce a distributionally robust meta-learning framework that provides worst-case guarantees for ICL under Wasserstein-based distribution shifts. Focusing on linear self-attention Transformers, we derive a non-asymptotic bound connecting adversarial perturbation strength ($ρ$), model capacity ($m$), and the number of in-context examples ($N$). The analysis reveals that the maximum safe perturbation radius scales as $ρ_{\max} \propto \sqrt{m}$, while maintaining performance under adversarial shift requires additional in-context examples with $N_ρ- N_0 \propto ρ^2$. Experiments on synthetic tasks confirm these scaling laws, and experiments on 21 real pretrained models (0.1B--7B parameters, 5 families) provide qualitative evidence consistent with the theory's predictions, while revealing that ICL capability is a prerequisite for robustness. These findings advance the theoretical understanding of ICL under adversarial conditions and formalize the sense in which larger models are safer under distributional shift.

arXiv

From Multi-Agent to Single-Agent: When Is Skill Distillation Beneficial?

Binyan Xu, Dong Fang, Haitao Li, Kehuan Zhang

Multi-agent systems (MAS) tackle complex tasks by distributing expertise, though this often comes at the cost of heavy coordination overhead, context fragmentation, and brittle phase ordering. Distilling a MAS into a single-agent skill can bypass these costs, but this conversion lacks a principled answer for when and what to distill. Instead, the empirical outcome is surprisingly inconsistent: skill lift ranges from a 28% improvement to a 2% degradation across metrics of the exact same task. In this work, we reveal that skill utility is governed not by the task, but by the evaluation metric. We introduce Metric Freedom (F), the first a priori predictor of skill utility. F measures the topological rigidity of a metric's scoring landscape by quantifying how output diversity couples with score variance via a Mantel test. Guided by F, we propose AdaSkill, a two-stage adaptive distillation framework. Stage 1 acts as a selective extraction mechanism, extracting tools and knowledge while discarding restrictive structures on "free" metrics to preserve exploration. Stage 2 applies iterative refinement selectively on free metrics, exploiting their forgiving scoring landscape to safely maximize remaining headroom. Evaluating across 4 tasks, 11 datasets, and 6 metrics, F strongly predicts skill utility (r=-0.85, p<0.0001). Strikingly, identical agent trajectories yield diametrically opposite skill lifts under rigid versus free metrics, demonstrating that skill utility is fundamentally a metric-level property. Driven by this signal, AdaSkill matches or exceeds the original MAS while reducing cost up to 8x and latency by up to 15x.

arXiv

Agent psychometrics: Task-level performance prediction in agentic coding benchmarks

Chris Ge, Daria Kryvosheieva, Daniel Fried, Uzay Girit, Kaivalya Hariharan

As the focus in LLM-based coding shifts from static single-step code generation to multi-step agentic interaction with tools and environments, understanding which tasks will challenge agents and why becomes increasingly difficult. This is compounded by current practice: agent performance is typically measured by aggregate pass rates on benchmarks, but single-number metrics obscure the diversity of tasks within a benchmark. We present a framework for predicting success or failure on individual tasks tailored to the agentic coding regime. Our approach augments Item Response Theory (IRT) with rich features extracted from tasks, including issue statements, repository contexts, solutions, and test cases, and introduces a novel decomposition of agent ability into LLM and scaffold ability components. This parameterization enables us to aggregate evaluation data across heterogeneous leaderboards and accurately predict task-level performance for unseen benchmarks, as well as unseen LLM-scaffold combinations. Our methods have practical utility for benchmark designers, who can better calibrate the difficulty of their new tasks without running computationally expensive agent evaluations.

arXiv

Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling

Zhen Zhang, Changyi Yang, Zijie Xia, Zhen Yang, Chengzhi Liu, Zhaotiao Weng, Yepeng Liu, Haobo Chen, Jin Pan, Chenyang Zhao, Yuheng Bu, Alkesh Patel, Zhe Gan, Xin Eric Wang

Tokens are the fundamental units of computation in modern autoregressive models, and generation length directly influences both inference cost and reasoning performance. Despite its importance, existing approaches model length primarily at the coarse sequence level. We introduce the Length Value Model (LenVM), a token-level framework that estimates the remaining generation length at every decoding step. By formulating length modeling as a value estimation problem and assigning a constant negative reward to each generated token, LenVM predicts a bounded, discounted return that is a monotone proxy for the remaining generation horizon. This value formulation provides annotation-free, dense, unbiased, and scalable supervision. Experiments on LLMs and VLMs show that LenVM supports exact control, continuous performance--efficiency steering, length prediction, and interpretation. On LIFEBench-token, it raises the exact-length score of Qwen2.5-7B-Instruct from $30.9$ to $64.8$ in one pass and reaches $83.6$ when combined with LCG. Near $200$ tokens on GSM8K, LenVM retains about $63\%$ Pass@1 versus $6\%$ under a hard token budget. Across other settings, LenVM improves matched-length quality over prompting and EOS calibration, while its length-prediction accuracy improves with scale and exceeds specialized predictors. Its token-level values also identify markers of shifts between longer and shorter trajectories. These results establish generation length as an effective scalable token-level value signal for control, prediction, and future value-based training.

arXiv

Measuring and Mitigating Post-hoc Rationalization in Reverse Chain-of-Thought Generation

Guangyue Peng, Zongchao Chen, Wen Luo, Yuntao Wen, Wei Li, Ruixiang Feng, Ran Le, Chen Yang, Zhenwei An, Yang Song, Tao Zhang, Houfeng Wang

Reverse Chain-of-Thought Generation (RCG) synthesizes reasoning traces from query-answer pairs, but answer-visible generation can justify a pre-committed answer rather than derive it. This post-hoc rationalization creates a train-inference mismatch because student models are trained on answer-conditioned traces but must reason without answer access at inference time. We quantify this mismatch through lexical, trajectory, and probabilistic anchoring, measuring surface overlap, answer-conditioned generation dynamics, and answer recoverability from the trace, respectively. We find that semantic suppression, a seemingly intuitive mitigation, reduces lexical overlap but increases trajectory anchoring: avoiding the answer requires continually tracking it, thereby strengthening its influence on generation. We therefore propose Structural Skeleton-guided Reasoning (SSR), which replaces suppression with structural decoupling by first generating an abstract functional skeleton and then using it to guide the full reasoning trace. Anchoring analyses show that SSR reduces all three forms of answer dependence. Across in-domain and out-of-distribution benchmarks, its distilled variant, SSR-D, improves performance by up to 10 points over suppression baselines and better preserves out-of-distribution performance. Code is available at https://github.com/viniferagy/SSR.

arXiv

Bayesian Signal Component Decomposition via Diffusion-within-Gibbs Sampling

Yi Zhang, Rui Guo, Yonina C. Eldar

In signal processing, the data collected from sensing devices is often a noisy linear superposition of multiple components, and the estimation of components of interest constitutes a crucial pre-processing step. In this work, we develop a Bayesian framework for signal component decomposition, which combines Gibbs sampling with plug-and-play (PnP) diffusion priors to draw component samples from the posterior distribution. Unlike many existing methods, our framework supports incorporating component-wise model-driven and data-driven priors into diffusion models in a unified manner. Moreover, the proposed posterior sampler allows component priors to be learned separately and flexibly combined for different decomposition tasks at inference time. Under suitable assumptions, the proposed Diffusion-within-Gibbs (DiG) sampler provably produces samples from the posterior distribution. We also show that DiG can be interpreted as an extension of a class of recently proposed diffusion-based samplers, and that, for suitable classes of sensing operators, DiG better exploits the structure of the measurement model. Numerical experiments demonstrate the superior performance of our method over existing approaches.

arXiv

Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification

Zehai He, Wenyi Hong, Zhen Yang, Ziyang Pan, Mingdao Liu, Xiaotao Gu, Jie Tang

Recent advances in large language models have improved the capabilities of coding agents, yet systematic evaluation of complex, end-to-end website development remains limited. To address this gap, we introduce Vision2Web, a hierarchical benchmark for visual website development, spanning from static UI-to-code generation, interactive multi-page frontend reproduction, to long-horizon full-stack website development. The benchmark is constructed from real-world websites and comprises a total of 193 tasks across 16 categories, with 918 prototype images and 1,255 test cases. To support flexible, thorough and reliable evaluation, we propose workflow-based agent verification paradigm based on two complementary components: a GUI agent verifier and a VLM-based judge. We evaluate multiple visual language models instantiated under different coding-agent frameworks, revealing substantial performance gaps at all task levels, with state-of-the-art models still struggling on full-stack development.

arXiv

Hippasus: Effective and Efficient Automatic Feature Augmentation for Machine Learning Tasks on Relational Data

Serafeim Papadias, Kostas Patroumpas, Dimitrios Skoutas

ML models critically depend on feature quality, yet in real-world settings, useful features are often distributed across multiple relational tables rather than a single dataset. Feature augmentation addresses this problem by automatically discovering and joining additional tables to enrich a base table with predictive features. However, scaling feature augmentation to complex schemas with many tables and multi-hop relationships is challenging. It requires exploring a large space of join paths, executing costly joins, and selecting useful features from noisy results. Existing approaches suffer from either limited effectiveness or efficiency. Restricting exploration to simple joins limits predictive performance, while more expressive methods rely on expensive training data, lack scalability, or fail to fully exploit schema-level semantics. We present Hippasus, a cost-aware, LLM-augmented feature discovery framework over relational schemas that addresses these challenges. Hippasus combines lightweight statistical signals with adaptive semantic reasoning, invoking stronger (LLM-based) analysis only when necessary. It further introduces efficient multi-way join execution with cross-path feature consolidation, and a hybrid feature selection strategy that integrates statistical relevance with semantic refinement. Experiments on real-world datasets show that Hippasus improves feature augmentation accuracy by up to 26.8% over state-of-the-art methods, while achieving a favorable effectiveness-cost tradeoff.

arXiv

Singular Bayesian Neural Networks

Mame Diarra Toure, David A. Stephens

Bayesian neural networks promise calibrated uncertainty but require $O(mn)$ parameters for standard mean-field Gaussian posteriors. We argue this cost is often unnecessary, particularly when weight matrices exhibit fast singular value decay. By parameterizing weights as $W = AB^{\top}$ with $A \in \mathbb{R}^{m \times r}$, $B \in \mathbb{R}^{n \times r}$, we induce a posterior that is \emph{singular} with respect to the Lebesgue measure, concentrating on the rank-$r$ manifold. This singularity captures structured weight correlations through shared latent factors, geometrically distinct from mean-field's independence assumption. We derive PAC-Bayes generalization bounds whose complexity term scales as $\sqrt{r(m+n)}$ instead of $\sqrt{m n}$, and prove loss bounds that decompose the error into optimization and rank-induced bias using the Eckart-Young-Mirsky theorem. We further adapt recent Gaussian complexity bounds for low-rank deterministic networks to Bayesian predictive means. Empirically, across MLPs, LSTMs, and Transformers on standard benchmarks, our method achieves competitive predictive performance while using up to $33\times$ fewer parameters than 5-member Deep Ensembles. It substantially improves OOD detection and often improves calibration relative to mean-field and perturbation baselines, while Deep Ensembles can still be stronger on in-distribution likelihood-based metrics.

arXiv

VAE with Hyperspherical Coordinates: Improving Anomaly Detection from Hypervolume-Compressed Latent Space

Alejandro Ascarate, Leo Lebrat, Rodrigo Santa Cruz, Clinton Fookes, Olivier Salvado

Variational autoencoders (VAE) encode data into lower-dimensional latent vectors before decoding those vectors back to data. Once trained, one can hope to detect out-of-distribution (abnormal) latent vectors, but several issues arise when the latent space is high dimensional. This includes an exponential growth of the hypervolume with the dimension, which severely affects the generative capacity of the VAE. In this paper, we draw insights from high dimensional statistics: in these regimes, the latent vectors of a standard VAE are distributed on the `equators' of a hypersphere, challenging the detection of anomalies. We propose to formulate the latent variables of a VAE using hyperspherical coordinates, which allows compressing the latent vectors towards a given direction on the hypersphere, thereby allowing for a more expressive approximate posterior. We show that this improves both the fully unconditional-OOD and conditional-OOD anomaly detection ability of the VAE, achieving the best performance on the datasets we considered, outperforming existing methods. For the unconditional-OOD and conditional-OOD modalities, respectively, these are: i) detecting unusual landscape from the Mars Rover camera and unusual Galaxies from ground based imagery (complex, real world datasets); ii) standard benchmarks like Cifar10 and subsets of ImageNet as the in-distribution (ID) class.

arXiv

SCA: Segment-Wise CoT Compression with Answer Alignment

Ye Tian, Hongyu Lin

Chain-of-thought (CoT) reasoning improves problem solving, but long think traces increase inference cost. Existing CoT compression methods usually optimize completion-level length. For structured thinking models, however, a completion contains both a think segment and an answer segment, so completion-level compression can save tokens by compressing not only the CoT but also the answer. We call this failure mode answer drift. We propose Segment-wise CoT Compression with Answer Alignment (SCA), an answer-preserving think-compression method. SCA parses completions into functional segments, routes compression rewards only to successful think tokens, and protects answer tokens through length and distribution alignment to a frozen base model. Experiments show that, across datasets from multiple domains, SCA achieves state-of-the-art-level chain-of-thought compression while preserving the base model's performance and answer alignment. Training data and code are included in the supplementary code and data package.

arXiv

No Certificate, No Categorical Speech Act: A Brouwerian Assertibility Constraint for Public Reason

Michael Jülich

Generative AI can convert uncertainty into authoritative-seeming verdicts, displacing the justificatory work on which democratic epistemic agency depends. As a corrective, I propose a Brouwer-inspired assertibility constraint for responsible AI: in high-stakes domains, systems may assert or deny claims only if they can provide a publicly inspectable and contestable certificate of entitlement; otherwise they must return Undetermined. This constraint yields a three-status interface semantics (Asserted, Denied, Undetermined) whose statuses mark entitlement to categorical speech rather than truth values of the underlying world-claim. The framework distinguishes internal entitlement from public standing while connecting them through the certificate as a boundary object. Philosophically, it develops a form of applied intuitionism: resistance to unwarranted completeness becomes a norm of public machine speech, under which categorical commitment requires exhibited and contestable entitlement. I operationalize the constraint through structured certificates, decision-layer gates for threshold and argmax predicates, and a reason-coded output contract. A design lemma shows that, under a refutation-soundness condition on the contractually specified negative side, certificate-sound binary totality presupposes witnessed decidability on the declared scope. Under the output contract, Undetermined is therefore not a tunable reject option but a mandatory status whenever the available certificates do not uniquely license assertion or denial. By making outputs answerable to contestable warrants rather than confidence alone, the paper aims to preserve epistemic agency where automated speech enters public justification.

arXiv

Rooted Absorbed Prefix Trajectory Balance with Submodular Replay for GFlowNet Training

Xi Wang, Wenbo Lu, Shengjie Wang

Generative Flow Networks (GFlowNets) enable fine-tuning large language models to approximate reward-proportional posteriors, but they remain prone to mode collapse, manifesting as prefix collapse and length bias. We attribute this to two factors: (i) weak credit assignment to early prefixes, and (ii) biased replay that induces a shifted, non-representative training flow distribution. We propose Rooted absorbed prefix Trajectory Balance RapTB, an objective that anchors subtrajectory supervision at the root and propagates terminal rewards to intermediate prefixes via absorbed suffix-based backups, providing dense prefix-level learning signals. To mitigate replay-induced distribution shift, we further introduce SubM, a submodular replay refresh strategy that promotes both high reward and diversity. Empirically, on tasks such as molecule generation with LLM using SMILES strings, RapTB combined with SubM consistently improves optimization performance and molecular diversity while preserving high validity. The code is released on https://github.com/ComDec/ChemGFN.

arXiv

Evaluating Style-Personalized Text Generation: Challenges and Directions

Anubhav Jangra, Bahareh Sarrafzadeh, Silviu Cucerzan, Adrian de Wynter, Sujay Kumar Jauhar

With the surge of large language models (LLMs) and their ability to produce customized output, style-personalized text generation--"write like me"--has become a rapidly growing area of interest. However, style personalization is highly specific, relative to every user, and depends strongly on the pragmatic context, which makes it uniquely challenging. Although prior research has introduced benchmarks and metrics for this area, they tend to be non-standardized and have known limitations (e.g., poor correlation with human subjects). LLMs have been found to not capture author-specific style well, it follows that the metrics themselves must be scrutinized carefully. In this work we critically examine the effectiveness of the most common metrics used in the field, such as BLEU, embeddings, and LLMs-as-judges. We evaluate these metrics using our proposed style discrimination benchmark, which spans eight diverse writing tasks across three evaluation settings: domain discrimination, authorship attribution, and LLM-generated personalized vs non-personalized discrimination. We find strong evidence that employing ensembles of diverse evaluation metrics consistently outperforms single-evaluator methods, and conclude by providing guidance on how to reliably assess style-personalized text generation.

arXiv

Long Range Frequency Tuning for QML

Michael Poppel, Markus Baumann, Sebastian Wölckert, Claudia Linnhoff-Popien, Jonas Stein

Angle-encoded variational quantum circuits admit a truncated Fourier series representation of their output, but approximating functions with maximum frequency $ω_{\max}$ using fixed unary encoding requires $\mathcal{O}(ω_{\max})$ encoding gates. Trainable-frequency (TF) circuits promise a reduction by learning the data-encoding prefactors alongside the ansatz parameters, adapting the accessible frequency spectrum to the target during training. We identify a practical barrier that prevents this promise from being realized: the prefactor gradient is suppressed by the spectral gap between the circuit's accessible frequencies and the target spectrum, independently of the ansatz parameters, confining gradient-driven prefactor movement to a narrow neighborhood of initialization. We propose \emph{ternary grid initialization} -- setting prefactors to $\{1, 3, 9, \ldots, 3^{k-1}\}$ -- which ensures every target frequency within $[-ω_{\max}, ω_{\max}]$ lies within $\tfrac{1}{2}$ unit of the accessible spectrum at initialization, so that the spectral-gap bound no longer constrains the target-driven gradient to be small. This is a necessary condition for reliable convergence, whose sufficiency we establish empirically. On a synthetic benchmark with target frequencies shifted well beyond the standard initialization range, ternary initialization achieves median $R^2 = 0.997$ versus $0.18$ for unary initialization, with $100\%$ of runs achieving $R^2 > 0.95$ against $0\%$. CMA-ES with $20\times$ the evaluation budget reaches only $25\%$ success, confirming the limitation is a property of the optimization landscape rather than of gradient-based optimization specifically. Real-world validation on two benchmark datasets demonstrates consistent advantages over both fixed and trainable unary baselines.

arXiv

Arbor: A Framework for Reliable Navigation of Critical Conversation Flows

Luís Silva, Diogo Gonçalves, Catarina Farinha, Clara Matos, Luís Ungaro

Large language models struggle to maintain strict adherence to structured workflows in high-stakes domains such as healthcare triage. Monolithic approaches that encode entire decision structures within a single prompt are prone to instruction-following degradation as prompt length increases, including lost-in-the-middle effects and context window overflow. To address this gap, we present Arbor, a framework that decomposes decision tree navigation into specialized, node-level tasks. Decision trees are standardized into an edge-list representation and stored for dynamic retrieval. At runtime, a directed acyclic graph (DAG)-based orchestration mechanism iteratively retrieves only the outgoing edges of the current node, evaluates valid transitions via a dedicated LLM call, and delegates response generation to a separate inference step. The framework is agnostic to the underlying decision logic and model provider. Evaluated against single-prompt baselines across 10 foundation models using annotated turns from real clinical triage conversations. Arbor improves mean turn accuracy by 29.4 percentage points, reduces per-turn latency by 57.1%, and achieves an average 13.8x reduction in per-turn cost. These results indicate that architectural decomposition reduces dependence on intrinsic model capability, enabling smaller models to match or exceed larger models operating under single-prompt baselines.

arXiv

UnMaskFork: Test-Time Scaling for Masked Diffusion via Deterministic Action Branching

Kou Misaki, Takuya Akiba

Test-time scaling strategies have effectively leveraged inference-time compute to enhance the reasoning abilities of Autoregressive Large Language Models. In this work, we demonstrate that Masked Diffusion Language Models (MDLMs) are inherently amenable to advanced search strategies, owing to their iterative and non-autoregressive generation process. To leverage this, we propose UnMaskFork (UMF), a framework that formulates the unmasking trajectory as a search tree and employs Monte Carlo Tree Search to optimize the generation path. In contrast to standard scaling methods relying on stochastic sampling, UMF explores the search space through deterministic partial unmasking actions performed by multiple MDLMs. Our empirical evaluation demonstrates that UMF consistently outperforms existing test-time scaling baselines on complex coding benchmarks, while also exhibiting strong scalability on mathematical reasoning tasks.

arXiv

Evaluating LLMs When They Do Not Know the Answer: Statistical Evaluation of Mathematical Reasoning via Comparative Signals

Zihan Dong, Zhixian Zhang, Yang Zhou, Can Jin, Ruijia Wu, Linjun Zhang

Evaluating mathematical reasoning in LLMs is constrained by limited benchmark sizes and inherent model stochasticity, yielding high-variance accuracy estimates and unstable rankings across platforms. On difficult problems, an LLM may fail to produce a correct final answer, yet still provide reliable pairwise comparison signals indicating which of two candidate solutions is better. We leverage this observation to design a statistically efficient evaluation framework that combines standard labeled outcomes with pairwise comparison signals obtained by having models judge auxiliary reasoning chains. Treating these comparison signals as control variates, we develop a semiparametric estimator based on the efficient influence function (EIF) for the setting where auxiliary reasoning chains are observed. This yields a one-step estimator that achieves the semiparametric efficiency bound, guarantees strict variance reduction over naive sample averaging, and admits asymptotic normality for principled uncertainty quantification. Across simulations, our one-step estimator substantially improves ranking accuracy, with gains increasing as model output noise grows. Experiments on GPQA Diamond, AIME 2025, and GSM8K further demonstrate more precise performance estimation and more reliable model rankings, especially in small-sample regimes where conventional evaluation is pretty unstable.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.