NeurIPS 2026 Papers

Street-level bureaucrats, such as caseworkers, triage nurses, and school administration, operate under a fundamental tension. The official policy on who to prioritize for services may conflict with bureaucrats estimation of who would benefit most from those services. The social theory of street-level bureaucracy argues that frontline agents use discretionary authority to bridge this gap, overriding policy recommendations when professional judgment identifies cases where doing so would meaningfully improve outcomes. As algorithmic recommendations become embedded in allocation settings, understanding the structure of this discretionary behavior is fundamental to designing tools that account for, rather than suppress, this value of professional judgment. We formalize discretion as a finite-horizon dynamic allocation problem. An agent implements a default policy but holds a limited budget of K overrides across T periods. Each period, the agent observes whether deviating from the default would yield a welfare gain and must decide whether to spend a scarce override now or conserve it for future opportunities. We show that the optimal policy is a state-dependent threshold rule and prove a behavioral invariance: for location-scale families of improvement distributions, the rate at which an optimal agent exercises discretion is independent of the amount of welfare gains and depends only on the distribution's shape. Fat-tailed gains induce patience (conserving overrides for rare, high-stakes cases); thin-tailed gains induce routine, front-loaded spending. We also compare the conceptual implications of our model with operational data from a homelessness service system. Override rates shift with short-run capacity openings in scarce housing, rising when recent exits free slots, and falling when interventions are congested. This could be the result of a perceived increase in the discretionary budget. Further, the temporal pattern of spending is front-loaded: override rates peak on Mondays and early in the fiscal year, consistent with the routine, early-spending behavior our model predicts under thin-tailed improvement distributions. These findings identify a systematic structure in both human and theoretically optimal decision-making, with direct implications for the design of human-AI allocation systems that are both more efficient and equitable.

Explainable AI (XAI) offers far-reaching consequences for human-AI collaboration, but the fundamental question of how best to evaluate these systems has proven frustratingly elusive. Methodologies typically revolve around either proxy metrics which tell us little about the utility of an explanation, or human evaluation which---while generally considered the gold standard---is expensive, difficult to scale, and challenging to reproduce. In this paper, we investigate if Multimodal Large Language Models (MLLMs) can act as surrogate human participants to automate the evaluation of explainable AI. To do this, we first propose a ``purpose-grounded'' evaluation framework with specific metrics applicable to both MLLMs and human users alike. Then, we offer some conceptual insights as to why and when we can trust these models to automate this process. Lastly, we conduct extensive empirical testing across five purpose-grounded areas and nine popular MLLM APIs. Results indicate that all models trend positively, but OpenAI models (particularly GPT-5 mini) are the best aligned at replicating user responses, and for complex visual reasoning tasks more advanced modes (like GPT-5) perform best, which would be the recommended choices in practice. We expect our insights not to replace user testing, but rather to help XAI researchers complement their current evaluation methodologies and move the field away from heavy reliance on proxy metrics for measuring explanation utility.

Learning semantically meaningful human movement representations from 3D pose sequences is essential for human behavior modeling. Existing self-supervised methods derive supervision from artificial assumptions, e.g., invariance to augmentation within a certain range or recoverability of masked regions at a certain temporal scale, introducing unintuitive hyperparameters that make the learned semantics sensitive to their settings. We propose BAL, a self-supervised framework that instead exploits supervision intrinsic to human movement: the natural ordering structure of everyday behavior. BAL autoregressively predicts in a semantically abstract latent space, analogous to how humans anticipate others' actions rather than their precise joint configurations. Bidirectional autoregressive prediction, both forward and backward in time, fully leverages these ordering constraints while enabling representation extraction using both past and future context at inference. To suppress representation collapse, BAL decomposes each prediction target into a global component capturing sequence-level semantics and a local component capturing context-independent movement. We prove that the local targets retain nontrivial angular diversity unless the representations are completely collapsed, preserving a meaningful training signal. Experiments on large-scale datasets demonstrate that BAL's representations are highly effective for motion captioning, forecasting, and interpolation, outperforming existing self-supervised methods across all tasks.

Multimodal Large Language Models (MLLMs) have achieved impressive performance across a wide range of reasoning tasks, yet it remains unclear whether their internal representations and decision criteria align with the latent structure of human cognition. In this work, we propose a novel framework to align MLLMs with the multidimensional mental representations underlying human similarity judgements. We build on a previously established behavior-derived semantic embedding space, in which objects are represented along 66 interpretable dimensions derived from 4.70 million human Odd-One-Out (O1O) judgements. For each triplet, we infer the most salient decision dimension and convert it into a dimension-grounded linguistic rationale. This enables MLLMs to learn both the final choice and a justification consistent with the inferred behavioral criterion. Experimental results demonstrate that our approach improves model consistency with human similarity judgements while retaining competitive performance on standard multimodal benchmarks. Furthermore, using searchlight Representational Similarity Analysis (RSA) on an independent fMRI Natural Scenes Dataset (NSD), we observe increased representational alignment between model and human. Ultimately, our findings point to a data-driven route for incorporating human cognitive structure into MLLMs alignment.

Foundation models are increasingly deployed as agentic systems that perceive context and act autonomously, raising the need to evaluate their moral behavior as active decision-makers. Existing benchmarks, however, largely ask models to judge human actions from third-person textual vignettes, leaving untested how models behave when they themselves become the acting agent. We introduce ABHBench, a multimodal benchmark built from \emph{Detroit: Become Human}, an interactive narrative game, to evaluate how foundation models make moral decisions in first-person human-android dilemmas. ABHBench places foundation models in branching narrative scenarios with visual observations, dialogue, memory, and action choices. Inspired by Asimov's Three Laws of Robotics, our framework measures Harmlessness, Obedience, and Self-preservation to capture the core tensions in human-AI relations, along with Autonomy, Stability, and Certainty to assess decision-making patterns. Across 12 foundation models, we find that moral behavior changes substantially under situated agentic context: 75\% of models shift significantly between a text-only abstraction and the matched multimodal DBH scenario, and reasoning-enhanced models show no monotonic gains in consistency or human-oriented behavior. These results highlight the necessity of evaluating moral behavior in situated agentic contexts rather than relying on third-person judgment alone.