NeurIPS 2026 论文列表

We introduce Panoptic Saliency Ranking (PSR), a new task that aims to estimate the visual saliency of all objects within a scene. Unlike salient object ranking, which focuses only on ranking salient regions, PSR extends the problem to the entire scene, providing a holistic and fine-grained understanding of visual saliency. To facilitate this task, we construct PSR18K, the first large-scale benchmark for panoptic saliency ranking. It contains 17,861 images with 146,460 annotated object instances spanning 13 superclasses and 376 fine-grained categories. In addition, PSR18K provides relation graph annotations to explicitly model semantic and spatial relationships among objects. We further propose an efficient Intrinsic Relation Graph Network (IRGNet), which reinterprets Transformer self-attention as an implicit relational matrix and seamlessly transforms it into an explicit relation graph, thereby obviating the need for additional relation modeling and significantly improving inference speed. Extensive experiments on PSR18K and three widely used SOR benchmarks demonstrate that our IRGNet achieves new state-of-the-art performance and faster inference with fewer parameters. The dataset and code are publicly available at https://sites.google.com/view/PSR18K.

Multimodal large language models (MLLMs) still struggle with spatial understanding under the dominant perspective-image paradigm, which inherits the narrow field of view of human-like perception. For navigation, robotic search, and 3D scene understanding, 360$^\circ$ panoramic sensing offers a form of supersensing by capturing the entire surrounding environment at once. However, existing MLLM pipelines typically decompose panoramas into multiple perspective views, leaving the spherical structure of equirectangular projection (ERP) largely implicit. In this paper, we study pano-native understanding, which requires an MLLM to reason over an ERP panorama as a continuous, observer-centered space. To this end, we first define the key abilities for pano-native understanding, including semantic anchoring, spherical localization, reference-frame transformation, and depth-aware 3D spatial reasoning. We then build a large-scale metadata construction pipeline that converts mixed-source ERP panoramas into geometry-aware, language-grounded, and depth-aware supervision, and instantiate these signals as capability-aligned instruction tuning data. On the model side, we introduce PanoWorld with Spherical Spatial Cross-Attention, which injects spherical geometry into the visual stream. We further construct PanoSpace-Bench, a diagnostic benchmark for evaluating ERP-native spatial reasoning. Experiments show that PanoWorld substantially outperforms both proprietary and open-source baselines on PanoSpace-Bench, H$^\ast$Bench, and R2R-CE Val-Unseen benchmarks. These results demonstrate that robust panoramic reasoning requires dedicated pano-native supervision and geometry-aware model adaptation. All source code and proposed data will be publicly released.

Existing human-activity datasets typically cover only one or two sensor channels and often focus on short, isolated actions, leaving long-duration compositional tasks largely unsupported as benchmarks for temporal cross-modal sensorimotor learning. We introduce PULSE ($\mathbf{P}$hysiological $\mathbf{U}$nified $\mathbf{L}$ong-duration $\mathbf{S}$ynchronized $\mathbf{E}$mbodiment), a synchronized five-modality dataset that enables benchmarking temporal cross-modal sensorimotor learning: how models recognize ongoing activities, anticipate future contact, reconstruct unobserved signals, and remain robust when physiological, motion, gaze, inertial, and tactile channels are partially observed. Collected from 40 volunteers across 8 ecologically valid daily-activity scenarios together with a dedicated motion-primitive collection, PULSE records full-body optical motion capture with finger-level hand articulation, surface EMG, binocular eye tracking, wearable IMU, and a fingertip pressure array recording quantitative grip force, all sampled at 100 Hz. Each scenario is a long-horizon compositional task containing many smaller sub-tasks, yielding over 7,700 annotated action segments. Each segment is tagged with a motor primitive, the hand involved, the manipulated object, and a natural-language description with four paraphrased variants. On top of the dataset, we define multiple benchmark tasks for temporal cross-modal sensorimotor learning: scene and fine-grained action recognition, grasp onset anticipation, missing-modality robustness, tactile-driven sub-second grasp-state prediction, cross-modal pressure reconstruction, etc. Together these tasks evaluate how information transfers across physiology, motion, gaze, and touch, rather than reducing the dataset to an activity-label catalog. We evaluate three backbone architectures, nine fusion strategies, seven published baselines, and task-specific models including SyncFuse and DailyActFormer. Detailed task definitions and results are in the Appendices. Data, annotations, and baseline code will be publicly released.

RGB cameras struggle on the road surface itself: pavement is nearly texture-less at the centimetre scale, and dry, wet, and icy asphalt can look photometrically identical. Polarization is the natural complement, since Fresnel optics ties the angle of linear polarization (AoLP) to surface-normal azimuth and the degree of linear polarization (DoLP) to refractive index, but no public dataset has made it possible to test whether this physical promise translates into measurable gains on real roads. We introduce PRISM, a polarimetric road-surface dataset of 47,098 time-synchronized frames combining trichromatic linear polarization, co-boresighted RGB, 128-channel LiDAR, and RTK-GNSS/INS, collected across proving-ground and open-road environments under clear, overcast, rainy, foggy, and snowy conditions. Dense road-surface elevation ground truth, validated against as-designed proving-ground geometry, accompanies frame-level labels over five materials and five surface states. Two benchmark tracks are defined on these data: road-surface condition classification and bird's-eye-view elevation estimation. Both share a five-variant input ablation crossing RGB, monochromatic and trichromatic polarization, and their combinations. Reference baselines turn the physical promise into a measurable one. Adding trichromatic polarization to RGB reduces elevation MAE from 3.61 to 3.26cm and improves every threshold metric, while yielding consistent gains on condition classification across backbones. The improvement is task-conditional: trichromatic resolution drives the elevation gain, where AoLP encodes geometric information that intensity cannot, and stratified evaluation localizes where polarimetric channels matter most. PRISM, together with reference baselines, evaluation and stratification scripts, and a datasheet, is publicly available at \url{https://huggingface.co/datasets/NeurIPS-2026-PRISM/PRISM-Dataset}.

The Physics-Informed Neural Network (PINN) literature has produced a rapidly expanding catalogue of methods. We divide a PINN training pipeline systematically into four components and observe that many existing benchmarks study these components in isolation. These components are domain-importance adaptation, loss balancing, the neural network architecture, and the optimization scheme. Moreover, the studies set the surrounding and not considered components of the pipeline to a minimal default that is no longer representative of current best practice approaches. Conclusions drawn under such conditions lead to suboptimal results. In this work, we propose a new evaluation protocol that (1) treats the four components as an integrated system; (2) requires as a key aspect a competitive implementation of the components not currently under investigation; (3) specifies the metrics and reporting format, and (4) is accompanied by structured documentation that ensures the traceability of evaluations across studies. Based on this comprehensive protocol, we release with PINNeval an innovative JAX package containing 18 different methods from the current literature as well as nine efficiently implemented benchmark problems. Using PINNeval, we illustrate our evaluation protocol in relation to established benchmark studies and report on results that challenge established assumptions.

We introduce PINNBench, a benchmark and evaluation study for training-policy selection in hybrid PINN-operator PDE solvers, contributed to the NeurIPS 2026 Evaluations and Datasets (ED) Track. PINNBench comprises 1,539 recorded runs and probes across 13 PDE configurations drawn from 8 equation families, 3–5 training policies per PDE, and 10 seeds per condition, with seven standardized metrics, Wilcoxon signed-rank tests, paired bootstrap 95% confidence intervals, Cohen's d, and Benjamini–Hochberg FDR correction. Baseline evaluation of 5 routing methods establishes three findings that bear directly on how PDE solvers should be evaluated. (i) No single policy dominates: hybrid_full wins 7/13, hybrid_no_gate 4/13, pinn_only 2/13. (ii) Accuracy and regret dissociate: under relative L2, a physics-loss heuristic and a learned Random Forest router reach identical 62% top-1 accuracy, but the heuristic suffers mean regret 0.0850 versus 0.0011 — a 77x gap. Evaluation by accuracy alone is misleading for PDE policy routing. (iii) Causal-loss effectiveness is stage-mediated: 7/9 PDEs improve with pretraining (+5.9% mean) while 5/9 degrade without it (-4.2%). A new leave-one-PDE-family-out evaluation shows that the strongest family-level generalizer uses no PDE-identity features, addressing the concern that PDE-aware routing memorizes PDE identity. All 1,539 result JSONs, PDE implementations, policy configurations, statistical pipeline, and routing infrastructure are released as open-source artifacts.

Shapley value-based explanations are widely used to attribute model predictions, but standard variants capture associative rather than causal relationships. Recent causally informed Shapley methods add a path-wise decomposition, but suffer from two key limitations: they assign zero attribution to chain-mediated paths, and their per-path values do not align with classical mediation estimands. We propose PE-SHAP, a Shapley-inspired path-wise decomposition built on the portion-eliminated effect. Per-mediator values sum to the total mediated effect at every sample, aggregate to the Natural Indirect Effect at the population level, and match the Path-Specific Effect under additive separability. These are properties prior Shapley-based path-wise methods do not provide. We validate PE-SHAP analytically and through Monte-Carlo experiments on synthetic SCMs with known path effects, and demonstrate it on real-world fairness case studies using the German Credit and COMPAS datasets.

Layout-to-Image generation has made substantial progress in image generation with spatial and object-level control. However, existing methods still struggle with complex scenes containing many overlapping and interacting objects. We argue that training data is a particular bottleneck: existing datasets lack examples with dense, complex, object interactions. To address this gap, we introduce **OverLay++**, a large-scale Layout-to-Image dataset with structurally complex scenes. OverLay++ contains approximately 500K images with an average of 6.6 objects per image, exceeding existing datasets by $1.67\times$ in annotation density. Beyond annotation density, OverLay++ provides rich semantic detail with object captions over six times longer than current datasets. Our dataset generation pipeline is simple and robust, producing accurate overlapping regions with rich per-object captions. Across multiple benchmarks, state-of-the-art Layout-to-Image methods trained on OverLay++ dataset show consistent improvement and faster convergence demonstrating the importance of dense, overlap-aware, and caption-rich supervision for controllable image generation.

Machine learning models often suffer performance degradation under subpopulation shift, particularly when spurious correlations cause models to rely on shortcut features that fail to generalize across subgroups. A recent line of work mitigates this issue by using loss-based signals to identify informative samples, but these signals can become severely distorted under label noise: mislabeled samples may also incur large losses and contaminate subsequent reweighting or retraining. Despite its practical importance, this intersection remains largely underexplored. We propose POTER, a reweighting framework based on optimal transport that derives sample importance from the transport geometry between the training distribution and a reference distribution constructed from limited validation group annotations. By measuring alignment at the individual-sample level rather than relying on loss, POTER downweights mislabeled or strongly bias-aligned samples while assigning higher importance to samples better aligned with the reference distribution. In addition, POTER requires only a single ERM training stage, moving beyond the retraining paradigm common in recent work. Across standard benchmarks and noisy-label settings, POTER achieves state-of-the-art worst-group accuracy, including cases where label corruption is concentrated within minority subgroups.

Learning to generalise from limited data is a fundamental challenge for both artificial and biological systems. A common strategy is to extract reusable structure from abundant unlabelled data, enabling efficient adaptation to new tasks from limited labelled data. This two-stage paradigm is now standard in modern training pipelines, where pretraining is followed by fine-tuning or linear probing. We provide an analytical model of this process: structure extraction is formalized as principal component analysis on unlabelled data, and downstream learning as linear regression on a separate labelled dataset. In the high-dimensional regime, we derive exact expressions for training and generalisation error showcasing their dependence on representation dimensionality, unlabelled and labelled sample sizes, and task alignment. Our results show that pretrained representations strongly influence downstream generalisation, and we characterize the optimal representation size as a function of task parameters: with abundant pretraining data but scarce downstream data, maximally compressed representations are optimal, whereas with limited pretraining data, higher-dimensional representations generalise better. Furthermore, we establish an exact trade-off between pretraining and supervision, quantifying how much unlabelled data is required to replace a single labelled sample. Beyond our idealised model, we observe similar phenomenology in autoencoders and pretrained LLMs. Altogether, we highlight that optimising representation size is critical, giving conditions for when compression during pretraining improves generalisation.

We study in-context learning problems where a Transformer is pretrained on tasks drawn from a mixture distribution $\pi=\sum_{\alpha\in\mathcal{A}} \lambda_{\alpha} \pi_{\alpha}$, called the pretraining prior, in which each mixture component $\pi_{\alpha}$ is a distribution on tasks of a specific difficulty level indexed by $\alpha$. Our goal is to understand the performance of the pretrained Transformer when evaluated on a different test distribution $\mu$, consisting of tasks of difficulty $\beta\in\mathcal{A}$, and with potential distribution shift relative to $\pi_\beta$. In particular, we consider nonparametric regression problems with random smoothness, and multi-index models with both random smoothness and random effective dimension. We prove that a Transformer pretrained on such mixture distributions can adapt to the difficulty of a new task in-context and achieve the optimal prediction rate, uniformly over test distributions in a chi-squared divergence ball. Thus, the pretrained Transformer is able to achieve faster rates of convergence on easier tasks and is robust to distribution shift at test time. Finally, we prove that even if an estimator had access to the test distribution $\mu$, the convergence rate of its expected risk over $\mu$ could not be faster than that of our pretrained Transformers, thereby providing a more appropriate optimality guarantee than minimax lower bounds.

Recent advances in bioacoustics have been driven by large-scale corpora and standardized benchmarks, yet existing resources are overwhelmingly bird-centric and shallow per species, limiting their use for studying the structure of a single species' communication system. This gap is particularly acute for cetaceans: despite bottlenose dolphins (\textit{Tursiops truncatus}) being a compelling case of complex vocal communication among non-human mammals, existing dolphin datasets are small, fragmented, and largely closed. We introduce OpenWhistle, the largest publicly available dataset of dolphin vocalizations. It comprises approximately 180,000 whistles (114 hours) recorded over five years from a stable pod of five individuals in a semi-natural environment, paired with a curated subset of 8,354 expert-annotated whistles and reproducible evaluation protocols for whistle-type detection and classification. We further release the full processing pipeline for whistle detection, segmentation, and categorization. To demonstrate its utility, we pretrain a Wav2Vec2.0 model adapted to dolphin acoustics on the OpenWhistle corpus and show that it learns effective representations, outperforming general-purpose bioacoustic models such as AVES and BioLingual on both tasks while leaving meaningful headroom for future work. By releasing the dataset, pipeline, and evaluation protocol, we provide the first open dolphin whistle dataset tailored for training self-supervised models, laying the groundwork for advancing dolphin communication research and developing models that capture fine-grained acoustic structure within species.

Recent multimodal large language models (MLLMs) show great potential in natural image understanding. Yet, they perform well, mainly on reasoning in-view contents within the image frame. This paper presents the first study on out-of-view (OOV) understanding, i.e., the ability to reason about objects, activities, and scenes beyond the visible frame of a perspective view. Our technical contributions are threefold. First, we design OpenView, a four-stage pipeline that leverages panoramic imagery for large-scale and diverse multi-choice VQA synthesis, where full-scene coverage enables flexible view framing while preserving global scene awareness. Second, we curate OpenView-Dataset, a high-quality synthetic dataset from diverse real-world panoramas to empower MLLMs upon supervised fine-tuning. Third, we build OpenView-Bench, a benchmark that jointly measures choice and rationale accuracy for interpretable and diagnosable evaluation. Experimental results show that despite having a large gap from human performance in OOV VQA answer selection, multiple MLLMs, upon empowered by OpenView, can consistently boost their performance, uplifted from 48.3% to 62.3% on average. Code, benchmark, and data will be released.

Recent advances in diffusion models have significantly elevated the visual fidelity of Virtual Try-On (VTON) systems, yet reliable evaluation remains a persistent bottleneck. Traditional metrics struggle to quantify fine-grained texture details and semantic consistency, while existing datasets fail to meet commercial standards in scale and diversity. We present OpenVTON-Bench, a large-scale benchmark comprising approximately 100K high-resolution image pairs (up to $1536\times1536$). The dataset combines Gemini-powered dense captioning with stratified sampling across 20 visual clusters obtained by DINOv3 K-Means. To support reliable evaluation, we propose a multi-modal protocol that measures VTON quality along five interpretable dimensions: background consistency, identity fidelity, texture fidelity, shape plausibility, and overall realism. The protocol integrates VLM-based semantic reasoning with a novel Multi-Scale Representation Metric based on SAM3 segmentation and morphological erosion, enabling the separation of boundary alignment errors from internal texture artifacts. Under the specified evaluation rubric, our representation metric shows strong agreement with human model rankings (Kendall's $\tau$ of 0.833 vs. 0.611 for SSIM). Code and dataset are available at https://github.com/RenxingIntelligence/OpenVTON-Bench.

We release OpenSanctions Pairs, the first large-scale public benchmark for entity matching on sanctions and OSINT data. The dataset includes 755,540 expert-labeled pairs over 1 million entities, aggregated from 293 source datasets across 45 jurisdictions. It captures real-world diversity in compliance data, spanning multiple languages and writing systems (e.g., Latin, Cyrillic, Arabic), inconsistent structure, and time-varying provenance, and is substantially more heterogeneous than prior entity matching benchmarks. As baselines, we evaluate the production rule-based matcher (nomenklatura RegressionV1) alongside open- and closed-source LLMs in both zero- and few-shot settings, each tested with and without MIPROv2 prompt optimization to control for prompt sensitivity. The rule-based baseline reaches 91.3\% F1; GPT-4o achieves the best result at 99.0\% F1, and a locally deployable open-source model (DeepSeek-R1-Distill-Qwen-14B) achieves 98.2\% F1. The rule-based baseline and LLMs fail in complementary ways: rules over-match, while LLMs struggle with cross-script transliteration. These results suggest that pairwise matching performance is approaching a practical ceiling and shift attention toward pipeline components such as blocking, clustering, and uncertainty-aware review.

We study sequential learning for classification metrics that are non-decomposable across instances and are general functions of the confusion matrix, such as the Jaccard index and the $F$-measure. The learner observes instances sequentially, makes irrevocable predictions, and is evaluated by the final empirical performance metric, leading to an online-regret objective. After observing the sequence, the learner also returns a classifier for future population use, leading to a population-regret objective. This single protocol connects two central frameworks for optimizing non-decomposable metrics: Expected Test Utility (ETU) and Population Utility (PU). For smooth concave utilities of the confusion matrix, we show that the ETU benchmark is controlled, in expectation, by the population PU comparator, and that an algorithm with small online regret also yields PU guarantees by an online-to-batch conversion. We then turn to non-concave linear-fractional metrics. We devise a general stochastic Dinkelbach-root method that tracks the relevant metric parameter online and gives online and population regret of order $n^{-1/2}$, up to logarithmic factors, with explicit dependence on conditional-probability estimation error. Our empirical studies evaluate these methods on benchmark datasets.

Inference-time sampling is a central mechanism for improving large language models, with temperature controlling the tradeoff between reliability and diversity. While recent work suggests that varying temperature across samples can improve multi-sample inference, simple fixed-temperature schemes remain the standard in practice. We investigate this discrepancy in the canonical pass@$K$ setting, asking when temperature diversity is useful, and when a single tuned temperature suffices. To this end, we analyze per-question success probabilities across multiple models and datasets, tracking how they vary with sampling temperature. We identify several consistent properties of these profiles, including that harder questions are less likely to share efficient sampling temperatures, although this trend is only logarithmic in hardness. We then derive an efficient method for evaluating the theoretically optimal multi-temperature allocation scheme, and show that although such a scheme improves performance, its gains over single-temperature baselines are typically modest when temperature is tuned as a function of the sampling budget $K$. We explain this phenomenon through mathematical mechanisms that we call dulling effects, and show how they ensure the robustness of temperature choices across different question distributions. Our results clarify when multi-temperature sampling is useful, and how temperatures should ideally be chosen.

Recent work proposes hypernetworks that convert a document into a low-rank parameter update (LoRA) in a single forward pass, enabling fast injection of new knowledge into a frozen LLM. However, storing separate LoRA adapters for each document can be unsustainable in streaming settings. In this paper, we study the problem of composing hypernetwork-generated LoRAs using two representative systems, Doc-to-LoRA and SHINE. We show that hypernetwork-generated LoRAs have a geometry sharply different from directly fine-tuned LoRAs. Hypernetwork-generated LoRAs are highly spectrally concentrated and strongly aligned, while next-token prediction and QA fine-tuned adapters are diffuse and nearly orthogonal across documents. This suggests that hypernetwork-generated LoRAs are not arbitrary low-rank weight updates, but decoded points on a structured adapter manifold. Guided by this observation, we propose latent-space LoRA merging: instead of merging in the weight space, we compose the intermediate hypernetwork latents and then decode the merged latent through the frozen LoRA-generation head. We show that latent-space merging techniques consistently improve over weight-space and factor-space baselines while preserving a single fixed-rank adapter. These results establish latent-space composition as a promising mechanism for bounded accumulation of hypernetwork-generated LoRAs in long document sequences.

We study the existence of uniformly optimal stationary policies, which are optimal for any starting distribution in Markov Decision Processes (MDPs) under epistemic (parameter) uncertainty within two complementary frameworks. For robust MDPs, we establish that (s- and sa-) rectangular models are equivalent to the existence of uniformly optimal stationary policies and provide several interesting counterexamples, including a simple r-rectangular instance with only history-dependent optimal policies. For ambiguity-averse MDPs where the uncertain transition kernel is a random variable, and the return is evaluated via a law-invariant risk measure, we show that uniformly optimal stationary policies can only exist for the essential supremum or essential infimum (under mild monotone continuity assumptions), thus excluding other risk measures such as CVaR, VaR, and the Entropic Risk Measure. We conclude our work by relating dynamic programming and the existence of uniformly optimal policies. One of the main takeaways of our impossibility results is the need for new models to go beyond uniformly optimal stationary policies.