NeurIPS 2026 论文列表

Synergistic multi-modal question generation aims to generate questions grounded in synergistic semantics distributed across multiple modalities, rather than being specified by isolated information from any single modality alone. In this setting, modality-specific cues and cross-modal shared semantics must be jointly well-organized so that the generated question reflects synergistic multi-modal contribution and captures genuinely synergistic information. However, this task faces two core challenges: *how to disentangle multi-modal information for synergistic information modeling*, and *how to preserve synergistic contribution during question generation*. To address these challenges, we propose **SynMQG**, a synergistic dependency-aware framework derived from a mutual-information-based theoretical analysis, which integrates disentangled representation learning with synergy-aware optimization for synergistic multi-modal question generation. Specifically, SynMQG disentangles the multi-modal context into visual-specific, textual-specific, and shared semantics, thereby explicitly organizing heterogeneous information for synergistic information modeling. Based on these disentangled representations, it further introduces a reasoning chain as an intermediate scaffold to structure cross-modal information before generation. Moreover, SynMQG proposes GRPO with a mutual-information-based synergy reward, which explicitly measures whether the generated question preserves effective contribution of synergistic information, encouraging the model to generate questions that depend on synergistic multi-modal semantics rather than superficial single-modal cues. Experiments on ScienceQA and MultimodalQA show that SynMQG consistently outperforms representative MQG baselines across standard text-generation metrics, MLLM-based evaluation, and human evaluation, while generating questions with stronger synergistic multi-modal grounding and higher multi-modal dependency. The source code is available at https://anonymous.4open.science/r/MQG-8E57.

A central objective in multimodal learning is to capture synergy: task-relevant information that arises only from the joint use of multiple modalities, and is not available from any single modality alone. While most approaches operate at the architectural level through larger or more complex fusion models, we propose a complementary axis: shaping the training objective itself. Standard training often emphasizes unimodal or redundant information, falling short on examples that require cross-modal reasoning. We formalize multimodal synergy through information theory and introduce the Synergistic Information Bottleneck (SynIB), a scalable objective that targets synergy directly. To prioritize learning synergy, SynIB motivates the model to predict accurately from all modalities while penalizing confidence when information from any modality is withheld. Alongside the standard task loss, the model runs forward passes with one modality masked at a time and is penalized for remaining confident, which would indicate reliance on unimodal cues rather than cross-modal interactions. We validate SynIB in two regimes. On synthetic XOR tasks where the ground-truth synergy is known by construction, standard training fails to recover it while SynIB does. On five real-world benchmarks, including three MultiBench affective tasks, Hateful Memes with CLIP-ViT and DeBERTa backbones, and a controllable irony extension of CREMA-D we introduce, SynIB improves accuracy on synergy-dependent examples by up to 7.8\% and overall accuracy by up to 3.8\%.

Geometry problem solving is a canonical testbed for machine intelligence, requiring systems to interpret diagrams, ground symbolic constraints, and perform rigorous deduction. Yet current approaches expose a persistent gap between seeing and proving: multimodal large language models can flexibly inspect diagrams but may hallucinate unsupported relations, while symbolic solvers provide checkable derivations but are brittle to incomplete or misgrounded formalization. We introduce SynGeo, a state-centric framework for synergizing seeing and proving in geometry problem solving by making the geometric representation revisable during inference. SynGeo first constructs a predicate state from diagram--text evidence and tests it with symbolic reasoning. When proof search fails or stagnates, symbolic diagnostics guide image-grounded revisiting, repairing solver-incompatible predicates before another deductive attempt. When the symbolic route remains unresolved, a complementary MLLM branch provides an image-grounded reasoning path from the problem evidence. On Geometry3K and PGPS9K, SynGeo achieves state-of-the-art performance across both Choice and Completion settings. With GPT-4o as the backbone, it reaches \(90.2\%\) accuracy in both settings on Geometry3K, and 90.1% Choice accuracy and 88.6% Completion accuracy on PGPS9K. Ablations show that both feedback-guided revisiting and complementary reasoning are necessary, supporting a broader view of geometry problem solving as an adaptive loop between what a system sees, what it formalizes, and what it proves.

3D Gaussian Splatting enables high-fidelity, real-time novel view synthesis, yet adapting it to dynamic, evolving scenes remains challenging. Existing methods rely on fixed static-dynamic partitioning, which suffers from persistent partitioning errors and cannot adapt during optimization. To tackle this issue, we introduce SynGS, a unified 3D Gaussian framework for continual scene updating and multi-view change detection. We cast static-dynamic decomposition as a learnable task, jointly optimizing per-Gaussian change probabilities and scene representations for iterative refinement. We further propose a change-guided update strategy that modulates illumination-aware photometric corrections via learned probabilities, separating transient appearance variations from authentic scene changes. Projecting Gaussian-level probabilities to the image plane yields 3D-consistent change masks, enabling mutual optimization between reconstruction and change localization. An illumination-decoupled refinement step further improves mask quality. Evaluations on standard benchmarks show that SynGS outperforms baseline methods on scene update and change detection tasks, while fully retaining the real-time rendering capability of 3D Gaussian Splatting. Code will be released upon acceptance.

Large vision-language models (LVLMs) have demonstrated remarkable performance on multimodal reasoning benchmarks, yet their perceptual reliability under physically constrained imaging conditions remains poorly understood. Existing evaluations predominantly assume ideal visual inputs and therefore fail to characterize how camera distance, illumination, viewpoint, and pixel density fundamentally affect semantic recoverability. We introduce SynDORBench, the first physically grounded benchmark for evaluating LVLM perceptual robustness under DORI-calibrated conditions aligned with human visual capability standards. SynDORBench comprises over 54k question--answer pairs generated through a controllable synthetic pipeline that systematically varies viewing distance, lighting, camera geometry, and action pose according to physically interpretable pixel-density regimes. To support scalable low-visibility supervision, we further propose a discernibility annotation framework that propagates human perceptual labels using mask-conditioned statistical features and ensemble learning. We evaluate 16 open-source LVLMs, a commercial LVLM baseline, and YOLO11x across human-presence classification and action recognition tasks under progressively degraded visibility conditions. Our results reveal that perceptual failure in LVLMs is strongly governed by pixel density and physical imaging constraints rather than model scale alone. Surprisingly, several compact open-source LVLMs outperform larger commercial baselines and substantially exceed YOLO11x robustness under long-range and low-light conditions. SynDORBench establishes a new benchmark paradigm for physically grounded multimodal evaluation, enabling systematic analysis of LVLM reliability under real-world perceptual constraints and direct comparison against human visibility thresholds.

Synthetic text generation with Differential Privacy (DP) guarantees emerges as a principled approach that can enable the sharing of sensitive datasets across institutional and regulatory boundaries, while bounding the risks of re-identification and membership inference. LLM-based methods deliver promising results; however, comparisons are exacerbated by differing evaluation setups and "private" datasets, potential pre-training contamination is not considered and guarantees are not verified with DP audits. To advance this field, we introduce a unified evaluation framework with standardised utility and fidelity metrics and privacy audits, encompassing nine curated datasets that capture domain-specific complexities such as technical jargon, long-context dependencies, and specialised document structures. In a large-scale empirical study, we benchmark LLM-based state-of-the-art DP text generators of varying sizes (between 1--8B). Our results indicate that DP synthetic text generation remains an unsolved challenge, with quality deteriorating more as the private datasets deviate further from the generators' pre-training corpora. Our novel synthetic text membership inference attack (MIA) explains this observation: Synthetic data quality is overestimated when LLMs have been pre-trained---without DP---on portions of the "private" data to be generated. Finally, our work provides the first quantitative evidence that this "public pre-training and private generation" paradigm invalidates the guaranteed privacy bounds of real-world private datasets.

Programmable photonic circuits are a natural platform for physical learning, where the learned computation is performed by the hardware itself rather than by a digital model. We propose a physical learning algorithm and architecture for continuous-variable quantum systems where a photonic circuit is configured in situ to invert an unknown lossless Gaussian quantum operation. Our contributions are threefold. (i) Theory: we prove that generic $S \in \mathrm{Sp}(2 N, \mathbb{R})$ admits a triangular decomposition into two-mode and single-mode symplectic gates. (ii) Architecture: we introduce the symplectic Reck mesh, a triangular array of two-mode symplectic gates that physically realizes this factorization, generalizing the unitary Reck mesh widely used in optical neural networks. (iii) Physical learning: when placed after an unknown Gaussian unitary, the mesh is trained in situ by nulling local response blocks one at a time using simple optical probes and measurements. We identify a measure that predicts when training becomes hard and numerically demonstrate finite-shot training and robustness to gate errors. Our work extends self-configuring optical networks from unitary to symplectic transformations, bringing physical learning to continuous-variable quantum photonic hardware.

Neural Hamiltonian models provide a principled approach to learning dynamical systems by embedding the symplectic structure of physical evolution into the model class. However, this geometric prior alone has not made Hamiltonian neural networks efficient long-horizon simulators: an M-step trajectory is still typically generated by M sequential updates, and common training objectives based on force matching or one-step prediction provide only local supervision. We introduce Symplectic Parallel Scan (SPS), a neural Hamiltonian framework that algebraizes the learned dynamics through a finite-dimensional Poisson algebra of Hamiltonian generators. SPS constructs a Neural--Poisson Composition DAG closed under projected products and Poisson brackets, whose elements induce symplectic Hamiltonian drivers organized as a Lie-group walk. The associativity of this flow composition enables a parallel prefix-scan over Hamiltonian drivers, reducing the sequential depth of trajectory propagation from $\mathcal O(M)$ to $\mathcal O(\log M)$ while keeping each prefix inside the symplectic flow family. The resulting model is trained through a global trajectory-matching objective rather than only local force labels. Across quantum spin and molecular dynamics benchmarks, SPS achieves substantial wall-clock acceleration over sequential HNN baselines, improves long-horizon prediction accuracy, and reduces energy drift and symplectic violation under extrapolation.

The modeling and simulation of infinite-dimensional Hamiltonian systems are central problems in mathematical physics and engineering, but they pose significant computational and structural challenges for standard data-driven architectures. In this work, we introduce the Symplectic Neural Operator, a neural operator architecture designed to preserve the symplectic structure intrinsic to Hamiltonian PDEs. We provide a theoretical characterization of their symplecticity and establish a rigorous long-term stability result based on the combination of symplectic structure preservation and learning accuracy. Numerical experiments on canonical Hamiltonian PDEs corroborate this theoretical result and show that SNOs exhibit improved energy behavior compared with non-structure-preserving neural operators.

Variational inference (VI) is a central tool in modern machine learning, used to approximate an intractable target density by optimising over a tractable family of distributions. As the variational family cannot typically represent the target exactly, guarantees on the quality of the resulting approximation are crucial for understanding which of its properties VI can faithfully capture. Recent work has identified instances in which symmetries of the target and the variational family enable the recovery of certain statistics, even under model misspecification. However, these guarantees are inherently problem-specific and offer little insight into the fundamental mechanism by which symmetry forces statistic recovery. In this paper, we overcome this limitation by developing a general theory of symmetry- induced statistic recovery in variational inference. First, we characterise when variational minimisers inherit the symmetries of the target and establish conditions under which these pin down identifiable statistics. Second, we unify existing results by showing that previously known statistic recovery guarantees in location–scale families arise as special cases of our theory. Third, we apply our framework to distributions on the sphere to obtain novel guarantees for directional statistics in von Mises–Fisher families. Together, these results provide a modular blueprint for deriving new recovery guarantees for VI in a broad range of symmetry settings.

Explanations of multiple instance learning (MIL) models are widely used for validation and discovery in digital histopathology. Existing methods primarily rely on heatmaps that highlight influential regions but do not explain how evidence from different tissue regions is combined to produce a prediction. This limits interpretability, especially when decisions depend on interactions between tissue features. We introduce Symbolic explainable MIL (Symb-xMIL), a post-hoc explanation framework that quantifies how a MIL model’s behavior aligns with human-readable decision rules, expressed as logical relationships (e.g., AND, OR, NOT) between input features. These alignment scores reveal semantic patterns underlying the model’s predictions. We evaluate Symb-xMIL on synthetic and real-world histopathology datasets. On synthetic MIL data, Symb-xMIL reliably recovers ground-truth logical rules. In a clinical tumor detection task, the best-aligned rules uncover heterogeneous decision patterns and expose hidden model errors. On an HPV-prediction task on TCGA-HNSCC, a cohort of head and neck cancer, our framework refines patient survival stratification beyond HPV status with potential clinical relevance. Overall, Symb-xMIL extends MIL explainability beyond visual attribution toward structured, rule-based reasoning, enabling more transparent and semantically grounded interpretation of model predictions.

Code-switching (CS) involves shifting between two or more languages during a conversation or utterance, a phenomenon often driven by social context and the identity of the speaker. The scarcity of authentic CS data remains a critical bottleneck in developing robust Automatic Speech Recognition (ASR) systems for bilingual communities. While Large Language Models (LLMs) have been employed to synthesize CS data, direct generation approaches typically produce utterances that are pragmatically unnatural and fail to capture the nuanced stylistic variations of real-world bilingual speakers. In this paper, we propose an archetype-driven code-switching dialogue generation framework via agent-based digital cloning. Moving beyond static prompting, our method facilitates autonomous conversations between multi-agent digital clones imbued with specific bilingual archetypes and established linguistic theories. The generation pipeline integrates constrained persona sampling, real-world topic injection, turn-by-turn accommodation dynamics, and multi-dimensional language checking. Applying this framework, we synthesize highly idiomatic bilingual dialogues across 14 diverse language pairs with 260K textual dialogues (1.04M turns of text samples) and convert them into over 1,558 hours of high-fidelity audio data, yielding the SwitchLingua V2 dataset. To rigorously validate our approach and the generated data, we employ a comprehensive evaluation protocol including both advanced LLM assessments and human bilingual expert evaluations. Extensive experiments demonstrate that SwitchLingua V2 produces CS dialogues that are structurally diverse, pragmatically motivated, and socially situated, holding the potential to offer a highly scalable and reliable pathway for advancing multilingual ASR research.

Mixture-of-experts (MoE) is a common approach for increasing parameter capacity, but applying MoE to state space model (SSM) token mixers can multiply the cost of the recurrent state update. We study how to introduce expert specialization into selective SSMs while preserving computational efficiency. We show that MoE--SSM can refer to two designs: (1) MoE over separated SSMs, which maintains multiple state trajectories and thus scales compute with the number of experts; and (2) MoE-parameterized SSM, which mixes experts in parameter space, maintains a single state trajectory, and evaluates the recurrence once. Our method, Switch Mamba (Swimba), follows the second design by routing over expert-produced SSM streams. Theoretically, we establish well-definedness and stability for MoE-parameterized SSMs and characterize the relationship between the two designs. Empirically, we evaluate Swimba on standard benchmark tasks and measure real-time throughput and latency. Under matched FLOPs, Swimba achieves slightly better average performance than the baseline, with a small slowdown in real-time latency and throughput. Overall, these results suggest that parameter-space MoE can increase SSM capacity while keeping the dominant recurrence cost fixed.

This position paper argues that environmental cost should be part of the objectives used to train, select, and deploy AI models. Today, sustainability is mainly measured after design choices have been made, while architectures, data, hyperparameters, and serving strategies are chosen through accuracy, latency, and other proxies. This separation weakens model development. In training, newer models can move toward higher emissions without a comparable gain in performance. In inference, small differences in energy per query can become large lifecycle costs at scale. FLOPs, parameter counts, and latency do not capture these effects, because the real cost depends on hardware, datacenter efficiency, energy mix, water demand, and deployment volume. We call for multiobjective model development where performance and lifecycle environmental cost guide decisions together. The paper explains how to distinguish sustainable progress from costlier forms of progress, addresses alternative views, and outlines research directions for objectives, benchmarks, and model selection practices that make sustainability part of optimization.

Neural networks that are trained on data arising from a physical system must somehow learn regularities induced by the underlying physical laws. In this setting, concepts from statistical physics provide powerful tools for analysing how physical laws influence the learning process. In this paper we explain how *susceptibilities* --- which measure the response of a neural network to perturbations of a parameter of the data distribution --- can be applied when learning from physical data. We show that such susceptibilities can identify the conditions under which specific input features are most informative for learning, guiding data selection, a prediction we validate experimentally. For distributions lacking a natural parameter, we propose introducing one via an arbitrary scalar function of the physical state, and demonstrate that susceptibilities track the emergence of specific capabilities (such as collision detection) during training.

Self-play reinforcement learning trains language models on their own generated tasks, co-evolving a proposer and solver without human labels. Recent systems report strong reasoning gains, but collapse and instability are widely observed and poorly understood. The dominant response treats this as a reward-design problem. We argue instead that self-play stability is governed by two distinct levers: a data-level gate that decides which proposer-generated tasks enter the training pool, and the reward signal that updates the policy on tasks already admitted. Through controlled experiments on a Python output-prediction task and a deterministic-DSL twin task that strips pretraining priors, output ambiguity, and executor noise, we find the two levers are asymmetric. A strict gate is sufficient for stability under every reward variant we test, including a self-consistency reward with no access to ground truth; while no reward variant is sufficient once the gate is removed. This asymmetry exposes a counter-intuitive coupling we call the Grounded Proposer Paradox: a proposer with ground-truth access accelerates collapse faster than an ungrounded one when paired with a self-consistency solver, by concentrating training on clean tasks that form the fastest path to a spurious self-consistent attractor. Replacing the binary gate with a continuous strictness parameter $\varepsilon$ further reveals a two-stage phase transition: training-side metrics decouple at low $\varepsilon$, while validation accuracy holds until $\varepsilon$ is much higher. Data-level gating, not reward calibration, is the binding constraint on self-play stability.

Survival analysis provides a powerful statistical framework for modeling time-to-event outcomes in the presence of censoring. However, selecting an appropriate estimator from the many specialized survival approaches often requires substantial methodological and domain expertise. We introduce SurvivalPFN, a prior-data fitted network that amortizes Bayesian inference for censored observations through in-context learning. SurvivalPFN is pretrained on a diverse family of synthetic, identifiable, and right-censored data-generating processes, enabling it to amortize survival analysis in a single forward pass during inference. As a result, the model adapts to the effective complexity of each dataset without task-specific training or hyperparameter tuning, avoids restrictive parametric assumptions, and produces calibrated survival distributions. In a large-scale benchmark spanning 61 survival datasets, 21 methods, and 5 evaluation metrics, SurvivalPFN achieves strong predictive performance and often improves upon established survival models. These results suggest that SurvivalPFN offers a principled and practical foundation model for survival analysis, with potential applications in high-impact domains such as healthcare, finance, and engineering.

Atrial fibrillation (AFib) is the most common sustained cardiac arrhythmia and a major risk factor for stroke, yet it often remains undetected until a first clinical event. Existing deep learning approaches typically estimate AFib risk from a single, cross-sectional ECG, discarding the longitudinal information contained in serial recordings. We propose a transformer-based survival model, SurviFormer, that leverages sequences of ECG representations to model patient-specific AFib risk over time. Our approach encodes irregular inter-visit time gaps as pairwise relative attention biases within a causally masked transformer, and outputs a discrete hazard function trained with a multi-landmark strategy. We train and evaluate our model on the CODE dataset (140k patients, 517k ECGs), with external validation on MIMIC-IV. Compared to static baselines (DeepSurv and DeepHit applied to ResNet-50 embeddings) and a dynamic competing-risk model (DynamicDeepHit), our model achieves a C-index of 0.853, a mean TD-AUC of 0.867, and an IBS of 0.030 on the internal test set, and a C-index of 0.805, a mean TD-AUC of 0.848, and an IBS of 0.106 on the external cohort, demonstrating that longitudinal ECG trajectories provide prognostic value beyond a single recording.

Passenger-order cancellation is a critical source of uncertainty in on-demand ride-sharing systems, where requests, assignments, and vehicle routes evolve continuously in real time. Due to shared capacity and coupled routes, a single cancellation may propagate through downstream matching and routing decisions, degrading system efficiency and service reliability. However, cancellation prediction in on-demand ride-sharing remains difficult to study systematically due to the lack of large-scale longitudinal datasets, standardized evaluation protocols, and fine-grained real-time benchmarks. In this work, we introduce SurvCancel, a real-world longitudinal dataset and benchmark for dynamic passenger-order cancellation prediction in on-demand ride-sharing systems. Constructed from operational logs collected across four service regions in the real world, SurvCancel contains 173K released orders and 7.9M time-dependent snapshots. Each order is represented as a longitudinal sequence of system states, capturing both order-level attributes and the evolving supply-demand environment throughout its lifecycle. We formulate cancellation prediction as a stage-conditioned dynamic survival task: at each landmark time, models estimate short-term cancellation risk from the historical state sequence observed so far. We evaluate representative survival and dynamic-risk models under both in-distribution and distribution-shift protocols. Our results show stage-dependent model behavior in the in-distribution setting and reveal distinct robustness patterns under regional and temporal distribution shifts.