NeurIPS 2026 Papers

Structured road understanding of lane geometry, topology, and traffic element relationships is foundational to safe autonomous driving. While vision-language models (VLMs) offer promising semantic flexibility, they lack the geometric and relational grounding required for precise road reasoning. Conversely, traditional modular systems, e.g., HD maps and topological road graphs, provide structural precision but remain semantically rigid. To bridge this gap, we introduce the Combined Road Substrate (CRS), a graph-grounded framework that makes geometric road structure and open-vocabulary semantics jointly executable in a single representation. CRS enables the automatic generation of compositionally complex and linguistically varied question-answer pairs via recursive graph queries, augmented with a ``grounding for free'' mechanism that ensures logical traceability to specific map elements, and procedurally extracted chain-of-thought supervision traces. We demonstrate that state-of-the-art VLMs - including large, closed-source models - struggle significantly with structured road reasoning, yet training a small 2- or 4-billion-parameter model with as few as 20 to 80 CRS-enriched scenes yields stable gains in compositional reasoning tasks of varying depth. Analysis of model behavior via verifiable reasoning traces reveals a systematic shift in failure modes: whereas baseline models fail at relational scene understanding, CRS-trained models reduce failures to attribute recognition, suggesting that the primary bottleneck in road understanding is not model scale, but the absence of structured supervision.

Bridging the gap between simulation and reality remains a fundamental challenge for end-to-end autonomous driving. Existing approaches primarily focus on appearance-level features. This often leads to suboptimal transfer, where visually aligned models still produce inconsistent or unsafe behaviors in the real world. In this paper, we propose a unified domain adaptation framework that jointly aligns perception and decision processes through geometry-aware and vector-space representations. At the perception level, we introduce a Geometry-Aware Perception Alignment (GAPA) module that enforces cross-domain consistency in both explicit geometry and implicit structural representations extracted from latent features. This reduces domain discrepancy by targeting scene structure rather than appearance. At the decision level, we propose a Latent Vector Space Guidance Decision Alignment (LVDA) module that aligns vector-conditioned policy distributions. By modeling driving decisions as conditioned on structured representations of map topology and multi-agent interactions, we minimize discrepancies between source and target action distributions via adversarial and structural alignment. To further enhance stability, we introduce a progressive adversarial transfer strategy that improves cross-domain feature alignment and training stability. Extensive experiments on a newly partitioned nuScenes dataset demonstrate that Bridge-AD achieves excellent performance, effectively narrowing the gap between simulation and real-world autonomous driving.

Drug-target interaction (DTI) prediction is pivotal for accelerating drug discovery, yet existing methods struggle with generalization under cross-domain distribution shifts. Conventional approaches typically rely on static protein representations and assume distributional consistency between source and target domains, often resulting in poor calibration when encountering novel protein families or scaffolds. To address this issue, we present $\textbf{Bi}$-view $\textbf{Fold}$-aware prediction (BiFold), a novel unified interaction-adaptive framework that enhances generalization via dual-view representation learning and calibration-aware adaptation. BiFold synergistically models sequence and structural evidence, introducing a routing module to dynamically fuse the two views at the feature-channel level and exploit complementary information. Furthermore, we design a calibration-aware training principle that aligns domain statistics and encourages flatter source solutions. Crucially, target supervision is introduced only after a forward-only warm-up phase to prevent reinforcement of miscalibrated early predictions, effectively mitigating confirmation bias in pseudo-label learning. Extensive experiments demonstrate that BiFold consistently outperforms the state-of-the-art methods across diverse datasets in both in-domain and cross-domain settings. Additionally, interpretability analysis reveals that the routing module partitions the feature space into view-dominant and view-neutral subspaces, providing concrete evidence of adaptive multi-modal complementarity.

End-to-end autonomous driving aims to generate safe and plausible planning policies from raw sensor inputs, and constructing an effective scene representation is a critical challenge. Driving world models have shown great potential in learning rich representations by predicting the future evolution of a driving scene. However, existing driving world models primarily focus on visual scene representation, and motion representation is not explicitly designed to be planner-shared and inheritable, leaving a schism between the optimization of visual scene generation and the requirements of precise motion planning. We present WorldDrive, a holistic framework that couples scene generation and real-time planning via unifying vision and motion representation. We first introduce a Trajectory-aware Driving World Model, which conditions on a trajectory vocabulary to enforce consistency between visual dynamics and motion intentions, enabling the generation of diverse and plausible future scenes conditioned on a specific trajectory. We transfer the vision and motion encoders to a downstream Multi-modal Planner, ensuring the driving policy operates on mature representations pre-optimized by scene generation. A simple interaction between motion representation, visual representation, and ego status can generate high-quality, multi-modal trajectories. Furthermore, to exploit the world model’s foresight, we propose a Future-aware Rewarder, which distills future latent representation from the frozen world model to evaluate and select optimal trajectories in real-time. Extensive experiments demonstrate that WorldDrive achieves leading planning performance among vision-only methods while maintaining high-fidelity action-controlled video generation capabilities. Our code and model will be made publicly available.

Reinforcement learning (RL) is essential for adaptive decision-making, e.g., robust Meta-RL and domain randomization (DR), and efficient post-training of large language models (LLMs), where tasks are commonly structured as Markov decision processes or prompts. A central bottleneck in all these settings is the cost of agent-environment interactions required for task selection and policy optimization. Model predictive task sampling (MPTS) has emerged as a promising approach to improve sample efficiency by querying informative tasks via a risk predictive model (RPM) trained on optimization history. However, existing RPMs suffer from two failure modes: (i) sparse optimization histories, which limit RPMs' generalization across the task space, and (ii) a one-step lag in risk approximation, which causes biased difficulty estimation as the policy evolves. This work formalizes these failure modes, derives lagged-risk decompositions that expose the resulting temporal bias, and recasts risk prediction as an in-context sequence modeling problem. The developed RPM integrates a one-step look-ahead mechanism with a temporal-weighted sliding window to mitigate data non-stationarity and scarcity, without requiring additional rollouts. Empirically, our approach yields more reliable difficulty estimates and consistent performance gains across robust Meta-RL, DR, and prompt curriculum RL post-training of LLMs.

Understanding ultra-long videos such as egocentric recordings, live streams, or surveillance footage spanning days to weeks, remains a challenge. For current multimodal LLMs: even with million-token context windows, frame budgets cover only tens of minutes of densely sampled video, and most evidence is discarded before inference begins. Memory-augmented and agentic approaches help with scale, but their retrieval remains fragmented across modalities and lacks long-range narrative summaries that span days or weeks. We propose MAGIC-Video, a training-free framework built around a multimodal memory graph with interleaved narrative chain: the graph unifies episodic, semantic, and visual content through six typed edges and supports cross-modal retrieval, while the chain distils long-horizon entity biographies and recurring activity events. At inference time, an agentic loop interleaves graph retrieval with narrative fact injection, covering both the modality and time dimensions of ultra-long video in a single retrieval pipeline. On EgoLifeQA, Ego-R1 and MM-Lifelong, MAGIC-Video consistently outperforms strong general-purpose, long-video, and agentic baselines, with gains of 10.1, 7.4, and 5.9 points over the prior best agentic system on each benchmark.

Robust visual recognition under degraded imaging conditions is critical for real-world vision systems.Task-oriented image restoration addresses this problem by enhancing degraded inputs before recognition, but it must preserve natural visual appearance without disrupting the evidence used by downstream recognizers.Existing optimization strategies struggle to satisfy these requirements simultaneously.Restoration driven mainly by visual quality may weaken task-critical cues or introduce texture shifts that are visually plausible but poorly matched to the recognizer.Directly tuning restoration networks with recognition loss is also insufficient, as the restoration model may exploit recognizer-specific shortcut cues and produce visually unnatural artifacts while improving task scores.We propose Causal Mediated Unrolling (CaMeU), an optimization-based framework for task-oriented image restoration with frozen recognition models.Starting from a joint optimization objective, the proposed framework derives a Half-Quadratic Splitting formulation that separates restoration into alternating task-driven and prior-guided updates.The task-driven update introduces an explicit mediator for front-door guided optimization, reducing shortcut bias induced by the recognizer.The prior-guided update performs manifold projection, pulling the intermediate result back toward the natural-image manifold to preserve visual quality.Through this mediated alternating process, the restored image is progressively optimized for both recognition performance and visual quality.Experiments on VOC Haze, VOC Dark, and degraded CUB-200-2011 show that CaMeU consistently improves downstream recognition while reducing visual artifacts.

Graph-structured data is crucial in various domains like biology and social networks. Comparing graphs, which is a fundamental problem in graph data analysis, is nonetheless highly challenging. Recently, the Gromov-Wasserstein (GW) distance has provided a principled way to compare two graphs. However, computing the GW distance involves solving a complex non-convex optimization problem, making it computationally expensive, especially when the graphs are large. In this work, we propose a neural approximation of the GW distance, called NeuralGW. In NeuralGW, we use a combination of a graph isomorphism network and a transformer to represent the nodes of two graphs as two sets of vectors, treated as two discrete distributions, on which we compute multiple maximum mean discrepancy values given by different kernels. We then use a multilayer perceptron to convert the vector formed by these values into a single value, which is the prediction of the GW distance. Once trained, the model allows for efficient inference, enabling fast structural comparisons between graphs across diverse domains. We also provide a theoretical guarantee for the generalization ability of NeuralGW. Experiments demonstrate the effectiveness and practical applicability of our approach on real-world datasets, in comparison to baselines.

Synthetic time series generation is critical for data augmentation, privacy-preserving sharing, and simulation. Autoregressive models extend to arbitrary horizons but suffer compounding errors, while diffusion models achieve high fidelity through bidirectional refinement only at fixed lengths. We propose **Horizon Diffusion**, a unified framework that resolves this tension by decomposing generation into variable-length *horizon blocks* produced autoregressively across blocks yet jointly denoised within each block. Two innovations make this hybrid practical: *Horizon-Causal Attention*, a structured mask that simultaneously preserves cross-block causality and within-block bidirectional refinement, and the *Block Size Curriculum*, a deterministic Warmup--Stable--Decay schedule that fluidly transitions training from full-sequence diffusion to block-wise autoregressive generation. Our method consistently outperforms strong baselines on fidelity, diversity, and downstream utility, and remains stable when sequence length scales by $4\times$ beyond training from a single trained model. Code is available at Supplementary Material.

Classical compute-optimal scaling laws assume an unbounded supply of fresh pretraining data, yet pretraining is increasingly entering the regime where compute is growing faster than high-quality data. We propose *Compute-Data (CD) scaling laws*, a unified framework that bridges *compute-optimal* scaling, in which data scales freely with compute, and *data-optimal* scaling, in which the corpus is fixed and compute can grow unbounded. CD scaling extends classic scaling by introducing a *token effectiveness function* that quantifies how much a *derived* token, produced for instance by multi-epoch repetition or paraphrasing, is worth relative to a fresh one, ranging from a perfect substitute to no value at all. Fitting $\eta$ for two data expansion strategies (multi-epoch repetition, paraphrasing) across model sizes ranging from 14M to 600M parameters on the Dolma-3 corpus, we find that it is far from constant: it depends jointly on model size, the tokens-per-parameter ratio, and the amount of derived data, and saturates as the corpus is expanded. The functional form of $\eta$ implies that substituting compute for data *diminishes* with both model size and data availability, and partitions training into three operational regimes: compute-bound, data-bound, and model-bound, showing that classic compute-optimal allocation is suboptimal across most of the practically relevant regime.

Vision-language models such as CLIP enable zero-shot classification from arbitrary class names, but under test-time corruptions, their visual encoder projects distorted features onto incorrect textual anchors with high confidence—a phenomenon we term \textbf{modality bias}. Standard entropy minimization reinforces these overconfident errors, triggering a destructive confirmation-bias loop that collapses the predicted label distribution. We propose \textbf{CDIM} (Cross-Modal Information Maximization), an online test-time adaptation method that breaks this loop at three levels. At the \textit{representation level}, a frozen self-supervised DINO backbone is combined with CLIP via a cross-covariance logit ensemble; by projecting text prototypes into DINO's geometric space—rather than the reverse—the classifier operates free of text bias. At the \textit{optimization level}, a gated information-maximization objective (GT-IM) pairs per-sample sharpening with batch-level diversity, while an adaptive gate excludes unreliable samples from the gradient. At the \textit{prediction level}, an online logit adjustment corrects the residual class-distribution skew. Ultimately, \textbf{CDIM} validates its broad effectiveness by delivering robust, competitive performance across demanding setups, including continual TTA, mixed-domain shifts, and diverse domain generalization benchmarks.

Attributed Graph Clustering (AGC) is a fundamental unsupervised task that integrates structural topology and node attributes to uncover latent patterns in graph-structured data. Despite its significance in industrial applications such as fraud detection and user segmentation, a significant chasm persists between academic research and real-world deployment. Current evaluation protocols suffer from the small-scale, high-homophily citation datasets, non-scalable full-batch training paradigms, and a reliance on supervised metrics that fail to reflect performance in label-scarce environments. To bridge these gaps, we present `PyAGC`, a comprehensive, production-ready benchmark and library designed to stress-test AGC methods across diverse scales and structural properties. We unify existing methodologies into a modular Encode-Cluster-Optimize framework and, for the first time, provide memory-efficient, mini-batch implementations for a wide array of state-of-the-art AGC algorithms. Our benchmark curates 12 diverse datasets, ranging from $2.7 \times 10^3$ to $1.1 \times 10^8$ nodes, specifically incorporating industrial graphs with complex tabular features and low homophily. Furthermore, we advocate for a holistic evaluation protocol that mandates unsupervised structural metrics and efficiency profiling alongside traditional supervised metrics. Battle-tested in high-stakes industrial workflows at Ant Group, this benchmark offers the community a robust, reproducible, and scalable platform to advance AGC research towards realistic deployment. The code and resources are publicly available via [GitHub](https://github.com/Cloudy1225/PyAGC), [PyPI](https://pypi.org/project/pyagc), and [Docs](https://pyagc.readthedocs.io). We invite readers to our [latest survey](https://arxiv.org/abs/2603.20829) for the unified framework and industrial perspective that complement this benchmark.

The world appears spatially three-dimensional, however, existing large-scale multimodal models are often built mostly around 1D, 2D and 2.5D modalities. To move toward a more complete and actionable understanding of physical reality, models that natively model and generate across 1D, 2D, and 3D modalities are desirable. We present OmniDiMM, an any-to-any foundation model trained on a diverse set of 3D including meshes, Gaussian splats, voxels, and NeRFs; 2D such as images, and 1D like text. We develop tokenizers for a diverse set of 3D modalities, such as NeRF weights triplanes, UDFs, and 3DGSs, that convert them into discrete tokens, which enables multimodal masked modeling for joint training. Out of the box, OmniDiMM can directly perform standard tasks such as novel-view image synthesis and 3D generation from images, where it matches or outperforms existing specialized methods. As an any-to-any model, it can also convert between different 3D representations, such as NeRF weights and UDF, and generate various 3D modalities from 2D inputs, including DINOv2 features and surface normals. Joint training across 3D modalities also leads to strong transfer performance on downstream tasks such as grasping, classification, and object pose prediction. We scale training to a 1B-parameter model using 1T tokens across 1M objects. The pretrained models, training code, and multimodal dataset will be open-sourced.

Generative models are often required to serve many downstream conditional tasks. In the training-free setting, a pretrained unconditional model is reused as a prior and combined with a task-specific likelihood to sample from the corresponding posterior, without retraining the generative model. While such conditional samplers have been developed for diffusion and flow generative models, training-free conditional sampling for Schrödinger bridge (SB) generative models remains unexplored, despite the growing appeal of SBs as flexible stochastic transport models between general source and target distributions. We propose BridgeTwist, a training-free conditional sampler for pretrained SB generative models, formulated as a tempered twisted sequential Monte Carlo scheme. At each step, the sampler uses a closed-form plug-in twist surrogate, which combines the velocity and score fields exposed by SB pretraining into an explicit endpoint predictor. Since this surrogate is less reliable near the source, we use a time-varying temperature to temper the twist, downweighting its influence at early steps. Telescoping importance weights then cancel all intermediate twist factors, yielding an asymptotically exact conditional sampler without any auxiliary twist network or pilot rollouts. Empirically, BridgeTwist consistently outperforms training-free baselines on class-conditional sampling and inpainting on MNIST and CIFAR-10, and on text-to-image generation on CelebA-HQ.

Multi-View Stereo (MVS) and Monocular Depth Estimation (MDE) provide complementary cues for dense 3D reconstruction. MDE offers rich contextual and structural priors but lacks explicit geometric constraints, whereas MVS exploits multi-view geometry but often struggles in ambiguous regions, such as weakly textured or reflective surfaces. Existing MDE-assisted MVS methods mainly use monocular cues in a unidirectional manner, leaving the interaction between monocular representations and multi-view cost volumes insufficiently explored. To address this limitation, we introduce BridgeMVS, a unified cascaded MVS framework that bridges monocular depth features and multi-view cost-volume representations through Bidirectional Dynamic Fusion (BDF). At each cascade stage, BDF performs dynamic mono-to-volume and volume-to-mono updates: monocular structural features are injected into cost-volume regularization, while multi-view geometric evidence recalibrates intermediate monocular representations. The enhanced monocular features are propagated across stages together with MVS representations and supervised by stage-wise monocular losses, establishing an auxiliary structural guidance path for the MVS branch. Unlike stage-isolated designs, BridgeMVS preserves fused representations from coarse to fine stages, allowing low-resolution cross-branch interactions to guide subsequent high-resolution depth estimation. Moreover, BDF is designed as a plug-and-play module and can be conveniently integrated into existing cascaded MVS frameworks. Extensive experiments show that BridgeMVS achieves state-of-the-art performance on the DTU, Tanks and Temples, and ETH3D benchmarks.

Continuous-time generative models compress endpoint-conditioned bridges into Markov velocity fields, but no existing diagnostic measures how much information this compression discards. We introduce the \emph{Markovization gap}---the integrated conditional variance of the bridge velocity given the Markov state---which quantifies this loss before any neural network is trained. To make the gap well-defined across model families, we define \emph{Bridge Graphical Models} (BGMs), separating endpoint coupling, bridge law, Markovian projection, and dynamics representation as independent design choices. This decomposition also formalizes Poisson and electrostatic models as field-line bridge kernels with their own caustic gap. Across synthetic, latent, and pixel-space pilots on CIFAR-10 and Fashion-MNIST, a proxy gap estimated in minutes on CPU consistently ranks coupling/bridge choices in the same direction as training loss and FID measured after hours of GPU training.

Flicker-banding (FB), arises from temporal aliasing between a camera's rolling shutter and a display's brightness modulation, degrading screen-captured image readability with color shifts and jagged patterns. Existing single-frame methods with simplified parametric stripe models cannot reliably distinguish these artifacts from genuine texture. To address this, we conduct a systematic analysis of complex FB morphologies and reveal their significant variation across exposure settings, motivating a multi-frame bracketed RAW restoration paradigm. We construct Bricker, a synthetic–real bracketed RAW dataset built via ray-tracing-based physical simulation and automated multi-exposure capture tool. We further propose BRACE: Bracketed RAW Flicker-Banding Removal, a multi-frame restoration model that utilizes frequency-aware banding prior and a multi-scale spatial cross-attention modulator (MSCAM) for cross-exposure spatial fusion. We also introduce the Stripe Frequency Consistency (SFC) metric to evaluate banding removal. Extensive experiments demonstrate state-of-the-art performance on both synthetic and real benchmarks. Our dataset and code will be publicly released.

Generating physically buildable brick structures from 3D shapes requires more than geometric reconstruction: the output must also satisfy discrete part constraints and structural stability. Existing brick generation methods either rely on heuristic optimization, which can break down when the target 3D shape does not admit a feasible structure under predefined constraints, or generate brick sequences without explicitly modeling the underlying 3D geometry and assembly relations. In this work, we present BrickAnything, a geometry-conditioned autoregressive framework for generating buildable brick structures from diverse 3D representations. BrickAnything uses point clouds as a unified geometric interface and predicts brick sequences that reconstruct the target shape under assembly constraints. To model structural dependencies among bricks, we introduce a structure-aware tree tokenization, which represents brick structures through local attachment relations. This formulation makes sequence generation more consistent with the physical construction process, and reduces invalid intermediate states. We further introduce preference-based alignment post-training, validity-constrained decoding and adaptive rollback to improve buildability objectives such as stability and geometric fidelity. Extensive experiments demonstrate that BrickAnything produces geometrically faithful and physically realizable brick structures, and that the proposed tokenization effectively reduces rollback and regeneration compared with conventional ordering strategies.

Distributionally robust optimization (DRO) provides a principled framework for learning under distribution shift, but its practical use is hindered by the difficulty of evaluating worst-case risks for nonconvex loss functions. We study a penalized DRO formulation in which the adversary may choose any distribution but incurs a Wasserstein penalty for deviating from the empirical distribution. We show that the adversary’s problem can be reformulated as an optimization problem over transport maps that push empirical samples to adversarial ones, and we prove that optimal maps are cyclically monotone. We also show that standard adversarial training---based on per-sample local optimization---violates cyclical monotonicity and wastes transport costs unless the adversary is severely restricted. We propose two remedies. First, we introduce multi-start particle ascent, which alternates parallel gradient ascent with reassignment to enforce cyclical monotonicity across samples. Second, we parameterize adversarial maps as gradients of input-convex neural networks, which guarantees cyclical monotonicity by construction. Experiments on robust regression, image classification, and robust control show that our methods consistently outperform standard adversarial training and state-of-the-art baselines, achieving improved robustness and better generalization under distribution shift.