NeurIPS 2026 论文列表

Personalized portrait generation aims to synthesize portraits that align with both the identity of the reference face and the semantics of the text. However, existing methods suffer from two critical challenges. First, they exhibit a trade-off between ID fidelity and text controllability. We identify that this limitation stems from the self-supervised training, where the model learns a trivial mapping rather than understand identity. Second, despite exploring various encoding strategies, recent approaches still tend to discard fine-grained facial details. To address these challenges, we propose CrossID-11M, a large-scale curated portrait dataset comprising 11 million images with high-quality annotations, paired with millions of identity-preserving face video clips. Furthermore, we introduce CrossID, a novel framework integrating a lightweight GatedFaceAdapter and a cross-supervised training paradigm. Additionally, we establish CrossID-Bench alongside new VLM-based evaluation metrics. Extensive experiments demonstrate that CrossID successfully breaks the trade-off between ID fidelity and controllability, achieving state-of-the-art performance by producing photorealistic images with faithfully preserved ID details.

Large Vision-Language Models (LVLMs) have achieved remarkable multimodal reasoning capabilities but remain critically vulnerable to adversarial perturbations. In practical black-box scenarios, severe architectural discrepancies, misaligned feature spaces, and divergent multimodal fusion strategies often undermine adversarial transferability, preventing perturbations crafted on surrogate models from generalizing to unseen target models. Existing ensemble-based LVLM attacks attempt to mitigate this by integrating multiple surrogates, yet they often fail to bridge the structural generalization gap across heterogeneous LVLM families. In this work, we propose a novel framework that enhances cross-model adversarial transferability via model diversity enrichment and stochastic parameter sampling. We show that transfer success is fundamentally governed by cross-model gradient variance. To mitigate this, we introduce (1) a diversity-induced continuous surrogate manifold that captures cross-model decision variability, and (2) stochastic parameter sampling during optimization to approximate the gradient distribution of unseen LVLM models. We provide a theoretical analysis bounding the expected adversarial transfer gain by gradient variance and prove that stochastic sampling reduces adversarial overfitting to surrogate decision boundaries. Extensive experiments across multiple state-of-the-art LVLMs demonstrate substantial improvements in black-box transferability under strict evaluation protocols.

Visual Foundation Models (VFMs) have demonstrated strong capabilities in visual-geometric tasks such as image matching and 3D reconstruction. Recent work has revealed the potential of VFMs in label-free indoor RGB-D registration, where the visual modality is intrinsic to the task. A natural question follows: can this success extend to more challenging outdoor scenarios characterized by sparse, irregular LiDAR data with only partial visual coverage? We answer this with PROMOTER, a novel teacher–student framework for unsupervised LiDAR registration that bridges visual priors and sparse LiDAR geometry. To leverage pre-trained geometric priors for 3D registration, we must disentangle them from confounding signals introduced by modality and task gaps under partial visual coverage. To this end, we introduce a lightweight Adaptive Prior Integrator that extracts and integrates these priors into an enhanced teacher, together with a Perceptual-Geometric Labeler that produces high-quality pseudo-labels. For student training, we propose Matchability-Anchored Training to improve robustness under noisy supervision. Experiments on KITTI and nuScenes demonstrate state-of-the-art performance and improved scalability across registration models and VFMs, while incurring no additional parameters at inference. Code will be released.

Although Vision-Language Models (VLMs) demonstrate impressive capabilities, their deployment in high-stakes domains is severely hindered by hallucinations. Existing detection methods mostly focus on static internal feature probing with coarse-grained annotations, failing to achieve fine-grained token-level localization without expensive dense labels. In this paper, we reconceptualize VLM hallucination detection as a dynamic graph learning task, identifying hallucinations as distinct *trajectory drifts* across network layers. Consequently, we propose **TRACE** (Token Representation Analysis with Cross-layer Evolution), a non-intrusive probing framework that models internal states as heterogeneous spatio-temporal graphs. By employing Graph Neural Networks (GNNs) to capture computational dynamics, TRACE effectively isolates hallucinatory signals. Furthermore, a token-first Multiple Instance Learning (MIL) readout elegantly bridges the granularity gap, distilling macroscopic sentence-level supervision into precise zero-shot token-level localization. Extensive evaluations on M-HalDetect, ViGoR, and HalLoc demonstrate that TRACE achieves state-of-the-art detection performance while being competitive in localization accuracy, offering an efficient and interpretable solution for trustworthy VLMs.

Cross-distribution generalization in longitudinal behavioral data remains a persistent obstacle. Models trained on one cohort degrade on new populations, and prior approaches operating through feature-distribution alignment or model-capacity scaling have not substantially improved cross-distribution performance. We diagnose the limitation as insufficient representational constraints during training. We propose holding the language model frozen, so that the coherence criterion is defined by its pre-trained language prior and cannot shift during training. We introduce PRISM, a frozen-backbone framework that decomposes each behavioral trajectory into temporal, spectral, and semantic streams and integrates them through directed cross-attention with instance-specific gating. Training uses a dual-path objective combining discriminative and coherence constraints on a shared representation. On GLOBEM, a widely used cross-cohort benchmark for behavioral health prediction, PRISM achieves 79.93\% out-of-distribution accuracy, 27.13 points above the previous best, with an in-distribution to out-of-distribution accuracy gap of 1.24 points. PRISM also achieves the highest OOD accuracy on three additional datasets (LifeSnaps anxiety, MFAFY engagement, CrossCheck schizophrenia symptoms), with consistently smaller ID-to-OOD gaps than fine-tuned VLM baselines. Ablations identify the frozen coherence constraint as the component responsible for the distribution-invariance pattern.

Multimodal geometry reasoning requires models to jointly perceive diagrams and perform symbolic derivations. Test-time verification methods such as majority voting assume voter errors are roughly independent, but in geometry this assumption breaks down: all candidates perceive the same diagram, so perceptual biases cascade into correlated errors that produce confident but wrong consensus. We observe that neuro-symbolic candidates contain compositional structure that can serve as a verification resource. We introduce SKETCH, a format decomposing each candidate into a neural pathway (natural-language claim) and a symbolic pathway (visual grounding → typed DSL → deterministic execution). These pathways are governed by different error processes—symbolic errors are correlated across candidates through shared perception, while neural errors are more dispersed. Our method, Compositional Consensus Verification (CCV), gates consensus on cross-channel agreement: because the two pathways fail for different reasons, accidental agreement is rare, making each concordant vote far more precise than a raw majority vote. Across four geometry benchmarks, CCV achieves 81.0% accuracy (+10.4 pp over majority voting) without external scorers or additional inference. Seven non-compositional baselines—including trained PRMs—cluster at 70–71%, while compositional cross-layer features jump to 79–81%, confirming that structural diversity is the core verification resource, and that a principled gating algorithm effectively unlocks its potential.

Foundation models are central to building general-purpose embodied agents. Minecraft has become the dominant testbed for this paradigm, but its computational and engineering overhead limits rapid iteration and controlled analysis. Crafter offers a lightweight alternative yet lacks the foundation model ecosystem required for such research. We introduce CrafterDojo, a suite of foundation models, datasets, and data-generation toolkits that establishes Crafter as a testbed for foundation-model-based embodied intelligence, providing CrafterVPT, CrafterCLIP, and CrafterSteve-1. To address Crafter's data scarcity, CrafterDojo includes automatic tools that generate diverse synthetic demonstrations and aligned video-language pairs, yielding the CrafterPlay and CrafterCaption datasets. Experiments show that these datasets support effective foundation model training and enable hierarchical agents that outperform non-hierarchical baselines on long-horizon tasks. We release all code, models, and datasets.

Automated structural health monitoring is essential to prevent catastrophic infrastructure failures. Precise, pixel-level defect segmentation is needed to accurately assess structural integrity, but progress in defect segmentation for civil infrastructures has been held back by an extreme scarcity of data, which requires costly expert annotation. The need for data is accentuated by algorithmic hurdles intrinsic to the problem, including center-bias and the need to rely more on shape when inspecting nearly textureless building materials. To remove the bottleneck, we introduce **Cracks in the Foundation (CiF)**, the largest and most detailed civil infrastructure (instance) segmentation dataset to date, comprising $\approx$150,000 high-resolution images meticulously curated over five years in collaboration with civil engineering experts. With the help of this unprecedented data source, we expose a blind spot of current visual AI: despite the advent of promptable Foundation Models (FMs) and Vision Language Models (VLMs), and despite the impressive abilities of today's specialised segmentation models, it turns out that dense image understanding in the built environment is nowhere near solved. Our evaluations indicate that even the most recent zero-shot FMs face significant challenges when deployed on real-world infrastructure and even the performance of specialised models with domain-specific supervision plateaus at $\approx$25\% mAP. CiF establishes inspection of civil infrastructure, an elementary and seemingly easy perceptual task, as an open challenge that reveals fundamental weaknesses of present-day models trained predominantly on internet images, literally and figuratively highlighting *cracks in the current foundation model paradigm*.

Post-training quantization (PTQ) compresses large language models to low bit-widths using a small calibration set, making calibration selection a consequential but under-specified part of the quantization pipeline. We study calibration selection for weight-only LLM PTQ backends such as AWQ and GPTQ. In this setting, calibration data does not define an inference-time activation clipping threshold; instead, it determines activation-conditioned quantities used for weight reconstruction, saliency estimation, and scale search. We identify a failure mode in which a calibration set misses rare, high-magnitude input channels of quantized linear modules, causing the backend to underrepresent reconstruction-sensitive directions. Motivated by this observation, we formulate calibration selection as weighted maximum coverage over module-level activation outlier channels. Each candidate sequence covers the module/channel pairs it activates above a reference threshold, and each covered pair is weighted by a dimensionally consistent surrogate for its potential contribution to squared output reconstruction error. The resulting objective is monotone submodular, so greedy selection admits the classical $(1-1/e)$ approximation guarantee for the coverage objective. We instantiate this formulation in COVERCAL, a backend-agnostic calibration selector operating on cached activation masks. Across LLaMA-2, LLaMA-3, and Mistral models, under AWQ and GPTQ INT4 backends and five downstream evaluations, COVERCAL improves over the reported fixed-pool calibration baselines, with the largest gains at small calibration budgets. Under AWQ at $K=128$, COVERCAL improves MMLU by $1.2$--$1.5$ points over random calibration; at $K=64$, it matches or exceeds random calibration at $K=256$ on LLaMA-3-8B. The contribution is not a new quantizer, but a module-correct coverage formulation, an efficient greedy selector, and empirical evidence that weighted outlier coverage is a useful calibration signal for existing weight-only PTQ backends.

Crowd counting is a vital computer vision task with wide-ranging applications, from public surveillance to urban planning. A persistent challenge is achieving scale invariance despite noisy labels, as humans often make counting errors with similar objects. Most existing methods struggle to maintain consistency and robustness when testing images differ in scale from the training data. We propose ScaLe-aware Invariance and Correcting Errors (SLICE) for robust crowd counting, a novel scale-aware loss function that improves scale invariance, enabling consistent and robust predictions under varying scales and annotation noise. The loss function combines two components: a scale-invariance term that applies multiscale supervision to enforce consistency across density maps of the same scene, and an error correction term that addresses inevitable annotation noise by modeling label uncertainty with probabilistic methods such as a mixture of Gaussians. SLICE is model-agnostic and integrates easily into existing crowd counting architectures without extra computational overhead during inference. Extensive experiments on five benchmark datasets show that SLICE greatly improves scale invariance and cross-dataset generalization, achieving the state-of-the-art results in challenging scenarios. Using CrowdDiff, Steerer, and CrowdFormer as baselines, SLICE lowers mean absolute error (MAE) by 5–7\% by addressing annotation noise and scale variation. Applying the scale-invariance term across multiple scales further cuts MAE by up to 15\%.

Vision-Language-Action (VLA) models inherit rich semantic priors from pretrained vision-language models, yet these priors do not guarantee persistent instruction sensitivity after narrow-domain fine-tuning. We observe a systematic failure mode: although VLA models remain visually reactive, their later actions can become weakly constrained by the commanded instruction, leading to unstable object interaction and visually plausible but instruction-inconsistent behavior. We trace this behavior to a collapse of instruction-conditioned representations across network depth and task progress. More fundamentally, this collapse is enabled by the learning objective itself: standard imitation learning admits Bayes-optimal solutions that are invariant to language. We address this by recasting instruction grounding as identifying the commanded instruction among counterfactual alternatives. Based on this view, we propose Counterfactual Instruction Grounding (CIG), a contrastive objective that encourages the generated trajectory to remain identifiable with the commanded instruction among counterfactual alternatives. CIG applies to both autoregressive and flow-matching VLA models, using exact action-chunk likelihoods for autoregressive models and an energy-based pseudo posterior for flow-matching models to avoid intractable trajectory likelihood estimation. Practically, CIG can be applied directly to already fine-tuned models as a continuation-stage grounding objective, restoring instruction sensitivity without retraining from scratch or changing the architecture. Extensive experiments on RoboCasa and LIBERO-Plus demonstrate that CIG achieves strong instruction-following performance with success rates of 71.3% and 86.3%, respectively. Real-world deployment further shows that CIG transfers to kitchen manipulation and preserves instruction-conditioned behavior in visually ambiguous scenes.

Correspondence pruning—separating sparse inliers from severe outliers in noisy feature matches—is a long-standing bottleneck for two-view geometry estimation in 3D vision. Existing learning-based pruners pursue this goal through ever more sophisticated context operators—local graphs, dynamic neighborhoods, or global attention—yet a controlled comparison reveals a striking phenomenon: when we freeze the structural assignment of these operators after a single forward pass, accuracy collapses by over $30\%$ mAP regardless of operator choice. This suggests that the limiting factor of current methods is not which operator is used, but the implicit assumption that structure can be committed in one shot. We argue that correspondence pruning should instead be cast as \emph{iterative structural rectification} (ISR), in which the geometric structure and the feature representation must \emph{co-evolve} layer by layer. Building on this view, we propose \textbf{ISR-Net}, a minimal architecture that pairs a layer-wise re-derived local-context operator (a dynamic $k$-NN graph in feature space) with a layer-wise re-derived global-context operator (soft assignment to learnable cluster prototypes), so that any gain over prior arts is attributable to the rectification \emph{schedule} rather than to operator novelty. Despite using deliberately off-the-shelf operators, ISR-Net attains state-of-the-art results on YFCC100M, SUN3D, and HPatches, surpassing the strongest attention-based competitor by $+8.2\%$ mAP@$5^\circ$ while running $35\%$ faster, indicating that \emph{how often structure is refined} matters more than the operator's complexity.

Existing methods for Virtual Try-On (VTON) often struggle to accurately transfer garment appearance, especially in unpaired settings where accurate person-to-garment correspondence is required. These methods do not explicitly supervise person-to-garment alignment, leaving correspondence to be learned implicitly within the generation model. In this paper, we first analyze full self-attention in DiT-based architectures and show that person-to-garment query-key matching aligned with local geometric correspondence is closely related to try-on quality. Building on this insight, we introduce CORrespondence ALignment (CORAL), a DiT-based framework that explicitly aligns query-key matching with robust external correspondences. CORAL integrates two complementary components: a correspondence distillation loss that aligns reliable matches with person-to-garment attention, and an entropy minimization loss that sharpens the attention distribution. For evaluation, we further propose a VLM-based evaluation protocol to better reflect human preference. CORAL consistently improves over the baseline, enhancing both global shape transfer and local detail preservation. Extensive ablations validate our design choices. Code and weights will be publicly available.

Coresets are a fundamental tool for scaling clustering algorithms, but standard constructions assume full access to the pairwise distances. In many modern settings, access to distances is costly and thus limited, and algorithms must instead rely on inexpensive comparison queries. We study coreset construction in the Rank-Measure (RM) model, where the algorithm can make noisy comparison queries to a quadruplet oracle alongside a small number of distance queries. In this model, we present the first algorithm to construct an $\epsilon$-coreset for $(k,z)$-clustering in Euclidean space. It matches the coreset-size bounds known in the classical model, while using only polylogarithmically many distance queries in the input size. Our approach is based on a uniform-sampling framework [Chen, SICOMP 2009], which decomposes the input into geometric rings and samples uniformly within each ring. We implement this approach using primarily noisy comparison queries, and then compress the result using a standard full-access coreset construction. In contrast, prior work in oracle-based clustering achieved only constant-factor guarantees. We complement our theoretical findings with experiments that demonstrate strong empirical performance against full-access baselines while using significantly fewer distance computations.

Control variables are widely used in statistical modelling to account for omitted variable bias. However, they have largely been underexplored in deep learning. This is surprising, given that deep learning models encode image-inferable covariates, such as demographic variables, into their predictions when these covariates are correlated with the outcome---a form of omitted variable bias referred to as 'shortcut learning'. While many existing confound-control or fairness methods try to restrict the correlation of such covariates with model predictions, we show that this fails to correct for omitted variable bias. We therefore propose a control variable approach for deep learning models, based on generalised additive modelling of the effects of model inputs and covariates. As flexible additive models can suffer from concurvity, we introduce an estimation procedure that refits the final layer of a pre-trained network to include covariate effects, using cross-fitting with ridge penalisation. We show how these effects can be orthogonalised with respect to covariates to exclude their mediated effects and that model predictions can be marginalised over the covariate distribution to control for their effect. This yields unbiased, interpretable predictions and offers flexibility to model the desired effects depending on the scientific or fairness objective. We verify our approach using simulated images, where it recovers the true covariate effects, even in small samples. Existing methods either require more data or fail to recover the true effects. We apply our method to real neuroimaging data with experimentally induced confounding, where it recovers prediction performance to near the level of a model trained on unconfounded data.

Traditional recommendation systems rely on latent (dense) representations, making them difficult to interpret and control. We propose the Controllable and Content-Based Recommendations (CCBR) framework, which builds its recommendations from textual user profile representations. CCBR plugs into collaborative filtering models and introduces controllability via text bottlenecks. We show that CCBR enables text-based and multimodal interventions, allowing users to steer the model towards the directions they prefer. Different from existing controllable recommendation systems, CCBR infers the text summaries directly from item contents (images, audio or video). Across image-, audio-, and video-based datasets, we demonstrate that the proposed framework obtains competitive model performance with standard (latent-representation) models while providing controllable model summaries via text. The model also outperforms TEARS, a recent baseline for controllable recommendation systems. Through systematic interventions, we demonstrate the efficacy of the user steering mechanism.

The rapid growth of video-based social media has increased users’ exposure to harmful content, creating a need for reliable automated video safety detection. Although recent Vision-Language Models (VLMs) show strong video understanding capabilities, existing harmful video detection systems face two key limitations: they typically reduce safety detection to binary classification, overlooking the inherently multi-label nature of unsafe videos, and they rely on static training objectives that do not support controllable precision-recall trade-offs, though the desired operating point may vary across moderation pipelines and unsafe categories. To address these gaps, we propose Adaptive Tversky Policy Optimization (ATPO), a reinforcement learning framework for Multi-label Video Safety Detection (Multi-VSD). ATPO introduces the Adaptive Tversky Reward (ATR), which dynamically adjusts false-positive and false-negative penalties during training to enable controllable precision–recall trade-offs. Experiments on SafeWatch-Bench and XD-Violence show that ATPO substantially improves multi-label performance, increasing the Jaccard Index from 40.66 to 75.44 on SafeWatch-Bench-Real. Moreover, ATR enables reliable steering of the precision–recall operating point, supporting deployment scenarios with heterogeneous policy requirements.

Model distillation provides an alternative when computing resources are limited. How to transfer the reasoning ability from teacher model to student model influences the performance of model distillation. We develop a mathematical framework for reasoning distillation grounded in dynamical systems theory. By modeling the teacher model's residual stream as a discrete dynamical system, we define a precise notion of *orbital stability*: perturbations transversal to the teacher's reasoning trajectory should contract, while the tangential reasoning signal propagates without active suppression. We derive the **Contractive Restoring Flow (CRF)** loss from first principles and prove that, at the loss minimum, it achieves strict transversal contraction at a constant rate $1-\alpha$, leaves the tangential dynamics unconstrained (tangential agnosticism), and produces a global basin of attraction with bounded steady-state error within the linearisation regime. Empirically, reasoning-trajectory distillation with the CRF loss outperforms other distillation methods, and shows greater robustness on long-chain reasoning tasks, supporting the predicted effect of contracting transversal error. The code is avaiable at: https://anonymous.4open.science/r/CRF-5945/.

Real-world image degradation is often unknown, spatially non-uniform, and compositional, requiring all-in-one restoration models to adapt a single set of weights to diverse local corruption patterns without test-time degradation labels. Existing methods typically modulate a shared backbone with global prompts or degradation descriptors, or route features through predefined expert pools. However, compact global conditioning can bottleneck localized degradation evidence, while static expert routing may produce homogeneous updates or rely on unstable sparse assignments. We propose \textbf{Continuous Expert Assembly} (CEA), a token-wise dynamic parameterization framework for all-in-one image restoration. CEA employs a lightweight \textbf{Cross-Attention Hyper-Adapter} to probe intermediate spatial features and synthesize instance-conditioned low-rank routing bases and residual directions. Each spatial token then assembles its own residual update via dense signed dot-product affinities over the generated rank-wise components, avoiding external prompts, static expert banks, and discrete Top-$K$ selection. The resulting assembly rule also admits a linear-attention perspective, making its dense token-wise routing behavior transparent. Experiments on AIO-3, AIO-5, and CDD-11 show that CEA improves average restoration quality over strong prompt-, descriptor-, and expert-based baselines, with the clearest gains on spatially varying and compositional degradations, while maintaining favorable parameter, FLOP, and runtime efficiency.