ICML 2026 论文列表

Multimodal Large Language Models (MLLMs) have demonstrated exceptional capabilities in high-level visual understanding. However, extending these models to fine-grained dense prediction tasks, such as semantic segmentation and depth estimation, typically necessitates the incorporation of complex, task-specific decoders and other customizations. This architectural fragmentation increases model complexity and deviates from the generalist design of MLLMs, ultimately limiting their practicality. In this work, we challenge this paradigm by accommodating standard MLLMs to perform dense predictions without requiring additional task-specific decoders. The proposed model is called DenseMLLM, grounded in the standard architecture with a novel vision token supervision strategy for multiple labels and tasks. Despite its minimalist design, our model achieves highly competitive performance across a wide range of dense prediction and vision-language benchmarks, demonstrating that a standard, general-purpose MLLM can effectively support dense perception without architectural specialization. This project is available at github.com/Eli-YiLi/DenseMLLM.

Efficient transfer learning of vision–language models (VLMs) commonly suffers from a Base–New Trade-off (BNT): improving performance on unseen (new) classes often degrades accuracy on known (base) classes. Addressing how to boost recognition of unseen classes without sacrificing known-class performance remains a central challenge. Existing work often simplistically attributes the BNT to overfitting on known classes. We observe an interesting phenomenon: VLMs frequently exhibit asymmetric confusion on certain downstream data, i.e., samples of class A are systematically mispredicted as class B, while the reverse confusion (B → A) rarely occurs. For known classes, this kind of bias can be mitigated by tuning using a cross-entropy loss, but for unseen classes, such pretraining-induced bias persists and harms generalization. Motivated by this, we propose NeRP, a plug-and-play prompting correction strategy that improves discrimination on unseen classes without modifying model parameters. NeRP leverages neutral text prompts and reference images to measure class-wise prior preferences along the pre-trained inter-class geometry, and combines them with the sample likelihood to obtain the model’s surrogate score. If, for a given sample, the prior strongly favors the current prediction while the observed evidence is clearly insufficient, we perform a local flip between easily confusable class pairs, thereby correcting prior-dominated mispredictions. Extensive experiments across multiple backbones and 15 few-shot and cross-domain benchmarks show that NeRP substantially improves accuracy on unseen classes while preserving known-class prediction performance.

Efficiently enhancing the reasoning capabilities of Vision-Language Models (VLMs) by merging them with Large Reasoning Models (LRMs) has emerged as a promising direction. However, existing methods typically operate at a coarse-grained layer level, which often leads to a trade-off between injecting reasoning capabilities and preserving visual capabilities. To address this limitation, we propose FRISM (Fine-grained Reasoning Injection via Subspace-level model Merging), a fine-grained reasoning injection framework based on subspace-level model merging. Observing that different SVD subspaces contribute differently to reasoning and perception, FRISM decomposes LRM task vectors via Singular Value Decomposition (SVD) and adaptively tunes the scaling coefficients of each subspace through learning to realize fine-grained reasoning injection. Furthermore, we introduce a label-free self-distillation learning strategy with dual-objective optimization using common vision-language perception datasets. Extensive experiments demonstrate that FRISM effectively improves reasoning capabilities while largely preserving the model's visual capabilities by consistently achieving strong performance across diverse visual-language reasoning benchmarks.

Large Multimodal Models (LMMs) exhibit shortfalls when interpreting images and, by some measures, have poorer spatial cognition than young children or animals. Despite this, they attain high scores on many popular visual benchmarks, with headroom rapidly eroded by model progress. This creates a need for difficult benchmarks that remain relevant for longer. We introduce ZeroBench—a lightweight visual reasoning benchmark curated using adversarial filtering to be “impossible” for frontier LMMs at its original release, with initial SotA scores of 0% pass@1 and pass∧5. We track progress on ZeroBench over the subsequent year, observing SotA reaching 6% pass∧5 and 19% pass@5, indicating the potential longevity of the benchmark. We evaluate 46 LMMs on ZeroBench, compare performance to a human baseline, analyse strengths and weaknesses, chart a year of progress in visual capabilities, and publicly release ZeroBench at https://zerobench.github.io/.

While recent multimodal large language models (MLLMs) have made impressive strides, they mostly employ a conventional autoregressive architecture as their backbone, leaving significant room for exploring effective and efficient alternatives in architectural design. Meanwhile, recent studies have successfully applied discrete diffusion models to natural language processing, revealing their considerable potential as a promising new approach in this domain. Drawing inspiration from these pioneering studies, we introduce Omni-Diffusion, the first any-to-any multimodal language model built entirely on mask-based discrete diffusion models, which unifies understanding and generation across text, speech, and images. Omni-Diffusion employs a unified mask-based discrete diffusion model to directly capture the joint distribution over discrete multimodal tokens. This approach supports not only bimodal tasks but also more complex scenarios involving multiple modalities. On a diverse set of benchmarks, our method outperforms or performs on par with existing multimodal systems that process two or more modalities, highlighting the significant promise of diffusion models in powering the next generation of multimodal foundation models. Our codes are released at [GitHub](https://omni-diffusion.github.io).

Multimodal large language models (MLLMs) frequently suffer from object hallucinations, yet the visual perceptual mechanism underlying this failure remains poorly understood. In this work, we reveal that hallucinations are strongly associated with a human-like attention distraction phenomenon, where humans under divided focus experience degraded visual clarity and produce inaccurate descriptions, while in models the same mechanism manifests as spatial inconsistency in multi-head attention and temporal fading of attention to image tokens during decoding. We further provide theoretical insights that attention dispersion increases model complexity and degrades classification generalization. Motivated by these findings, we propose an Attention-Focused Approach for Improved Image Perception (AFIP), which corrects attention distraction via cross-head attention enrichment and reinforces visual grounding through dynamic historical attention enhancement. Extensive experiments on multiple benchmarks and models validate the effectiveness of AFIP without additional training.

Real-world degradations, such as noise, blur, and low resolution, significantly impair the performance of Multi-modal Large Language Models (MLLMs) in document understanding tasks. Despite recent advancements, progress in this field remains stifled by two critical bottlenecks: the scarcity of large-scale, aligned training data necessary for learning robustness, and the lack of transferable restoration solutions across diverse MLLM architectures. To bridge the data gap, we first present DocRobust-VQA, a large-scale dataset explicitly constructed to support robustness training. Comprising 189K aligned clean/corrupted document image pairs and 417K QA pairs, it provides the first substantial corpus for fine-tuning MLLMs to handle varying degradation conditions. Leveraging this data, we propose Uni-DocRobust, a universal plug-and-play framework that decouples restoration capabilities from specific visual encoders. Our method employs a frozen Universal Restoration Core pre-trained in a canonical feature space via multi-teacher distillation, which can be seamlessly integrated into target MLLMs (e.g., Qwen-VL, InternVL) through lightweight Feature Adapters. Extensive experiments demonstrate that Uni-DocRobust significantly enhances robust performance on MLLMs and enables a cost-effective ``pre-train once, deploy everywhere'' paradigm for robust MLLM deployment.

Large-scale vision–language models (VLMs) excel at multimodal reasoning, yet efficiency collapses when vision tokens—often orders of magnitude more than text—dominate compute and memory. Prior token-reduction strategies typically trade off salience (which is prone to position bias and incurs extra computation) against diversity (which can under-cover salient regions and is sensitive to hyperparameters). We present SPLIT, a theoretically grounded framework that jointly preserves salience and diversity while aggressively eliminating redundancy. SPLIT (i) estimates token importance via temporal shifts of hidden states across layers—eschewing attention scores and their biases; (ii) assigns adaptive region-level budgets to guarantee localized coverage; and (iii) selects tokens using a diversity score that prioritizes distinctive, non-redundant representations. Our analysis shows that adaptive budgeting yields tighter coverage guarantees than uniform allocation, and our selection rule maintains diversity without costly tuning. Empirically, SPLIT consistently outperforms state-of-the-art on image and video understanding benchmarks. On image understanding with LLaVA-1.5-7B, SPLIT preserves over 99\% accuracy with 192 vision tokens and about 92.8\% with only 64 tokens, demonstrating robust performance under severe token budgets. These results indicate that SPLIT delivers scalable, attention-score-free token reduction that makes multimodal reasoning substantially more efficient without sacrificing accuracy.

Multimodal visual object tracking can be divided into to several kinds of tasks (e.g. RGB and RGB+X tracking), based on the input modality. Existing methods often train separate models for each modality or rely on pretrained models to adapt to new modalities, which limits efficiency, scalability, and usability. Thus, we introduce OneTrackerV2, a unified multi-modal tracking framework that enables end-to-end training for any modality. We propose Meta Merger to embed multi-modal information into a unified space, allowing flexible modality fusion and robustness. We further introduce Dual Mixture-of-Experts (DMoE): T-MoE models spatiotemporal relations for tracking, while M-MoE embeds multi-modal knowledge, disentangling cross-modal dependencies and reducing feature conflicts. With a shared architecture, unified parameters, and a single end-to-end training, OneTrackerV2 achieves state-of-the-art performance across five RGB and RGB+X tracking tasks and 12 benchmarks, while maintaining high inference efficiency. Notably, even after model compression, OneTrackerV2 retains strong performance. Moreover, OneTrackerV2 demonstrates remarkable robustness under modality-missing scenarios.

Large-scale web-crawled datasets contain noise, bias, and irrelevant information, necessitating data selection techniques. Existing methods depend on hand-crafted heuristics, downstream datasets, or require expensive influence-based computations---all of which limit scalability and introduce unwanted data dependencies. To address this, we introduce the Mimic Score, a simple and geometry-based data-quality metric that evaluates utility by measuring alignment between a sample’s gradients and a target direction induced by a pre-trained reference model. This leverages readily available model weights, avoids needing validation datasets, and incurs minimal computational overheads. Building on this metric, we propose Grad-Mimic, a two-stage framework that re-weights samples online to accelerate training and aggregates sample utilities offline to construct effective data filters. Empirically, we show that using mimic scores to guide training improves data efficiency, accelerates convergence, yields consistent performance gains across six image datasets, and enhances CLIP models with 20.7\% fewer training steps. Additionally, mimic score-based filters augment existing filtering techniques, enabling improved CLIP models trained with 4.7 million fewer samples.

Vision-Language-Action (VLA) models have shown strong potential for robotic manipulation, yet adapting pretrained models to novel tasks typically relies on substantial task-specific demonstrations, limiting scalability. Current VLA methods mostly focus on action imitation, which ignores the richer structure contained in trajectories. In contrast, motion dynamics governing how actions evolve over time are more informative and transferable, making them better suited for few-shot adaptation. Motivated by this idea, we propose DynVLA, a few-shot adaptation system that reformulates VLA learning from action imitation to trajectory-level motion dynamics modeling. Specifically, we propose Motion Dynamics Mechanism (MDM), which distills latent embeddings from trajectories via flow-matching inversion, yielding compact representations that capture dynamics. We further design Dynamics-Constrained Modeling (DCM). DCM projects these inferred representations onto a Dynamics Bank, which stores prior motion knowledge pretrained from diverse demonstrations. By grounding action generation in these learned priors, the system enables interpolating between existing action paradigms to represent novel dynamics modes. Experiments on 13 real-world tasks demonstrate that DynVLA outperforms existing SOTA systems by 15.8\% in average success rate with around 20 demonstrations, highlighting its adaptation capabilities in real-world scenes. Our code is available at https://github.com/trantor2nd/Motion-DynVLA.

Large Multimodal Models encode extensive factual knowledge in their pre-trained weights. However, its knowledge remains static and limited, unable to keep pace with real-world developments, which hinders continuous knowledge acquisition. Effective knowledge injection thus becomes critical, involving two goals: knowledge adaptation (injecting new knowledge) and knowledge retention (preserving old knowledge). Existing methods often struggle to learn new knowledge and suffer from catastrophic forgetting. To address these challenges, we propose KORE, a synergistic method centered around KnOwledge-oRientEd controls. These controls are implemented through a two-stage optimization process: (1) KORE automatically converts individual knowledge items into structured and comprehensive knowledge to ensure that the model accurately learns new knowledge, enabling accurate adaptation. (2) KORE stores previous knowledge in the covariance matrix of LMM's linear layer activations and initializes the adapter by projecting the original weights into the matrix's null space, defining a fine-tuning direction that minimizes interference with previous knowledge, enabling powerful retention. Extensive experiments on various LMMs, including LLaVA-v1.5 (7B), LLaVA-v1.5 (13B), and Qwen2.5-VL (7B), show that KORE achieves superior new knowledge injection performance and effectively mitigates catastrophic forgetting.

The utility of Vision-Language Models (VLMs) in reasoning and auditing tasks hinges on their ability to exhaustively describe visual scenes. However, current models exhibit a pathology we term the Likelihood Trap: standard alignment objectives, specifically MLE and KL-regularization, drive generation toward generic, high-probability templates, systematically suppressing fine-grained details. To overcome this, we introduce Geo-RL, a framework that shifts the objective from probabilistic likelihood to geometric coverage. Geo-RL reformulates caption generation as maximizing the volume of a parallelotope in semantic space. By leveraging Determinantal Point Processes (DPPs), we enforce orthogonality among sampled descriptions, ensuring that they span the image's full semantic support. Crucially, we derive a closed-form leave-one-out marginal reward, enabling stable policy optimization. Empirically, Geo-RL escapes the trap, achieving a significant improvement in semantic richness and detail coverage without compromising visual grounding.

Large Language Models (LLMs) frequently generate output that contradicts explicit input evidence, limiting their reliability in real-world applications. We identify cognitive inertia in LLMs—a tendency to overly rely on co-occurrence associations learned during pretraining and to resist adaptation when conflicting input evidence appears—as a critical factor behind such hallucinations. We further empirically show that adherence to input evidence declines as co-occurrence associations are strengthened—driven by either higher data frequency or intensified training. Inspired by human counter-inertial thinking, we propose an adaptive counter-inertial reasoning framework that probes input-related cognitive inertia in the LLM and generates adaptive counter-inertial reminders, which are then injected into the prompt to promote evidence-based reasoning. Experiments on co-occurrence induction datasets show that LLMInertia reduces hallucination rates by up to 35\% and improves accuracy by up to 35.68\%. Extensive evaluations on four context-rich summarization and QA datasets, across three LLM backbones of varying scales, further validate its effectiveness and robustness. Our work provides new insight into the causes of input-unfaithful hallucinations in LLMs, contributing to the development of more reliable AI.

This paper presents a theoretical framework that explains why fine-tuning small, randomly selected subnetworks (slices) within pre-trained models is sufficient for downstream adaptation. We establish that pretrained networks exhibit a universal winning slice property, arising from two phenomena: (1) spectral balance— the eigenspectra of different weight matrix slices are remarkably similar—and (2) high task energy—their backbone representations (pretrained weights) retain rich, task-relevant features. This leads to the Universal Winning Slice Hypothesis, which provides a theoretical foundation for parameter-efficient fine-tuning (PEFT) in large-scale models. Inspired by this, we propose SliceFine, a PEFT method that uses this inherent redundancy by updating only selected slices of the origi- nal weights—introducing zero new parameters, unlike adapter-based approaches. Empirically, SliceFine matches the performance of SOTA PEFT methods across various language and vision tasks, while significantly improving training speed, memory efficiency, and model compactness. Our work bridges theory and prac- tice, offering a theoretically grounded alternative to existing PEFT techniques.

In critical domains like clinical reporting, legal analysis, and policy drafting, large language models (LLMs) are increasingly expected to produce extended, fact‑rich narratives rather than isolated sentences. Reliable uncertainty quantification in such long‑form outputs is crucial. Existing techniques either assign a single confidence score to an entire paragraph or evaluate factual consistency by comparing extracted atomic facts across multiple generations. Some recent approaches represent fact–paragraph relationships using bipartite entailment graphs and derive uncertainty from fact centrality. However, these methods ignore the explicit dependencies among facts within a paragraph and the structural and semantic variation across multiple LLM outputs for the same prompt, missing a key source of uncertainty specific to long‑form generation. We propose **GAUSS** (**G**raph‑**A**ssisted **U**ncertainty **Q**uantification using **S**tructure and **S**emantics), a principled framework that models each generated paragraph as a semantic graph of atomic facts and their relations. We posit that uncertainty arises from structural and semantic discrepancies among these graphs across different samples. **GAUSS** quantifies uncertainty as the expected alignment cost between the semantic graph of an anchor paragraph and those of alternative generations. By capturing both semantic content and structural coherence, **GAUSS** offers a more interpretable and theoretically grounded measure of uncertainty than coarse, sentence‑level scores.

Unified multimodal models capable of both understanding and generation have achieved remarkable strides. However, despite their unified designs, existing evaluations typically assess understanding and generation capabilities in isolation, overlooking the synergy between comprehension and generation. To bridge this gap, we introduce **Unison**, a comprehensive benchmark comprising 2,169 high-quality unified task samples, designed to evaluate joint understanding and generation in unified multimodal models. Unison offers three key strengths: **1) Comprehensive Dimensions**: Unison encompasses internal consistency, understanding-guided generation, generation-guided understanding, and mutual enhancement to enable holistic evaluation. **2) Diagnostic Evaluation**: it provides both unified and decoupled tracks for understanding and generation, allowing fine-grained attribution of failure modes and quantitative analysis of the gains from unified modeling. **3) Human Alignment**: we also train Unison-Judge, an evaluation model well aligned with human judgments to achieve reliable assessment. Based on systematic evaluations of state-of-the-art models on Unison, we uncover critical limitations in current unified multimodal systems and highlight promising directions for future research. Unison will be publicly available at [https://github.com/FudanCVL/Unison](https://github.com/FudanCVL/Unison).

Vision-Language Models (VLMs) are typically deterministic in nature and lack intrinsic mechanisms to quantify epistemic uncertainty, which reflects the model’s lack of knowledge or ignorance of its own representations. We theoretically motivate negative log-density of an embedding as a proxy for the epistemic uncertainty, where low-density regions signify model ignorance. The proposed method REPVLM computes the probability density on the hyperspherical manifold of the VLM embeddings using Riemannian Flow Matching. We empirically demonstrate that REPVLM achieves near-perfect correlation between uncertainty and prediction error, significantly outperforming existing baselines. Beyond classification, we also demonstrate that the model also provides a scalable metric for out-of-distribution detection and automated data curation.

Training-free test-time adaptation (TTA) for vision-language models (VLMs) can improve zero-shot classification under mild shifts, but often degrades under severe style/environment variation. We identify two shared failure modes in training-free pipelines: (i) retrieval confounding, where feature similarity is dominated by nuisance/style variation and corrupts retrieval evidence; and (ii) environment-biased priors, where VLM logits exhibit environment-dependent centered shifts that distort gating and prior-like terms. Therefore, we propose D$^2$O, a training-free debiasing operator that outputs three inference-time objects: a retrieval-oriented content feature for semantic matching, a style-aware routing coordinate for bias tracking, and debiased logits for corrected priors. D$^2$O composes plug-and-play with retrieval-based and closed-form Gaussian adapters in online and transductive settings. We further provide operator-to-decision guarantees: finite-difference covariance recovers a nuisance-sensitive subspace, routing-based EMA controls centered-logit bias estimates, and these errors yield bounded posterior log-odds perturbations, leading to a margin-based condition for label invariance. Extensive experiments show that D$^2$O achieves its clearest gains under style/environment-dominant shifts, with broader gains elsewhere. Code is available at \url{https://github.com/MAiTL-Group/D2O}.