ICML 2026 论文列表

Existing LLM test-time scaling laws emphasize the emergence of self-reflective behaviors through extended reasoning length. Nevertheless, this vertical scaling strategy often encounters plateaus in exploration as the model becomes locked into specific thinking pattern. By shifting from depth to parallelism, parallel thinking mitigates the narrowing of exploration. However, the extension of this paradigm to visual domain remains an open research question. In this paper, we first examine the role of visual partitioning in parallelized reasoning and subsequently propose two distinct strategies. Based on the above, we introduce Visual Para-Thinker, representing the inaugural parallel reasoning framework for MLLMs. To maintain path independence and promote diversity in reasoning, our approach integrates Pa-Attention alongside LPRoPE. Leveraging the vLLM framework, we have developed a native multimodal implementation that facilitates high-efficiency parallel processing. Empirical results on benchmark datasets such as V*, CountBench, RefCOCO, and HallusionBench confirm that Visual Para-Thinker successfully extends the benefits of parallel reasoning to the visual domain.

Recently, large language models (LLMs) have driven a systematic shift in the graph ML com- munity through the adoption of text-attributed graphs (TAGs). Although a variety of frameworks have been developed, most fail to properly ad- dress the challenge of data uncertainty in open- world environments. A representative source of such uncertainty is the limited availability of la- bels in large-scale datasets due to high annotation costs, where unlabeled nodes may belong to either known classes or novel, unknown classes. While node-level out-of-distribution detection and con- ventional open-world graph learning attempt to tackle this problem, two core limitations remain: ① Insufficient methods — existing approaches typically optimize semantics or topology in isola- tion for unknown-class rejection, failing to effec- tively integrate textual and structural information in TAGs; ② Incomplete pipelines — most stud- ies conduct only idealized analyses, such as as- suming a predefined number of unknown classes, which restricts practical utility for model updates and long-term deployment. To overcome these issues, we introduce the Open-world Graph Assis- tant (OGA), an LLM-based framework. OGA first performs unknown-class rejection via adaptive la- bel traceability (ALT), harmoniously combining semantic and topological cues, and then applies the graph label annotator (GLA) for unknown- class annotation, allowing unlabeled nodes to con- tribute to model training. In essence, OGA offers a new pipeline that fully automates the handling of unlabeled nodes in open-world environments, and we establish a systematic benchmark cover- ing four key aspects to validate its effectiveness and practicality through extensive experiments.

The field of computer architecture, which bridges high-level software abstractions and low-level hardware implementations, remains absent from current large language model (LLM) evaluations. To this end, we present QuArch (pronounced ‘quark’), the first benchmark designed to facilitate the development and evaluation of LLM knowledge and reasoning capabilities specifically in computer architecture. QuArch v1.0 provides a comprehensive collection of 2,671 expert-validated question-answer (QA) pairs covering various aspects of computer architecture, including processor design, memory systems, and interconnection networks. Our evaluation reveals that while frontier models possess domain-specific knowledge, they struggle with skills that require higher-order thinking in computer architecture. Frontier model accuracies vary widely (from 34% to 73%) on these advanced questions, highlighting persistent gaps in architectural reasoning across analysis, design, and implementation QAs. Furthermore, via fine-tuning we find that QuArch can translate to improved performance on a realistic memory hierarchy design task, resulting in up to 1.99× more area-efficient solutions and up to 40% more viable solutions overall. By holistically assessing fundamental skills, QuArch provides a foundation for building and measuring LLM capabilities that can accelerate innovation in computing systems. The QuArch benchmark and leaderboard are publicly available at: https://quarch.ai/.

Vision-Language Pre-trained (VLP) models are vulnerable to real-world query noise. Current cross-modal Test-Time Adaptation (TTA) methods often rely on high-confidence predictions, which induces confirmation bias and neglects the informative signals in ambiguous Low-Confidence Queries. To address this, we propose Generative Semantic Refinement and Exclusion-Guided Adaptation (ReEx), a robust retrieval framework that extends adaptation to the entire query stream. Specifically, textual structural noise is rectified by a Generative Semantic Refinement (GSR) module, which employs Confidence-Guided Dynamic Fusion to anchor LLM-based repairs and prevent semantic drift. To exploit ambiguous data, adaptation is driven by Exclusion-Guided Proxy Contrastive Learning (EPCL), which imposes negative constraints via Exclusion Sets of unlikely candidates. Experimental results on COCO-C and Flickr-C demonstrate that ReEx consistently outperforms existing TTA methods, achieving significant robustness gains with a justifiable computational trade-off.

While self-consistency methods have emerged as a promising approach to enhance the correctness of large language model (LLM) outputs by aggregating multiple stochastic samples, they suffer from two critical limitations, resulting in high computation cost. First, they evaluate output consistency monolithically, failing to efficiently combine partially correct answers across multiple samples. Second, they use static stopping criteria that cannot adapt to varying task complexities and model capabilities, resulting in suboptimal computational efficiency.In this work, we present Task-and-Model-Aware Fractal-Consistency (TMAFC), a novel self-consistency framework that addresses these limitations through two key innovations: (1) Fractal-Consistency, which evaluates output consistency at the granularity of output components to effectively combine partial correct answers across samples, and (2) Adaptive Stopping Criteria Calibration (ASCC), which dynamically adjusts sampling stopping criteria based on real-time assessment of both task difficulty and LLM capability. Through extensive experiments on diverse question-answering benchmarks, we demonstrate that TMAFC achieves superior efficiency-accuracy trade-offs, reducing sample cost by up to 55% while maintaining competitive accuracy compared to state-of-the-art baselines.

Vision-Language-Action (VLA) models show strong generalization for robotic control, but finetuning them with reinforcement learning (RL) is constrained by the high cost and safety risks of real-world interaction. Training VLA models in interactive world models avoids these issues but introduces several challenges, including pixel-level world modeling, multi-view consistency, and compounding errors under sparse rewards. Building on recent advances across large multimodal models and model-based RL, we propose VLA-MBPO, a practical framework to tackle these problems in VLA finetuning. Our approach has three key design choices: (i) adapting unified multimodal models (UMMs) for data-efficient world modeling; (ii) an interleaved view decoding mechanism to enforce multi-view consistency; and (iii) chunk-level branched rollout to mitigate error compounding. Theoretical analysis and experiments across simulation and real-world tasks demonstrate that VLA-MBPO significantly improves policy performance and sample efficiency. Crucially, our method maintains a universal set of hyperparameters across all tasks, underscoring its robustness and scalability for real-world robotic deployment.

Vision-language models can perform new tasks without parameter updates through in-context learning (ICL), whose core mechanism is utilizing the support set for task induction. In the standard ICL setting, once the task is induced, its decision criterion remains fixed. However, in real-world applications, many tasks exhibit a stable high-level intent, while their decision criteria shift according to specific requirements. Thus, we introduce a new setting, denoted as Criterion-Conditional In-Context Learning (CC-ICL), where models must infer the latent criterion from context and adjust predictions accordingly under fixed task semantics. To evaluate this capability, we propose two complementary metrics, Criterion Invariance and Criterion Sensitivity, capturing the model's robustness and adaptability under criterion shifts. We further construct CC-Bench, a multi-domain benchmark that supports evaluation under the CC-ICL setting. By employing a dual-level data hierarchy, CC-Bench enables legitimate ground-truth variation conditioned on the active criterion even when the task remains fixed. Experiments on CC-Bench reveal that most models exhibit a rigid boundary bias, struggling to align their decisions with the latent criterion. We also find that even a simple multi-criterion training strategy can significantly reduce this bias, improving Criterion Sensitivity and enabling 7B-scale models to surpass proprietary models without degrading general multimodal performance.

Real-time satellite video tracking poses distinct challenges, including accommodating high spatial-temporal resolution, dynamic backgrounds, and constrained onboard computational resources. While Discriminative Correlation Filter (DCF)-based methods offer high-speed inference, they suffer from limited accuracy. In contrast, Vision Transformer (ViT)-based trackers achieve strong performance by unifying representation and aggregation in a single-stream design, yet their heavy computational footprint limits practical deployment in real-time satellite scenarios. In this work, we present FATrack, a novel tracking framework that effectively balances tracking accuracy and computational efficiency. At its core is FA-ViT, a lightweight Vision Transformer backbone that introduces foreground-aware token routing, enabling the model to concentrate computation on target-relevant regions while suppressing redundancy. To mitigate semantic degradation caused by token sparsification, we propose the Adaptive Scatter Module (ASM), which selectively reinforces informative tokens via joint spatial-channel attention and sparse structural propagation, thereby enhancing both semantic fidelity and spatial coherence. By synergistically integrating FA-ViT and ASM, FATrack forms a unified architecture that delivers real-time performance with significantly improved tracking precision. Extensive evaluations on multiple satellite video benchmarks demonstrate that FATrack surpasses existing real-time trackers in accuracy and achieves inference efficiency comparable to DCF-based methods, highlighting its potential for practical deployment in large-scale aerial video tracking systems.

Graph alignment, the problem of identifying corresponding nodes across multiple graphs, is fundamental to numerous applications. Most existing unsupervised methods embed node features into latent representations to enable cross-graph comparison without ground-truth correspondences. However, these methods suffer from two critical limitations: the degradation of node distinctiveness due to oversmoothing in GNN-based embeddings, and the misalignment of latent spaces across graphs caused by structural noise, feature heterogeneity, and training instability, ultimately leading to unreliable node correspondences. We propose a novel framework employing a dual-pass encoder to inject high-frequency discriminability into node features, paired with a geometry-aware functional map module that learns bijective and isometric transformations to align latent spaces while acting as a low-pass filter on correspondences, enforcing smoothness and robustness as a structural prior in map space. Extensive experiments on graph benchmarks demonstrate that our method consistently outperforms existing unsupervised alignment baselines, exhibiting superior robustness to structural inconsistencies and challenging alignment scenarios. The implementation is available at https://github.com/maysambehmanesh/GADL.

Concept erasure aims to prevent image generative models from producing unsafe content while preserving their general generative capability. Meanwhile, next-scale autoregressive (AR) image generation has recently emerged as a new generative paradigm characterized by next-scale prediction, for which concept erasure remains largely unexplored. In this paradigm, semantic information is highly compressed at early scales, leading to severe entanglement between unsafe and unrelated semantics. In this paper, we propose ScaleErasure, an inference-time concept erasure method that performs minimal intervention. ScaleErasure precisely selects and guides predicted logits that are most relevant to the unsafe concept, thereby enabling effective erasure under severe semantic entanglement. Specifically, ScaleErasure performs two additional forward passes conditioned on the unsafe concept and the corresponding safe concept, and leverages their outputs to guide the target logits away from unsafe concepts toward safe concepts. To enable precise and minimal intervention, logits selection and guidance are conducted across three dimensions: scales, tokens, and bit channels. Experiments demonstrate that ScaleErasure outperforms adapted baselines in the next-scale AR paradigm, achieving more precise concept erasure while largely preserving general generative capability.

Recent progress in Reinforcement Learning (RL) provides a principled approach to optimizing Vision-Language-Action (VLA) models, facilitating a shift from trajectory imitation to active learning in the task environment. Despite improvements in control precision, most RL optimizers remain task-specific, which reduces VLA models from generalist controllers to policies that overfit to a narrow set of tasks. In this study, we conduct an in-depth analysis of this phenomenon and highlight the importance of cross-task feature representations for improving the generalizability of VLA models. Motivated by this finding, we introduce DyGRO-VLA, a two-stage optimization framework that 1) effectively captures cross-task latent representations based on information-theoretic principles, and 2) dynamically refines policy optimization via a mixture-of-RL-residuals. DyGRO-VLA enables the RL optimizer to exploit task-relevant latent information while strategically mitigating adverse interference on the learned representations throughout the optimization process. We evaluate our approach on LIBERO, RoboTwin2 benchmarks, and further validate it on real world, demonstrating consistent improvements over strong baselines under multi-task training and distribution shift. Our project page is available at DyGRO-VLA.

Generative AI is reshaping healthcare, yet most existing advances rely on hospital-grade devices, which limits their accessibility and potential for health management outside clinical settings. With the proliferation of portable devices and telemedicine, healthcare is shifting toward home-based Diagnosis-It-Yourself (DIY) care. Despite this promise, several distinctive challenges remain: (i) home-collected data are heterogeneous, exacerbated by the absence of standardized large-scale datasets; (ii) models require adaptation to variable task demands and evolving individual conditions; (iii) the broad spectrum of home care tasks lacks a unified benchmark for systematic evaluation. In this paper, we present **DIYHealth Suite**, a comprehensive framework designed to address these challenges through a tailored dataset, model, and benchmark. We first curate **DIYHealth-900K**, a large-scale multimodal dataset capturing diverse real-world home care scenarios. Building on this, we propose **DIYHealthGPT**, an adaptive foundation model for home-based health management, powered by the novel Hybrid Hyper Low-Rank Adaptation technique. Finally, we establish **DIYHealthBench**, the first benchmark to evaluate foundation models on home care tasks. Extensive experiments demonstrate that DIYHealthGPT delivers state-of-the-art performance over both general-purpose and medical-specific baselines on 11 home care tasks in both open-QA and closed-QA settings, laying the groundwork for the next generation of personalized health management at home.

Multimodal Large Language Models (MLLMs) have shown strong performance in multi-image cross-modal retrieval, yet suffer from severe position bias, where predictions are dominated by input order rather than semantic relevance. Through empirical analysis, we identify a phenomenon termed Logit-Attention Divergence, in which output logits are heavily biased while internal attention maps remain well-aligned with relevant visual evidence. This observation reveals a fundamental limitation of existing logit-level calibration methods such as PriDe. Based on this insight, we propose a training-free, attention-guided debiasing framework that leverages intrinsic attention signals for instance-level correction at inference time, requiring only a minimal calibration set with negligible computational overhead. Experiments on MS-COCO-based benchmarks show that our method substantially improves permutation invariance and achieves state-of-the-art performance, enhancing accuracy by over 40\% compared to baselines. Code is available at https://github.com/brightXian/LAD.

Vision-Language Models (VLMs) achieve strong multimodal performance but are costly to deploy, and post-training quantization often causes significant accuracy loss. Despite its potential, quantization-aware training for VLMs remains underexplored. We propose GRACE, a framework unifying knowledge distillation and QAT under the Information Bottleneck principle: quantization constrains information capacity while distillation guides what to preserve within this budget. Treating the teacher as a proxy for task-relevant information, we introduce confidence-gated decoupled distillation to filter unreliable supervision, relational centered kernel alignment to transfer visual token structures, and an adaptive controller via Lagrangian relaxation to balance fidelity against capacity constraints. Across extensive benchmarks on LLaVA and Qwen families, our INT4 models consistently outperform FP16 baselines (e.g., LLaVA-1.5-7B: 70.1 vs. 66.8 on SQA; Qwen2-VL-2B: 76.9 vs. 72.6 on MMBench), nearly matching teacher performance. Using real INT4 kernel, we achieve 3× throughput with 54\% memory reduction. This principled framework significantly outperforms existing quantization methods, making GRACE a compelling solution for resource-constrained deployment. Code and data are available at: https://github.com/ForeverBlue816/GRACE.

Image forgery is rapidly evolving, rendering forensic traces increasingly subtle and readily attenuated by post-processing. Although vision-language prompting can inject priors, open-ended LLM-generated prompts are difficult to constrain, and naive language descriptions can introduce semantic perturbations. To address these challenges, we propose *Forensic Prompting with Dual-Action policy optimization* (FPDA) for vision-language forgery detection and localization, where the *Forensic Prompting Module* (FPM) constructs a structured and reproducible forensic prompt bank and supports optional text input as a reliability-aware cue for stable conditioning. Moreover, *Dual-Action Policy Optimization* (DAPO) is applied to learn sample-adaptive evidence usage by routing forensic prompts and scheduling localization refinement on a per-image basis, stabilizing discriminative cues and improving mask spatial consistency. Extensive experiments are conducted on multiple public datasets covering manual manipulations, diffusion content, face forgeries, and text-enabled settings, demonstrating favorable detection and localization performance over representative state-of-the-art methods under comparable evaluation protocols.

LoRA and its variants have attracted considerable attention because of their abilities to tune a negligible number of parameters while achieving comparable downstream performance. This success is largely attributed to the intrinsic low-rank structure of model parameter spaces, which allows LoRA to train two projection matrices to project weights into a low-dimensional subspace and then map them back. However, it does not consider how to explore this low-rank subspace sufficiently and may lose the expression ability accordingly. Moreover, when using LoRA to tune convolution layers, a flatten operation is required to convert tensors into matrices. We argue that this will degrade the model's performance. In this paper, we address this issue from a general parameter sub-space perspective: we present a unified **L**anguage **A**nd **V**ision **A**daption finetuning framework (called **LAVA**). Specifically, we verify the existence of low-rank subspaces in convolution layers empirically and propose to parameterize the increment of both convolution kernels and matrices as sum of learnable rank-1 components. To improve training stability, we analyze the optimization dynamics of LoRA and incorporate orthogonal regularization into our parameterization, for which we give theoretical proof that it will help reduce the variance of the gradient. We conduct various experiments on different downstreaming tasks to validate LAVA's superiority. For example, when tuning LLaMA2-7b for commonsense tasks, the performance of our LAVA is **+1.9\%** higher than that of LoRA. For metric depth estimation tasks, LAVA only tunes $\sim$1.5\% of Depth-Anything (335.3M), and achieves **+3.5\%** $\delta_1$ accuracy against that of LoRA and **+5.6\%** $\delta_1$ accuracy against that of SVDiff.

The visual world offers a critical axis for advancing foundation models beyond language. Despite growing interest in this direction, the design space for native multimodal models remains opaque. We provide empirical clarity through controlled, from-scratch pretraining experiments, isolating the factors that govern multimodal pretraining without interference from language pretraining. We adopt the Transfusion framework, using next-token prediction for language and diffusion for vision, to train on diverse data including text, video, image-text pairs, and even action-conditioned video. Our experiments yield four key insights: (i) Representation Autoencoder (RAE) provides an optimal unified visual representation by excelling at both visual understanding and generation; (ii) visual and language data are complementary and yield synergy for downstream capabilities; (iii) unified multimodal pretraining leads naturally to world modeling, with capabilities emerging from general training; and (iv) Mixture-of-Experts (MoE) enables efficient and effective multimodal scaling while naturally inducing modality specialization. Through IsoFLOP analysis, we compute scaling laws for both modalities and uncover a scaling asymmetry: vision is significantly more data-hungry than language. We demonstrate that the MoE architecture harmonizes this scaling asymmetry by providing the high model capacity required by language while accommodating the data-intensive nature of vision, paving the way for truly unified multimodal models.

In large vision-language models, visual tokens typically constitute the majority of input tokens, leading to substantial computational overhead. To address this, recent studies have explored pruning redundant or less informative visual tokens for image understanding tasks. However, these methods struggle with pixel grounding tasks, where token importance is highly contingent on the input text. Through an in-depth analysis of CLIP, we observe that visual tokens within referent regions often exhibit low similarity to their textual representation. Motivated by this insight, we introduce LiteLVLM, a training-free, text-guided token pruning strategy for efficient pixel grounding inference. By reversing the ranking of CLIP's visual-text similarity, LiteLVLM effectively retains visual tokens covering the referent regions, while recovering context tokens to enable clear foreground-background separation. Extensive experiments demonstrate that LiteLVLM significantly outperforms existing methods by over 5% across diverse token budgets. Without any training or fine-tuning, LiteLVLM maintains 90% of the original performance with a 22% speedup and a 2.3$\times$ memory reduction. Our code is available at https://github.com/sejong-rcv/LiteLVLM.

As Large Multimodal Models (LMMs) scale up and reinforcement learning (RL) methods mature, LMMs have made notable progress in complex reasoning and decision making. Yet training still relies on static data and fixed recipes, making it difficult to diagnose capability blind spots or provide dynamic, targeted reinforcement. Motivated by findings that test driven error exposure and feedback based correction outperform repetitive practice, we propose Diagnostic-driven Progressive Evolution (DPE), a spiral loop where diagnosis steers data generation and reinforcement, and each iteration re-diagnoses the updated model to drive the next round of targeted improvement. DPE has two key components. First, multiple agents annotate and quality control massive unlabeled multimodal data, using tools such as web search and image editing to produce diverse, realistic samples. Second, DPE attributes failures to specific weaknesses, dynamically adjusts the data mixture, and guides agents to generate weakness focused data for targeted reinforcement. Experiments on Qwen3-VL-8B-Instruct and Qwen2.5-VL-7B-Instruct show stable, continual gains across eleven benchmarks, indicating DPE as a scalable paradigm for continual LMM training under open task distributions. Our code, models, and data are publicly available at https://github.com/hongruijia/DPE.