ECCV 2026 Papers

Unified multimodal models (UMMs) strive to consolidate visual understanding and visual generation within a single architecture. However, prevailing training paradigms independently optimize understanding via sparse text signals and generation through dense pixel objectives. Such a decoupled strategy yields misaligned representation spaces, isolating visual understanding from generation and hindering their mutual reinforcement. This work presents the first systematic investigation into generative post-training, where we formulate hierarchical visual tasks as generative proxies to bridge the isolation in UMMs. Our empirical investigation reveals that high-level semantic tasks, particularly image segmentation, serve as optimal proxies. Unlike low-level tasks that distract models with texture details, segmentation provides structural semantics that significantly enhance both vision-centric perception and generative layout fidelity. Building upon these insights, we introduce Semantic Generative Tuning (SGT), a novel paradigm that leverages segmentation as a generative proxy to align and synergize multimodal capabilities. Mechanistic analyses further demonstrate that SGT fundamentally improves feature linear separability and optimizes visualtextual attention allocation pattern. Extensive evaluations show that SGT consistently improves both multimodal comprehension and generative fidelity across mainstream benchmarks. Our code is available on the Project Page.

Despite rapid advances in chest X-ray (CXR) foundationmodels, most radiology report generation (RRG) systems still rely onheavily downsampled inputs (e.g., 256×256) due to the fixed visual tokenbudgets of pretrained vision encoders, suppressing subtle yet clinicallyimportant cues present in native-resolution images. However, enablinghigh-resolution (high-res) perception remains challenging: naïve tilingcauses prohibitive token inflation, while global compression suppressessubtle lesions and degrades diagnostic fidelity. Inspired by radiologists’workflow, localizing suspicious regions before detailed high-res assess-ment. We propose Lesion-Aware High-Resolution Patch Discovery andFusion for Chest X-ray Reporting (LePaX), the first RRG frameworkthat enables efficient high-res CXR perception (up to 1920×1920) with-out increasing the vision-token count. LePaX formulates high-res percep-tion as a constrained spatial resolution allocation problem under a fixedtoken budget and introduces two key components: Learnable Spatial Res-olution Allocation (LSRA), which learns a spatial utility map that adap-tively allocates limited high-res capacity to diagnostically relevant re-gions, enabling targeted extraction of high-res patches from native CXRs;and Global–Regional Fusion (GRF), which performs token-preservingregion-to-global refinement by projecting high-resolution regional evi-dence back onto the global feature grid through spatially aligned resolu-tion write-back, avoiding token inflation. Experiments on multiple CXRbenchmarks demonstrate that LePaX consistently improves both clinicaland linguistic metrics while enabling native-resolution CXR perceptionwith over 10× fewer visual tokens than naïve high-res tiling.

Cross-modal super-resolution (SR) on real-world misaligneddata is challenging, as only unlabeled low-resolution (LR) source andhigh-resolution (HR) guide images with complex spatial misalignmentare available. Previous methods either rely on simulated training dataor adopt suboptimal alignment strategies that overlook cross-modal de-pendencies, limiting their practical performance. To address these is-sues, we propose RobSelf, a self-supervised model that jointly optimizesa misalignment-aware feature translator and a content-aware referencefilter online. The translator resolves unsupervised cross-modal and cross-resolution alignment via weakly-supervised, misalignment-aware trans-lation, yielding an aligned guide feature. Guided by this feature, thefilter performs reference-based discriminative self-enhancement on thesource, enabling SR prediction with high resolution and high fidelity.Experiments on synthesized data and collected real-world data demon-strate that RobSelf achieves state-of-the-art performance, outperformingexisting self-supervised and supervised methods. Moreover, it achievessuperior efficiency, being up to 15.3× faster than prior self-supervisedmethods. https://github.com/palmdong/RobSelf

Reinforcement-learning alignment for autoregressive imagegeneration is bottlenecked by expensive rollouts: in GRPO-style groupoptimization, sampling G rollouts per prompt improves exploration andwithin-group credit estimation but multiplies sampling cost. This chal-lenge is amplified by vector-quantized (VQ) tokenization, where nearbycodebook IDs in embedding space can be locally substitutable; however,GRPO-style updates provide explicit positive credit only to sampled IDs,leading to insufficient coverage of success-supported alternatives undersmall-G training. We propose GR-GRPO (Graph-Regularized GRPO),which densifies token-level supervision without additional rollouts by dif-fusing positive evidence from positive rollouts over a precomputed code-book K-NN graph to construct graph-diffused soft targets, and regulariz-ing the policy toward these targets via an auxiliary cross-entropy term. Aconfidence-adaptive gate modulates diffusion strength based on the pol-icy’s logit margin, and a difficulty-aware budget allocation scheme fur-ther improves rollout utilization across prompts. Extensive experimentson compositional image generation (GenEval) and preference-style align-ment (DrawBench) validate the effectiveness of GR-GRPO. In particular,GR-GRPO achieves a substantially better compute–performance trade-off, surpassing large-group training (G=64) with ∼ 4.6× less rollout timeon Janus-Pro-1B. Our code and model weights are available athttps://github.com/vivoCameraResearch/GR-GRPO.

Unified anomaly detection requires modeling highly heteroge-neous normal data without access to anomalous samples. While foundationmodels like DINOv2 provide rich token representations, leveraging thesespaces for explicit density estimation remains challenging. Energy-BasedModels (EBMs) offer a principled formulation, but their training inhigh-dimensional token spaces is unstable due to anisotropy and strongcross-dimensional correlations, which degrades finite-step Markov ChainMonte Carlo (MCMC) sampling. We identify this instability as fundamen-tally geometric and introduce ReFP-AD (Rectified Flow Preconditioningfor Anomaly Detection), which learns a geometric reparameterization thatmaps high-dimensional embeddings into a well-conditioned latent spacevia an optimal transport (OT)-coupled rectified flow. This precondition-ing enables stable persistent contrastive divergence with preconditionedStochastic Gradient Langevin Dynamics (SGLD) in full-dimensional tokenspaces. Anomaly scores are then derived from the learned energy landscapeusing gradient norms. Under a strict unified protocol on the MVTec-ADand VisA datasets, ReFP-AD achieves 98.6%/97.9% Image/Pixel AUROCon MVTec-AD and 97.3%/99.0% on VisA, outperforming prior unifiedEBM baselines by up to +10.8% in Image AUROC. Ablation experimentsdemonstrate that geometric reparameterization is critical for finite-stepMCMC and accurate anomaly localization in high-dimensional tokenspaces. Code is available at https://github.com/CLendering/ReFP-AD

CLIP and its variants are widely adopted visual backbonesin multimodal systems, but their pretraining remains dominated by de-scriptive image-text alignment. As downstream applications increasinglydemand visually grounded commonsense inference and compositionalreasoning, it remains unclear whether CLIP-style encoders can supportsuch reasoning without architectural changes. To address this, we presentReasonCLIP-58M, a continual pretraining framework that integrates large-scale reasoning supervision into CLIP-style models through our two-stage strategy, which progressively integrates reasoning signals while pre-serving descriptive alignment, followed by category-structured reason-ing supervision. To support this framework, we construct two comple-mentary datasets and a benchmark: ReasonLite-42M, with open-form,visually verifiable reasoning captions; ReasonPro-16M, with category-specific reasoning supervision; and RCLIP-Bench for diagnostic evalua-tion of visually grounded reasoning. We train a family of ReasonCLIPthat improves visually grounded commonsense and compositional reason-ing while also enhancing zero-shot retrieval performance. As a drop-invisual encoder for multimodal large language models such as LLaVA-NeXT, ReasonCLIP delivers consistent gains without additional infer-ence cost, demonstrating that structured reasoning supervision enhancesthe expressive capacity of CLIP-style visual representations. All datasets,models, and training code are available at https://github.com/RISys-Lab/ReasonCLIP.

Robust 3D object detection in adverse weather conditions is challenging due to sensor limitations. Although combining complementary modalities such as LiDAR and 4D RADAR has shown promise, the sparsity of these sensors becomes apparent in adverse weather with reduced reflections, leading to objects with few or no point cloud returns. To address this limitation, camera sensors provide visual cues even when LiDAR and RADAR signals are weakened. However, cameras themselves are also vulnerable to adverse weather, where some regions become unreliable due to snow or rain occluding the camera lens. While some camerafusion methods designed for adverse weather learn to weigh image regions via confidence maps, these maps receive no direct supervision and are learned solely through the detection loss. We introduce Reliability-Aware Fusion (RAF), which explicitly supervises per-pixel reliability estimation and provides a direct learning signal for identifying and suppressing unreliable visual cues. Our framework leverages pretrained LiDAR–RADAR networks, keeping their backbones frozen while only training the added camera branch, BEV fusion encoder, and detection head. Extensive experiments on the K-Radar and VoD datasets demonstrate that integrating RAF consistently improves detection accuracy over LiDAR–RADAR baselines, achieving up to +6.5 AP and +7.4 AP gains. Code is BEV 3D available at https://github.com/parkie0517/RAF.

Real-world visual perception demands joint handling of low-level image degradation and high-level semantic understanding, yet thesetwo objectives inherently conflict: restoration seeks fine-grained texturerecovery while segmentation prioritizes semantic contrast. We proposethe nested unfolding network (NUN), a principled framework that re-solves this conflict by nesting one deep unfolding network (DUN) in-side another. NUN embeds a degradation-resistant unfolding network(DeRUN) within each stage of a segmentation-oriented unfolding network(SODUN), enabling both tasks to optimize in their own subspaces whileinteracting in a controlled manner. DeRUN handles unknown degrada-tion through proximal gradient unfolding with learnable operators thatimplicitly approximate it, while SODUN performs reversible foreground-background estimation. A bi-directional unfolding interaction mechanismuses IQA to select optimal DeRUN outputs, and a cross-stage consis-tency loss ensures robust predictions under varying restoration quality.Theoretically, under local assumptions, we show that NUN reduces di-rect parameter-level gradient conflict through disjoint parameter setsand achieves a degradation-sensitivity bound, where degradation affectssegmentation only through the inner-loop optimization and restorationapproximation errors. We instantiate NUN on concealed object segmen-tation and demonstrate consistent superiority over SOTA alternativesacross 12 benchmarks. The code is available at https://github.com/ChunmingHe/NUN.

Large Vision-Language Models (LVLMs) have shown remark-able potential across a wide array of vision-language tasks, leading totheir adoption in critical domains such as finance and healthcare. How-ever, their growing deployment also introduces significant security andprivacy risks. Malicious actors could potentially exploit these models toextract sensitive information, highlighting a critical vulnerability. Recentstudies show that LVLMs often fail to consistently refuse instructionsdesigned to compromise user privacy. While existing work on privacyprotection has made meaningful progress in preventing the leakage ofsensitive data, they are constrained by limitations in both generalizationand non-destructiveness. They often struggle to robustly handle unseenprivacy-related queries and may inadvertently degrade a model’s per-formance on standard tasks. To address these challenges, we introduceNeural Gate, a novel method for mitigating privacy risks through neuron-level model editing. Our method improves a model’s privacy safeguardsby increasing its rate of refusal for privacy-related questions, cruciallyextending this protective behavior to novel sensitive queries not encoun-tered during the editing process. Neural Gate operates by learning a fea-ture vector to identify neurons associated with privacy-related conceptswithin the model’s representation of a subject. This localization then pre-cisely guides the update of model parameters. Through comprehensiveexperiments on MiniGPT and LLaVA, we demonstrate that our methodsignificantly boosts the model’s privacy protection while preserving itsoriginal utility. The code is available at https://github.com/Xiangkui-Cao/Neural-Gate.

Visual encoding followed by token condensing has becomethe standard architectural paradigm in multi-modal large language mod-els (MLLMs). Many recent MLLMs increasingly favor global native-resolution visual encoding over slice-based methods. To investigate thistrend, we systematically compare their behavior on vision-language un-derstanding and attention patterns, revealing that global encoding en-hances overall capability but at the expense of greater computationaloverhead. To address this issue, we present NaVLM-PVC, an MLLM cen-tered upon our proposed Progressive Visual Compression (PVC) method,which can be seamlessly integrated into standard Vision Transformer(ViT) to enable efficient native-resolution encoding. The PVC approachconsists of two key modules: (i) refined patch embedding, which supportsflexible patch-size scaling for fine-grained visual modeling, (ii) windowedtoken compression, hierarchically deployed across ViT layers to progres-sively aggregate local token representations. Jointly modulated by thesetwo modules, a widely pretrained ViT can be reconfigured into an effi-cient architecture while largely preserving generality. Evaluated acrossextensive benchmarks, the transformed ViT, termed ViT-PVC, demon-strates competitive performance with MoonViT while reducing TTFT(time-to-first-token) by 2.4×, when developed within an identical MLLMarchitecture. Building upon ViT-PVC, NaVLM-PVC also achieves com-petitive performance to Qwen2-VL, while further reducing TTFT by1.9×. We will release all code and checkpoints to support future researchon efficient MLLMs.

Multimodal Large Language Models (MLLMs) have recentlydemonstrated strong performance across vision-language tasks. However,their high inference cost, arising from both the large number of inputvisual tokens and the heavy computation of the large language model(LLM), remains a key barrier to practical deployment. Recent work at-tempts to reduce the cost by adaptively optimizing individual dimen-sions, e.g., pruning redundant visual tokens or skipping LLM layers andheads. Nonetheless, prior approaches typically treat these dimensions in-dependently and overlook a fundamental coupling: the available computeresources must be dynamically allocated across all dimensions based onthe input content. To bridge the gap, we propose SmartVL, a unifiedadaptive inference framework that jointly controls vision token numberand model compute capability in response to varying input contents andcompute budgets. SmartVL introduces a vision-side token controllerthat dynamically selects informative visual tokens and an LLM-side com-pute controller that adaptively adjusts LLM computation. Importantly,these controllers are trained to coordinate with each other so that theoverall inference cost satisfies a target budget. To allow this joint schedul-ing, we connect the controllers using a shared budget encoding and lever-age a differentiable latency estimator for end-to-end training. This de-sign enables SmartVL to learn cross-stage allocation strategies thatadapt to both input complexity and runtime compute constraints. Exper-iments across multiple MLLM benchmarks demonstrate that, with jointscheduling, SmartVL consistently outperforms prior adaptive methodsand achieves superior accuracy–efficiency Pareto frontiers. Project page:https://www.schaterji.io/publications/2026/jointtokencompute.

Computer-use agents, which leverage multimodal large lan-guage models (MLLMs) to operate computers and complete tasks, haveattracted significant attention for their utility and versatility. A majorchallenge in developing these agents is collecting large-scale, high-qualitytrajectories. The standard approach generates synthetic data through aself-improving loop: an agent is placed in a verifiable environment anditeratively fine-tuned on its successful trajectories. Despite its effective-ness, this paradigm exploits only successful trajectories and discards thefailed ones, even though failures carry rich information about a model’sweaknesses. In this work, we explore a complementary failure-driven self-improvement loop, a data-centric paradigm that turns failed trajecto-ries into agent improvements. Specifically, we employ an LLM to diag-nose failure modes, propose inference-time solutions, and generate codepatches—lightly verified by humans—that upgrade the agent. We vali-date this approach with the state-of-the-art OpenCUA-72B model on theOSWorld benchmark, improving the success rate from 42.3% to 48.9%,a gain of 6.6 percentage points, without any additional training costand with only modest inference overhead. Our results demonstrate thatfailure-driven self-improvement is a viable complement to success-basedpipelines, enabling more efficient agent improvement. Code is availableat https://github.com/snow10072740/Learning_from_Failure.

Spike camera, a type of neuromorphic sensor with high-temporal resolution, shows great promise for high-speed visual tasks.Unlike traditional cameras, spike camera continuously accumulates pho-tons and fires asynchronous spike streams. Due to unique data modality,spike streams require reconstruction methods to become perceptible to thehuman eye. However, under low-light high-speed conditions, spike streamsbecome highly sparse and noisy, making faithful pixel-wise recovery in-trinsically difficult. In this work, we propose Diff-SPK, a diffusion-basedframework for perceptual reconstruction from low-light spike streams.Diff-SPK leverages generative priors to produce visually plausible re-constructions while remaining constrained by spike-derived structuralconditions. Specifically, it first employs an Enhanced Texture f rom Inter-spike Interval (ETFI) to aggregate sparse structural information fromlow-light spike streams. Then, the encoded ETFI by a suitable encoderserves as the input of ControlNet for high-speed scenes generation. Toimprove the quality of results, we introduce an ETFI-based feature fusionmodule during the generation process.

While Multi-modal Large Language Models (MLLMs) havedemonstrated superior performance on isolated tasks such as visual ques-tion answering, their reliability remains limited in real-world scenariosthat require highly compositional visual-language reasoning. A primeexample is automated exam grading, a real-world practical task thatdemands the integration of three core capabilities: detection and local-ization, text recognition, and reasoning for correctness judgment. Cur-rent benchmarks largely focus on isolated tasks and cannot fully eval-uate MLLMs’ end-to-end ability in such complex settings. To bridgethis gap, we present GradingBench, a comprehensive benchmark basedon automated exam grading in Chinese K–12 education, which system-atically evaluates MLLMs across the entire grading pipeline. Grading-Bench comprises full-page exam papers from real educational settings,containing 3,284 sub-questions annotated with reference answers. Weevaluate MLLMs across three levels: single-question, specified-question,and full-page grading. Our experiments show that localization failureis the main bottleneck and reveal limitations in integrated multi-taskexecution. Although multi-round interaction and supervised fine-tuningbring slight improvements, the limited gains reflect fundamental weak-nesses in compositional ability. GradingBench serves as a challenging,application-driven benchmark for the community, underscoring the needfor seamless perception-cognition fusion in MLLMs to unlock their po-tential in practical domains. All code and data are available at https://github.com/ERRORSEMI/GradingBench.

Visual Autoregressive modeling (VAR) has emerged as ahighly efficient alternative to diffusion-based frameworks, achieving com-parable synthesis quality. However, as this paradigm extends to Space-time Autoregressive modeling (STAR) for video generation, scaling res-olution and frame counts leads to a "token explosion" that creates amassive computational bottleneck in the final refinement stages. To ad-dress this, we propose FastSTAR, a training-free acceleration frame-work designed for high-quality video generation. Our core method, Spa-tiotemporal Token Pruning, identifies essential tokens by integratingtwo specialized terms: (1) Spatial similarity, which evaluates structuralconvergence across hierarchical scales to skip computations in regionswhere further refinement becomes redundant, and (2) Temporal similar-ity, which identifies active motion trajectories by assessing feature-levelvariations relative to the preceding clip. Combined with a Partial Up-date mechanism, FastSTAR ensures that only non-converged regions arerefined, maintaining fluid motion while bypassing redundant computa-tions. Experimental results on InfinityStar demonstrate that FastSTARachieves up to a 2.01× speedup with a PSNR of 28.29 and less than 1%performance degradation, proving a superior efficiency-quality trade-offfor STAR-based video synthesis.

Text-to-image diffusion models power everyday creative tasks,but they still reproduce the demographic biases in their training data. Oncommon prompts such as “a photo of a nurse,” “a photo of a CEO”, theyskew their outputs toward one gender, driven by the statistics of trainingdata rather than anything in the text. Existing debiasing methods showpromise in narrow settings but require retraining, batch-level control, orprompt-specific tuning, limiting their scalability. We propose EquiSteer,a training-free method that works per sample by steering cross-attention(CA) activations at inference time. For each target attribute, EquiSteerprecomputes steering vectors from contrastive prompts. Then at gener-ation time, a prompt-aware gate leaves attribute-specific prompts un-touched, while for neutral ones it clears existing attribute signals fromthe CA activations and injects a target attribute. Across SD-1.5, SD-2.1,SDXL, and SANA, EquiSteer reduces the average parity gap by up to87%, with minimal effect on image quality and text-image alignment.Code is available at https://github.com/Atmyre/EquiSteer.

Multimodal Large Language Models (MLLMs) have demon-strated remarkable success in various Remote Sensing (RS) tasks. How-ever, their ability to comprehend negation remains underexplored, limit-ing deployment in real-world applications where models must explicitlyidentify what is false or absent, e.g., emergency responders need to locatenon-flooded routes for evacuation. To comprehensively study this limi-tation, we introduce RS-Neg, the first benchmark to evaluate negationunderstanding across region-level to scene-level tasks. Specifically, we de-sign an automated data generation pipeline for RS imagery, using LLMsto synthesize diverse negation queries, and introduce a dynamic visualfocus module for verification. Our evaluation reveals that advanced RSMLLMs struggle with negation, exhibiting hallucinations and substantialperformance degradation. To close this gap, we propose NeFo, a noveltest-time adaptation method that explicitly incorporates the logical roleof negation into the model optimization. Remarkably, using about 5%unlabeled test samples, NeFo significantly improves the negation under-standing of models and shows strong generalization to unseen tasks.

Coordinating navigation and manipulation with robust per-formance is essential for embodied AI in complex indoor environments.However, as tasks extend over long horizons, existing methods oftenstruggle due to catastrophic forgetting, spatial inconsistency, and rigidexecution. To address these issues, we propose ESCAPE (EpisodicSpatial memory Coupled with an Adaptive Policy for Execution), op-erating through a tightly coupled perception-grounding-execution work-flow. For robust perception, ESCAPE features a Spatio-Temporal FusionMapping module to autoregressively construct a depth-estimation-free,persistent 3D spatial memory, and a Memory-Driven Target Groundingmodule for precise interaction mask generation. To achieve flexible ac-tion, our Adaptive Execution Policy dynamically orchestrates proactiveglobal navigation and reactive local manipulation to seize opportunis-tic targets. ESCAPE achieves state-of-the-art performance on the AL-FRED benchmark, reaching 65.09% and 60.79% success rates in test seenand unseen environments with step-by-step instructions. By reducing re-dundant exploration, our ESCAPE substantially improves path-length-weighted metrics and remains robust (61.24%/56.04%) even without de-tailed guidance for long-horizon tasks.

Precise control of 3D facial expressions from text is crucialfor virtual avatars, animation, and human–computer interaction, yet ex-isting text-to-3D methods jointly generate identity, expression, and tex-ture, making fine-grained expression control difficult. We instead for-mulate text-driven expression synthesis as a regression problem in thedisentangled parameter space of a 3D Morphable Model (3DMM). Thissetting, however, requires paired data linking detailed language to pre-cise expression parameters, which is missing from existing resources. Tofill this gap, we introduce Txt2Emote, a benchmark of diverse 3D facialexpressions with fine-grained textual annotations obtained from GPT-4o and a high-fidelity face tracker, providing both explicit descriptionsdetailing facial features and implicit descriptions referencing the situa-tional context behind the expression. Leveraging this dataset, we presentEmoteGPT, a text-to-3D expression framework based on a Multi-ModalLarge Language Model (MLLM) with a dedicated <Expr> token to se-mantically ground expression representations, which are then decodedinto 3DMM parameters. We further improves EmoteGPT by augmentingtraining with large-scale image-to-3DMM data, surpasses state-of-the-arttext-to-3D face synthesis methods on emotion recognition metrics andin perceived expressiveness. Integrated into avatar pipelines, our methodenables photorealistic and stylized 3D avatars, as well as expressive 3D-consistent 2D face synthesis from textual input.