ECCV 2026 论文列表

Grounding natural-language instructions into executable con-tinuous control remains a core challenge for quadruped vision-language-action (VLA) systems due to the gap between high-level semantic reason-ing and low-level locomotion actuation. Existing approaches often relyon implicit reasoning or purely behavioral supervision, leading to unsta-ble long-horizon grounding and limited robustness in real-world deploy-ment. To address these issues, we present MobileVLA-R1, a unifiedquadruped VLA framework that explicitly aligns hierarchical reason-ing with continuous control. Our method introduces a multi-granularitychain-of-thought (CoT) supervision scheme for embodied trajectories, in-stantiated in MobileVLA-CoT, together with a two-stage training paradigmthat combines supervised CoT alignment and reinforcement learning forreasoning-to-control consistency. We conduct comprehensive evaluationson VLN and quadruped VLA benchmarks, including VLN-CE, QUARD,and real-world Unitree Go2 deployment. MobileVLA-R1 consistently out-performs strong baselines, achieving about 5% gains on key metrics whiledemonstrating deployment-time closed-loop execution under a fixed hy-brid inference setup.

Detecting covariate shift is critical for building reliable vi-sion systems. While most prior work focuses on improving robustness toshift, explicitly detecting covariate shift remains underexplored. Existingapproaches typically rely on fully supervised training, requiring labeledexamples from both original and shifted distributions, which is often im-practical. In this paper, we show that covariate shift detection can beeffectively addressed with weaker supervision using Positive–Unlabeled(PU) learning. However, under covariate shift, in-distribution and shifteddata overlap significantly, making classical PU methods unstable and sen-sitive to noise. To overcome this challenge, we introduce Spectral PUNeighborhood Annotation (S-PUNA), a geometry-aware frame-work that progressively discovers shifted data by leveraging the localmanifold structure of visual features. Extensive experiments show thatS-PUNA achieves state-of-the-art performance in PU settings and re-markably matches the performance of fully supervised methods. More-over, our approach transfers robustly across different types of shifts,demonstrating strong generalization capabilities. Code is available athttps://github.com/fira7s/S-PUNA.

Answering questions accurately and efficiently in embodiedscenarios presents significant challenges due to limited computationaland memory resources for Vision Language Model (VLM) inference. Ex-isting methods adopt visual search key frame retrieval method to se-lect critical question-related key frames for VLM input. However, visualsearch methods are inefficient because they require visual search amongthousands of video frames for each individual user query. In this work, wepropose a memory tree guided key frame selection paradigm for efficient3D question answering in embodied scenarios. Our method leveragesa compact and reusable 3D scene representation, termed MemTree3D,which supports real-time online construction leveraging camera 6-DoFposes. MemTree3D captures multi-level 3D scene information, enabling aLarge Language Model to efficiently query and retrieve question-relevantkey frames through our scoring-based frame selection without repro-cessing the entire video stream. On OpenEQA, our method improvesthe LLM-Match of GPT-4o by 17.4%, LLaVA-OneVision-7B by 5.8%,outperforms existing visual search methods. Our code is available athttps://github.com/hsiangwei0903/MemTree3D.

Reasoning-induced vision-language models (VLMs) advanceimage quality assessment (IQA) with textual reasoning, yet their scalarscores often lack sensitivity and collapse to a few values, so-called discretecollapse. We introduce ME-IQA, a plug-and-play, test-time memory-enhanced re-ranking framework. It (i) builds a memory bank and re-trieves semantically and perceptually aligned neighbors using reason-ing summaries, (ii) reframes the VLM as a probabilistic comparator toobtain pairwise preference probabilities and fuse this ordinal evidencewith the initial score under Thurstone’s Case V model, and (iii) per-forms gated reflection and consolidates memory to improve future deci-sions. This yields denser, distortion-sensitive predictions and mitigatesdiscrete collapse. Experiments across multiple IQA benchmarks showconsistent gains over strong reasoning-induced VLM baselines, existingnon-reasoning IQA methods, and test-time scaling alternatives.

High-quality world-scale 3D data is scarce, making completeand generalizable 3D world generation difficult. Existing approaches typi-cally rely on either generated images/videos or domain-specific 3D worlddatasets, resulting in worlds that are often incomplete or restricted tonarrow domains. In this paper, we present Map2World, a map-conditionedframework that converts semantic layouts into complete and generalizablelarge-scale 3D worlds. Given a segment map with per-region text prompts,Map2World uses the map as a controllable world-level interface and lever-ages TRELLIS, a powerful pretrained 3D asset generator, as a general3D prior. To scale asset-level priors to world-level synthesis, Map2Worldenables arbitrary spatial expansion via MultiDiffusion, controls globalscale through initial noise optimization, and enhances local geometry andappearance with a dedicated enhancer. Experiments demonstrate thatMap2World generates complete, layout-controllable, scale-consistent, anddetailed 3D worlds across diverse layouts and semantics.

Self-supervised RGB-T tracking seeks to learn robust cross-modal representations without dense manual annotations. A commonsupervision strategy relies on forward-backward tracking loops, but inpractice, it is highly vulnerable to error accumulation and modality driftcaused by the inconsistent reliability of visible and thermal cues. To ad-dress this issue, we propose LGCTrack, a self-supervised RGB-T trackerunderpinned by a local-to-global coordination strategy within a closed-loop verification paradigm. At the local level, a hierarchical coordina-tion module performs layer-wise, reliability-aware token interaction toenable adaptive cross-modal fusion. At the global level, a turning-phasecoordination module refines the target representation during the criticalforward-to-backward transition, anchoring target semantics and improv-ing backward tracking. By jointly coordinating cross-modal interactionat two granularities, LGCTrack intrinsically filters pseudo-label noise andmitigates drift. Experiments on standard RGB-T tracking benchmarksshow that LGCTrack consistently outperforms existing self-supervisedmethods and substantially narrows the gap to fully supervised trackers.Our code is available at https://github.com/zyyyya/LGCTrack.

Current multimodal latent reasoning often relies on externalsupervision (e.g., auxiliary images), ignoring intrinsic visual attentiondynamics. In this work, we identify a critical Perception Gap in distil-lation: student models frequently mimic a teacher’s textual output whileattending to fundamentally divergent visual regions, effectively relyingon language priors rather than grounded perception. To bridge this, wepropose LaViT, a framework that aligns latent visual thoughts ratherthan static embeddings. LaViT compels the student to autoregressivelyreconstruct the teacher’s visual semantics and attention trajectories priorto text generation, employing a curriculum sensory gating mechanism toprevent shortcut learning. Extensive experiments show that LaViT sig-nificantly enhances visual grounding, achieving up to +16.9% gains oncomplex reasoning tasks and enabling a compact 3B model to rival largermodels on selected evaluated benchmarks.

In this paper, we present JoVA, a streamlined frameworkthat unifies joint video-audio generation and editing. While existingmethods often rely on fragmented, task-specific architectures or complexfusion mechanisms, JoVA employs native joint representation learningfor direct video, audio, and text interaction in a dual-branch architec-ture. This design eliminates redundant alignment modules and effectivelyunifies diverse multimodal tasks within a single model. Furthermore, weutilize channel-wise conditioning for flexible image and video reference toavoid massive token expansion, alongside a mouth-area loss to enhancelip alignment. To fully empower and systematically evaluate this frame-work, we construct a comprehensive training corpus encompassing video-audio generation and editing datasets, and introduce unified benchmarkstailored for these multimodal tasks. Extensive experiments demonstratethat JoVA achieves state-of-the-art performance across benchmarks, es-tablishing it as an extensible framework for versatile content creation.Project page: https://visual-ai.github.io/jova

Multi-task learning (MTL) has been widely adopted for itsability to simultaneously learn multiple tasks. While existing gradientmanipulation methods often yield more balanced solutions than simplescalarization-based approaches, they typically incur a significant compu-tational overhead of O(K) in both time and memory, where K is thenumber of tasks. In this paper, we propose LDC-MTL, a simple andscalable loss discrepancy control approach for MTL, formulated froma bilevel optimization perspective. Our method incorporates two keycomponents: (i) a bilevel formulation for fine-grained loss discrepancycontrol, and (ii) a scalable first-order bilevel algorithm that requiresonly O(1) time and memory. Theoretically, we prove that LDC-MTLguarantees convergence not only to a stationary point of the bilevelproblem with loss discrepancy control but also to an ϵ-accurate Paretostationary point for all K loss functions under mild conditions. Extensiveexperiments on diverse multi-task datasets demonstrate the superior per-formance of LDC-MTL in both accuracy and efficiency. Code is availableat https://github.com/OptMN-Lab/LDC-MTL.

Robust object detection under adverse visual conditions re-mains a long-standing challenge for multi-modal perception systems. Ex-isting fusion-based methods typically require both RGB and infrared(IR) inputs, and treat them equally during both training and inference,which compromises their robustness when the RGB modality becomesunreliable or unavailable. In this case, we propose InfraNet, an IR-centric quality-aware framework that regulates RGB guidance duringtraining and supports flexible RGB–IR or IR-only deployment. InfraNetemploys an asymmetric architecture where the primary IR pathway ex-tracts multi-scale infrared features for predictions, while the auxiliaryRGB pathway provides reliability-controlled supervisory signals. Thecore of InfraNet is QualGate, a quality-aware fusion module that learnsa task-oriented control signal to suppress unreliable RGB guidance andcompensate IR features during cross-modal training. Built upon InfraNet,we design two architectural variants: a lightweight IR-only architectureInfraNet-IR and an RGB–IR architecture InfraNet-RGB-IR. Our methodis evaluated through extensive experiments on four benchmark datasets(LLVIP, FLIR-Aligned, M3 FD, and DroneVehicle), showing strong orcompetitive accuracy in challenging low-light and adverse weather con-ditions. Notably, InfraNet maintains high efficiency in IR-only inference,making it both accurate and computationally efficient.

Achieving human-like reasoning in Vision-Language Models(VLMs) remains a long-standing challenge. Recent approaches leverageChain-of-Thought (CoT) rationales generated by human annotators orproprietary models to improve reasoning, which is costly and difficultto scale. Self-training offers a promising alternative by using model’sown outputs as supervision. However, existing methods often suffer fromvisual hallucinations - where rationales describe non-existent visual con-tent, and language shortcuts - where predictions rely on textual priorsrather than true visual grounding, as rationales are typically filtered onlyby answer correctness without verifying visual perception. To addressthis limitation, we propose a perception-verified self-training frameworkthat enforces visually grounded reasoning. First, our method employs aCoT template (caption-reasoning-conclusion) that disentangles percep-tion from reasoning, enabling independent verification of visual under-standing. To compensate for the absence of ground-truth captions, wepropose PerceptEval, an unsupervised method that evaluates captionquality based on its alignment with visual and textual elements presentin the image. Using caption verification together with answer correct-ness, we partition the data into three subsets: easy (correct caption andconclusion), medium (correct caption but incorrect conclusion), and hard(incorrect caption). Building on this partitioning, we design a two-stagecurriculum learning strategy. In Stage 1, the model is trained on easyexamples and subsequently in Stage 2, medium samples are incorporatedthrough a caption-guided reasoning enhancement procedure thatregenerates reasoning conditioned on verified captions. Only regeneratedsamples with the correct conclusions are retained. This ensures trainingin both stages is done exclusively on perceptually grounded reasoning,reducing hallucinations and language shortcuts. Extensive experimentsacross diverse domains and models demonstrate improvements of up to16% over standard self-training baselines. This demonstrates that theproposed framework is a scalable and cost-effective solution to advanc-ing multimodal reasoning without manually annotated CoT rationales.

Inertial measurement units (IMUs) are now standard in mostconsumer devices, such as smartphones, drones, and extended reality(XR) headsets. By fusing visual and inertial data, localization systemsgain significantly in speed and robustness compared to vision-only orIMU-only approaches. However, traditional pose estimation methods failto utilize the local geometric information embedded in feature descriptorslike SIFT. Recent work has proved the advantages of leveraging this in-formation for relative and absolute pose estimation, but its application topartially calibrated absolute pose estimation remains unexplored. In thispaper, we derive novel constraints for joint estimation of absolute poseand focal length, making use of a gravity vector obtained from IMU dataand the feature-induced local geometry, which we use to construct twoefficient solvers: UP1PfAC, that operates given a single affine correspon-dence and UP2PfORI, which requires two orientation-covariant features.Unlike traditional, semi-calibrated absolute pose methods requiring fourpoint correspondences, our solvers benefit from fewer samples and lowercomputational cost, simplifying robust estimation in modern RANSAC-like frameworks. We evaluate the proposed solvers against the state-of-the-art on large-scale public datasets and demonstrate that our methodachieves fast and accurate localization and focal length estimation.

Depth completion aims to predict a dense depth map froma color image with sparse depth measurements. Although deep learningmethods have achieved state-of-the-art (SOTA), effectively handling thesparse and irregular nature of input depth data in deep networks remainsa significant challenge, often limiting performance, especially under highsparsity. To overcome this limitation, we introduce the Gaussian BeliefPropagation Network (GBPN), a novel hybrid framework synergisticallyintegrating deep learning with probabilistic graphical models for end-to-end depth completion. Specifically, a scene-specific Markov Random Field(MRF) is dynamically constructed by the Graphical Model ConstructionNetwork (GMCN), and then inferred via Gaussian Belief Propagation(GBP) to yield the dense depth distribution. Crucially, the GMCN learnsto construct not only the data-dependent potentials of MRF but also itsstructure by predicting adaptive non-local edges, enabling the capture ofcomplex, long-range spatial dependencies. Furthermore, we enhance GBPwith a serial & parallel message passing scheme, designed for effectiveinformation propagation, particularly from sparse measurements. Exten-sive experiments demonstrate that GBPN achieves SOTA performanceon the NYUv2 and KITTI benchmarks. Evaluations across varying spar-sity levels, sparsity patterns, and datasets highlight GBPN’s superiorperformance, notable robustness, and generalizable capability.

We present FLEG, a feed-forward network that reconstructslanguage-embedded 3D Gaussians from multi-view images. Previous feed-forward language-embedded Gaussian reconstruction methods are re-stricted to a fixed number of input views and typically attach a language-aligned semantic embedding to each Gaussian, resulting in impracticalinput settings and semantic redundancy. In contrast, we introduce ageometric-semantic dual-branch distillation framework that supports avariable number of uncalibrated and unposed multi-view images as input.We also propose a novel-view-based distillation strategy during trainingthat mitigates overfitting to input views. In addition, we observe that se-mantic representations are significantly sparser than geometric ones, andper-Gaussian language embedding is unnecessary. To exploit this spar-sity, we design a decoupled language embedding strategy that representslanguage information with a sparse set of semantic Gaussians, ratherthan attaching embeddings to every Gaussian. Compared with densepixel-aligned per-Gaussian embedding schemes, our method uses only5% of the language embeddings while maintaining comparable semanticfidelity, effectively reducing storage costs. Extensive experiments demon-strate that FLEG outperforms state-of-the-art feed-forward reconstruc-tion and language-embedded Gaussian methods in both reconstructionquality and language-aligned semantic representation.

Class-incremental learning (CIL) aims to continually acquirenew classes while preserving previously learned knowledge, posing thewell-known stability–plasticity dilemma. With the emergence of largepre-trained models, parameter-efficient fine-tuning methods such as LoRAhave become a practical paradigm for rehearsal-free CIL by restrictingadaptation to low-rank weight updates. Recent LoRA-based approachesoften mitigate cross-task interference by enforcing orthogonality betweentask updates. However, under strict rank budgets and long task se-quences, uniformly protecting previously used directions can becomeoverly conservative, progressively shrinking the effective update spaceand limiting plasticity. To address this limitation, we propose Varia-tional Direction-aware LoRA (VD-LoRA), a Bayesian framework thatenables adaptive consolidation within the low-rank update space. VD-LoRA maintains a recursive variational posterior over LoRA directionsto estimate direction-wise uncertainty, using posterior precision to adap-tively control protection strength. Based on this uncertainty signal, wefurther design a direction selection mechanism that favors updates thatare both task-relevant and safe to adapt. This enables selective reuseof low-risk directions while preserving critical ones, maintaining a task-aligned update subspace under fixed rank constraints. Extensive experi-ments on multiple rehearsal-free CIL benchmarks demonstrate that VD-LoRA consistently outperforms existing LoRA-based methods while im-proving long-horizon stability and plasticity. Our code is available at

With the emergence of Vision-Language Models (VLMs),cloud-based VLM inference services have raised critical privacy concernsregarding the potential leakage of sensitive visual prompts. However, ex-isting defenses compromise either efficiency or model utility to achieveprivacy. Recently, the covariant obfuscation paradigm has shown greatpotential for private large language model inference, but it is incom-patible with cross-modal architectures and tasks. In this paper, we pro-pose COVERT, the first practical privacy-preserving VLM inferenceframework, which explicitly adapts covariant obfuscation for multimodalarchitectures. Specifically, it integrates exact architectural reparameter-ization to robustly obfuscate visual features against naive inversion at-tacks, alongside utility-aware parameter tuning tailored to neutralizehidden-state inversion attacks, thereby achieving a favorable privacy-utility-efficiency trade-off. Extensive evaluations show that COVERT ef-fectively shields sensitive visual attributes against hidden-state inversionattacks, while keeping the average accuracy degradation below 3% acrossmajor benchmarks and sustaining high inference throughput with a neg-ligible overhead of less than 6%.

4D Gaussian Splatting (4DGS) can render dynamic scenesphotorealistically. However, with limited viewpoint coverage, some spa-tiotemporal regions remain sparsely observed, leading to artifacts, partic-ularly in scenes with large motion. Existing approaches leveraging gener-ative models rely on heuristic virtual-viewpoint selection before refiningrendered views. As a result, they cannot actively explore such sparselyobserved regions. To address this issue, we propose a pipeline that ac-tively selects spatiotemporal virtual viewpoints to improve 4DGS recon-struction. Our method selects virtual viewpoints for generative enhance-ment based on the rendering sensitivity and motion-aware observationdensity of 4D Gaussians, prioritizing views that alleviate observationsparsity. In the refined images, we filter out regions that conflict withcaptured observations or are likely to contain generative artifacts andthen fine-tune 4DGS using only the reliable regions. We evaluate ourmethod on multi-view video benchmarks using new train/test splits de-signed to induce observation gaps. Results show consistent improvementsover prior viewpoint selection strategies and fine-tuning methods in bothqualitative and quantitative evaluations, while reducing artifacts. 1

Existing Referring Video Object Segmentation tasks focus on referring expressions describing events, actions or appearances of relevant objects within the observed frames, lacking evaluation in scenarios that require pre-decisive spatio-temporal reasoning, thereby limiting their applicability. To address this, we propose Foresight Expression Video Object Segmentation, a task that queries future events in upcoming video segments and requires masks of the objects in the observed frames as visual answers. For example, in ego-centric scenes, the question “What tool will be used?” demands reasoning over spatio-temporal cues to predict the masks of the next tool to be used, which helps with the understanding of future actions and decisions. To support this task, we introduce FeVOS, a dataset with 968 video clips, 14,525 foresight expressions, and 2,904 chain-of-thought annotations to provide explicit and interpretable reasoning steps. We further develop FeVOS-R1, an MLLM-based model trained on our dataset via a two-stage pipeline of supervised fine-tuning and reinforcement learning. FeVOS-R1 not only achieves state-of-the-art performance on FeVOS, but also demonstrates strong generalization to existing RVOS benchmarks. We hope this work can inspire more research on predictive reasoning in video perception.

In recent years, face super-resolution (FSR) methods haveachieved remarkable progress, generally maintaining high image fidelityand identity (ID) consistency under standard settings. However, underextreme degradation scenarios (e.g., scale ¿ > 8×), critical facial at-tributes and ID information are often severely lost in the input image,making it difficult for conventional models to reconstruct realistic andID-consistent faces. Existing methods tend to generate hallucinatedfaces under such conditions, producing restored images that lack faith-ful ID constraints. To address this challenge, we propose a novel FSRmethod with Identity Decoupling and Fitting (IDFSR), designed to en-hance ID restoration under large scaling factors while mitigating hallu-cination effects. Our approach involves three key designs: (1) maskingthe facial region in the low-resolution (LR) image to eliminate unre-liable ID cues; (2) warping a reference image to align with the LRinput, providing style guidance; and (3) leveraging ID embeddingsextracted from ground-truth (GT) images for fine-grained ID modelingand personalized adaptation. We first pretrain a diffusion-based modelto explicitly decouple style and ID by forcing it to reconstruct maskedLR facial regions using both style and ID embeddings. Subsequently, wefreeze most network parameters and perform lightweight fine-tuning ofthe ID embeddings using a small set of target ID images. This embeddingencodes fine-grained facial attributes and precise ID information, signifi-cantly improving both ID consistency and perceptual quality. Extensivequantitative evaluations and visual comparisons demonstrate that theproposed IDFSR substantially outperforms existing approaches underextreme degradation, particularly in terms of ID consistency.