paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

ECCV 2026

Symbiotic-MoE: Unlocking the Synergy between Generation and Understanding

Xiangyue Liu, Zijian Zhang, Miles Yang, Zhao Zhong, Liefeng Bo, Ping Tan

Empowering Large Multimodal Models (LMMs) with imagegeneration often leads to catastrophic forgetting in understanding tasksdue to severe gradient conflicts. While existing paradigms like Mixture-of-Transformers (MoT) mitigate this conflict through structural isola-tion, they fundamentally sever cross-modal synergy and suffer from ca-pacity fragmentation. In this work, we present Symbiotic-MoE, a unifiedpre-training framework that resolves task interference within a nativemultimodal Mixture-of-Experts (MoE) Transformers architecture withzero-parameter overhead. We first identify that standard MoE tuningleads to routing collapse, where generative gradients dominate expertutilization. To address this, we introduce Modality-Aware Expert Dis-entanglement, which partitions experts into task-specific groups whileutilizing shared experts as a multimodal semantic bridge. Crucially, thisdesign allows shared experts to absorb fine-grained visual semantics fromgenerative tasks to enrich textual representations. To optimize this, wepropose a Progressive Training Strategy featuring differential learningrates and early-stage gradient shielding. This mechanism not only shieldspre-trained knowledge from early volatility but eventually transformsgenerative signals into constructive feedback for understanding. Exten-sive experiments demonstrate that Symbiotic-MoE achieves rapid gener-ative convergence while unlocking cross-modal synergy, boosting inherentunderstanding with remarkable gains on MMLU and OCRBench.

ECCV 2026

SuperFlex: Deformable Superquadrics for Point Cloud Decomposition

Gabriel Tavernini, Elisabetta Fedele, Tiago Novello, Leonidas Guibas, Marc Pollefeys, Francis Engelmann

Superquadrics have proven to provide a compact, geometri-cally meaningful representation for 3D objects. However, existing meth-ods suffer from limited reconstruction accuracy, are restricted to rigidprimitives, and lack robustness to partial point clouds. In this work,we present SuperFlex, an enhanced framework that expands the expres-sive power and applicability of superquadric decompositions. First, weintroduce a novel loss formulation which significantly improves recon-struction accuracy. Second, we include bending and tapering deforma-tions, enabling high-fidelity representation of curved and asymmetricgeometries. Finally, we leverage these high-quality decompositions assupervision to train a model that is robust to partial real-world pointclouds. Experiments demonstrate substantial improvements in recon-struction accuracy over both optimization- and learning-based baselineswhile maintaining a highly compact primitive representation. Projectpage: https://superflex3d.github.io.Point Cloud

ECCV 2026

Spectral-Aware Analytic Class-Incremental Learning for Long-Tailed Distributions

Quyen Tran, Ngoc-Hai Nguyen, Minh-Quan Dao, Zhuowei Li, Nam Hai, Trung Le, Dimitris N. Metaxas

Analytic Continual Learning (ACL) offers a computationallyefficient alternative to gradient-based approaches. Recent ACL methodsare based on Recursive Least Squares (RLS) and have achieved the state-of-the-art results compared to other alternatives. However, they faltersignificantly in Class-Incremental Learning scenarios characterized byLong-Tailed distributions. While the ill-conditioning of the autocorrela-tion (Gram) matrix is a known limitation of RLS, we demonstrate thatclass imbalance exacerbates this issue into a distinct spectral pathology:"tail" classes suffer from severe spectral collapse, rendering their sub-spaces numerically indistinguishable from noise. Standard Ridge Regres-sion (L2 ) fails to address this effectively as it applies isotropic regular-ization - a uniform penalty that is insufficient to stabilize the tail with-out over-shrinking the head. To address this, we propose Geometry-Spectral Rectification (GSR), a theoretically grounded frameworkthat treats long-tailed learning as a spectral regularization problem. Un-like standard isotropic regularization (Ridge) which uniformly penalizesall eigenvalues, GSR acts as an anisotropic spectral filter, selectively in-flating the collapsed eigenvalues of tail classes. We construct a structured,data-dependent spectral perturbation matrix ∆ that selectively inflatescollapsed tail eigen-directions of the Gram matrix. Theoretical analysisproves that GSR guarantees an improved stable rank for the Gram ma-trix, ensuring numerical stability. Extensive experiments show that GSRestablishes a new state-of-the-art for analytic CIL, offering a superiortrade-off between computational efficiency and robust generalization inlong-tailed settings.

ECCV 2026

Silhouette-based Gait Foundation Model

Dingqiang Ye, Chao Fan, Kartik Narayan, Bingzhe Wu, Chengwen Luo, Jianqiang Li, Vishal Patel

Gait patterns play a critical role in human identification andhealthcare analytics, yet current progress remains constrained by small,narrowly designed models that fail to scale or generalize. Building a uni-fied gait foundation model requires addressing two longstanding barriers:(a) Scalability – Why have gait models historically failed to follow em-pirical scaling trends? (b) Generalization – Can one model serve thediverse gait tasks that have traditionally been studied in isolation? Weintroduce FoundationGait, the first scalable, self-supervised pretrain-ing framework for vision-based gait understanding. Its largest versionhas nearly 0.13 billion parameters and is pretrained on 12 public gaitdatasets comprising over 2 million walking sequences. Extensive exper-iments demonstrate that FoundationGait, with or without fine-tuning,performs robustly across a wide spectrum of gait datasets, conditions,tasks (e.g., human identification, scoliosis screening, depression predic-tion, and attribute estimation), and even input modality. Notably, itachieves 48.0% self-supervised rank-1 accuracy on the challenging in-the-wild Gait3D dataset (1,000 test subjects) and 64.5% on the largestin-the-lab OU-MVLP dataset (5,000+ test subjects), setting a new mile-stone in robust gait recognition. These results establish FoundationGaitas a strong and versatile foundation for future gait research. All codeand models: https://github.com/ShiqiYu/OpenGait.

ECCV 2026

SegVGGT: Joint 3D Reconstruction and Instance Segmentation from Multi-View Images

Jinyuan Qu, Hongyang Li, Lei Zhang

3D instance segmentation methods typically rely on high-quality point clouds or posed RGB-D scans, requiring complex multi-stage processing pipelines, and are highly sensitive to reconstructionnoise. While recent feed-forward transformers have revolutionized multi-view 3D reconstruction, they remain decoupled from high-level semanticunderstanding. In this work, we present SegVGGT, a unified end-to-endframework that simultaneously performs feed-forward 3D reconstructionand instance segmentation directly from multi-view RGB images. Byintroducing object queries that interact with multi-level geometric fea-tures, our method deeply integrates instance identification into the visualgeometry grounded transformer. To address the severe attention disper-sion problem caused by the massive number of global image tokens, wepropose the Frame-level Attention Distribution Alignment (FADA) strat-egy. FADA explicitly guides object queries to attend to instance-relevantframes during training, providing structured supervision without extrainference overhead. Extensive experiments demonstrate that SegVGGTachieves the state-of-the-art performance on ScanNetv2 and ScanNet200,outperforming both recent joint models and RGB-D-based approaches,while showing favorable generalization to ScanNet++. The code is avail-able at https://github.com/IDEA-Research/SegVGGT.

ECCV 2026

Robustness Emerges Early in Training Dynamics, but Is Not Preserved

Jiangang Yang, Wenhui Shi, Lu Hu, Jing Xing, Jian Liu

Robustness to natural corruptions remains a fundamentalchallenge for deep neural networks. In this paper, we identify a robustnessfading phenomenon where shallow layers spontaneously develop robustrepresentations and flat loss landscapes in early training, yet these prop-erties are not preserved during standard convergence. To address this,we propose a framework that performs strategic interventions on train-ing dynamics to stabilize the empirically identified early-emergent robustpriors. Our approach includes two parameter-free strategies: Early-PhaseStabilization (EPS) and Asymmetric Weight Reversion (AWR), whichstabilize or recover robust shallow configurations without modifying themodel architecture or introducing learnable parameters. Extensive exper-iments demonstrate the efficacy of our framework across various bench-marks and architectures, yielding significant gains in downstream trans-fer, dynamic adaptation, and diverse computer vision applications.

ECCV 2026

REGLUE Your Latents with Global and Local Semantics for Entangled Diffusion

Giorgos Petsangourakis, Christos Sgouropoulos, Bill Psomas, Theodoros Giannakopoulos, Giorgos Sfikas, Ioannis Kakogeorgiou

Latent diffusion models (LDMs) achieve state-of-the-art im-age synthesis, yet their reconstruction-style denoising objective providesonly indirect semantic supervision: high-level semantics emerge slowly,requiring longer training and limiting sample quality. Recent works injectsemantics from Vision Foundation Models (VFMs) either externally viarepresentation alignment or internally by jointly modeling only a nar-row slice of VFM features inside the diffusion process, under-utilizingthe rich, nonlinear, multi-layer spatial semantics available. We intro-duce REGLUE (Representation Entanglement with Global-Local UnifiedEncoding), a unified latent diffusion framework that jointly models (i)VAE image latents, (ii) compact local (patch-level) VFM semantics, and(iii) a global (image-level) [CLS] token within a single SiT backbone.A lightweight convolutional semantic compressor nonlinearly aggregatesmulti-layer VFM features into a low-dimensional, spatially structuredrepresentation, which is entangled with the VAE latents in the diffusionprocess. An external alignment loss further regularizes internal repre-sentations toward frozen VFM targets. On ImageNet 256×256, REGLUEconsistently improves FID and accelerates convergence over SiT-B/2 andSiT-XL/2 baselines, as well as over REPA, ReDi, and REG. Extensive exper-iments show that (a) spatial VFM semantics are crucial, (b) non-linearcompression is key to unlocking their full benefit, and (c) global tokensand external alignment act as complementary, lightweight enhancementswithin our global–local–latent joint modeling framework. The code isavailable at https://github.com/giorgospets/reglue.

ECCV 2026

Reflection-aware generative novel view synthesis

Geonu Kim, Shin Dong-Yeon, Tae-Hyun Oh

We propose Ref-GeNVS, a training-free, reflection-awaremethod for generative novel view synthesis (NVS) in mirror scenes. Ex-isting multi-view diffusion models often fail to recognize the mirror in thescene and cannot exploit reflected content for scene generation. To fix thisissue without additional training, our key idea is to treat a mirror imageas two complementary views. From input images, we estimate the mirrorplane and reflect camera poses to form virtual views. Based on this vir-tual view setup, we propose a two-stage generation method consisting ofMirror-gated attention and Reflection injection, which enables reflection-consistent NVS by explicitly leveraging reflection relationships in a multi-view diffusion model. Ref-GeNVS inherits the strong generalizability ofthe multi-view diffusion backbone, while it does not require finetuning.On synthetic and real scenes including mirrors, Ref-GeNVS outperformsrecent generative NVS methods by generating reflection-consistent andcontextually coherent novel views, revealing scene structure visible onlythrough mirrors. Project page: https://kim-geonu.github.io/Ref-GeNVS/

ECCV 2026

Reflect-R1: Evidence-Driven Reflection for Self-Correction in Long Video Understanding

Shuimu Chen, Yuteng Chen, Yuanshen Guan, Zebang Cheng, Zeyu Zhang, Shengqian Qin, Bin Xia, Jiaran Li, Wenming Yang, Fei Ma

Current multimodal re_x001D_ection mechanisms for long video un-derstanding predominantly rely on closed-loop self-re_x001D_ection within in-ternal parameters. Lacking objective external evidence, models are fre-quently trapped in blind con_x001C_dence and often fail to correct errors.Furthermore, applying reinforcement learning to multi-stage re_x001D_ectionpipelines introduces severe policy coupling, which is exacerbated by acritical scarcity of dedicated training data. To address these limitations,this work proposes Re_x001D_ect-R1, the _x001C_rst Evidence-Driven self-correctionframework for long video understanding. The framework constructs athree-stage pipeline consisting of intuition, veri_x001C_cation, and arbitration.By dynamically retrieving objective visual evidence to verify initial in-tuitions and autonomously executing multiple temporal searches to re-solve con_x001D_icts, it completely breaks the hallucination loop. To over-come policy coupling, we design a stage-decoupled reinforcement learn-ing algorithm named SD-GRPO that independently computes advantagefunctions across di_x001B_erent reasoning stages. Concurrently, we constructa dataset of 120K samples to bridge the training data gap. Extensiveexperiments on benchmarks such as VideoMME and LongVideoBenchdemonstrate that Re_x001D_ect-R1 achieves state-of-the-art performance. Ourmethod signi_x001C_cantly improves the genuine recti_x001C_cation rate and enablesauthentic self-correction strictly grounded in objective evidence.

ECCV 2026

Raw-JPEG Adapter: Efficient Raw Image Compression with JPEG

Mahmoud Afifi, Ran Zhang, Michael S. Brown

Digital cameras digitize scene light into linear raw representations,which the image signal processor (ISP) converts into display-ready outputs. Whileraw data preserves full sensor information—valuable for editing and vision tasks—formats such as Digital Negative (DNG) require large storage, making them im-practical in constrained scenarios. In contrast, JPEG is a widely supported format,offering high compression efficiency and broad compatibility, but it is not well-suited for raw storage. This paper presents Raw-JPEG Adapter, a lightweight,learnable, and invertible pre-processing pipeline that adapts raw images for stan-dard JPEG compression. Our method applies spatial and optional frequency-domain transforms, with compact parameters stored in the JPEG comment field,enabling accurate raw reconstruction. Experiments across multiple datasets showthat our method achieves higher fidelity than direct JPEG storage, supports othercodecs, and provides a favorable trade-off between compression ratio and recon-struction accuracy.

ECCV 2026

Pro-Pose: Unpaired Full-Body Portrait Synthesis via Canonical UV Maps

Sandeep Mishra, Yasamin Jafarian, Andreas Lugmayr, Yingwei Li, Varsha Ramakrishnan, Srivatsan Varadharajan, Alan Bovik, Ira Kemelmacher-Shlizerman

Photographs of people taken by professional photographerstypically present the person in beautiful lighting, with an interestingpose, and flattering quality. This is unlike common photos people takeof themselves in uncontrolled conditions. In this paper, we explore howto canonicalize a person’s "in-the-wild" photograph into a controllable,high-fidelity avatar—reposed in a simple environment with standard-ized minimal clothing. A key challenge is preserving the person’s uniquewhole-body identity, facial features, and body shape while stripping awaythe complex occlusions of their original garments. While a large paireddataset of the same person in varied clothing and poses would simplifythis, such data does not exist. To that end, we propose two key insights:1) Our method transforms the input photo into a canonical full-body UVspace, which we couple with a novel reposing methodology to model oc-clusions and synthesize novel views. Operating in UV space allows us todecouple pose from appearance and leverage massive unpaired datasets.2) We personalize the output photo via multi-image finetuning to ensurerobust identity preservation under extreme pose changes. Our approachyields high-quality, reposed portraits that achieve strong quantitativeperformance on real-world imagery, providing an ideal, clean biometriccanvas that significantly improves the fidelity of downstream applicationslike Virtual Try-On (VTO).

ECCV 2026

Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigation

Hong Chen, Daqi Liu, Zehan Zhang, Haiguang Wang, Tianhao Lu, Longfei Yan, Haiyang Sun, Fangzhen Li, Hongwei Xie, Bing Wang, Guang Chen, Hangjun Ye, Yihua Tan

Existing world model-based planners for visual navigationtypically follow a verification-centric paradigm, decoupling goal intentfrom trajectory synthesis. This approach suffers from candidate depen-dence, heavy computational overhead, and inconsistencies between sam-pled actions and predicted visuals. To address these issues, we proposeSWAM (Spatial-perceiving World Action Model), a task-centric jointobservation-action generation framework. Given start and goal RGB ob-servations, SWAM performs single-pass inference to simultaneously gen-erate intermediate RGB-D sequences and corresponding action trajecto-ries, promoting goal-consistent trajectory generation and improved spa-tial feasibility. SWAM leverages depth pseudo-labels during training tointernalize spatial priors, but it requires only monocular RGB input atinference time by using depth estimation model. We further introducea visual-guided action refinement module and a trajectory-scale regular-ization loss to enforce fine-grained alignment between motion and visualcues while stabilizing predictions across varying distances. Extensive ex-periments show that SWAM significantly outperforms state-of-the-arttwo-stage planners in success rate, trajectory accuracy, and inference ef-ficiency, while demonstrating robust zero-shot generalization to unseenenvironments.

ECCV 2026

Personalized Reward Modeling for Text-to-Image Generation

Jeongeun Lee, Ryang Heo, Dongha Lee

Recent text-to-image (T2I) models generate semantically co-herent images from textual prompts, yet evaluating how well they alignwith individual user preferences remains an open challenge. Conventionalevaluation methods, general reward functions or similarity-based metrics,fail to capture the diversity and complexity of personal visual tastes. Wepresent PIGReward, a personalized reward modeling framework that dy-namically generates user-conditioned evaluation dimensions and assessesimages through CoT reasoning. To address the scarcity of user data, PI-GReward adopts a self-bootstrapping strategy that reasons over limitedreference data to construct rich user contexts, enabling personalizationwithout user-speci_x001C_c training. Beyond evaluation, PIGReward providespersonalized feedback that drives user-speci_x001C_c prompt optimization, im-proving alignment between generated images and individual intent. Ex-tensive experiments demonstrate that PIGReward outperforms existingmethods in both accuracy and interpretability, establishing a scalable,reasoning-based foundation for personalized T2I evaluation and opti-mization. Project page: https://jeongeunnn-e/PIGReward/

ECCV 2026

Guiding the Blind: Generalizing GUI Agents to Unseen Websites via Multimodal Tutorials

Xinwei Long, Kai Tian, Peng Xu, Weibo Gao, Yihua Shao, Guoli Jia, Haozhe Geng, Sa Yang, Jingxuan Li, Huayong Hu, Kaiyan Zhang, Jiaqi Wang, Bowen Zhou

Despite the remarkable progress of Large Multimodal Models(LMMs), deploying autonomous agents to navigate web Graphical UserInterfaces (GUIs) remains a significant challenge. Most existing agentsare “blind” when encountering unfamiliar websites, as they rely heav-ily on patterns memorized during in-domain training, which inevitablyfails in the open-world web. To bridge this generalization gap, we pro-pose that agents should mimic human behavior: leveraging external ex-pertise to navigate unknown environments. In this paper, we introduceWebOne, a novel benchmark designed to evaluate an agent’s ability to mas-ter unseen websites by referencing Multimodal Tutorials—heterogeneousknowledge sources derived from instructional videos, historical trajec-tories, and human demonstrations. WebOne comprises 1,342 real-worldtasks and 970 high-quality tutorials across 60+ websites. Building uponthis, we propose WebLearner, a reinforcement learning-based frameworkthat utilizes a hierarchical referencing strategy to synthesize informa-tion from these multimodal tutorials. Experimental results demonstratethat WebLearner significantly outperforms current state-of-the-art open-source models, achieving a 56.9% success rate on held-out websites andshowing competitive performance against recent proprietary models.

ECCV 2026

On the Vulnerability of Parameter-Level Defenses to Model Merging

Kuangpu Guo, Jian Liang, Qingyan Zheng, Yu Yongcan, Zilei Wang, Ran He, Tieniu Tan

The training-free integration of expert models via modelmerging has exposed significant security risks, enabling free-riders tocombine specialized models without authorization. Recent works pro-pose parameter-level defenses that employ linear parameter transforma-tions to neutralize this threat. In this paper, we systematically analyzesuch defenses and reveal that their protected task vectors are inherentlysmall in magnitude. Consequently, the protected weights remain over-whelmingly dominated by the pretrained model. Based on this observa-tion, we designate the pretrained model as a static reference anchor andpropose the Anchor-Guided Attack (AGA) to circumvent existing safe-guards. Specifically, AGA aligns the protected model with this anchorto recover the transformation matrix analytically. Extensive evaluationsvalidate that AGA consistently bypasses both individual and compositedefenses under realistic defense-agnostic scenarios. Furthermore, we pro-vide Anchor-Repulsive Fine-tuning (ARF), a defense method to mitigatethe anchor dominance leveraged by AGA. Empirical results confirm thatARF effectively defeats the proposed attack. Our code is available athttps://github.com/krumpguo/secure-merge-attack.

ECCV 2026

MV-Forcing: Long Multi-View Video Generation via 4D-Grounded Spatio-Temporal Self-Forcing

Gal Fiebelman, Hadar Averbuch-Elor, Sagie Benaim

Recent advances in video diffusion models have enabled either long single-view generation through temporal autoregression, or short multi-view synthesis through bidirectional attention. However, generating long, multi-view consistent videos of dynamic scenes remains unsolved. In this work, we present MV-Forcing, a framework that composes temporal and view-wise autoregression within a single diffusion model by introducing a 4D geometric bridge between sequentially generated views. Our key insight is that an autoregressive 3D reconstruction model naturally interfaces between autoregressively generated views. Given a completed source view, we reconstruct its 3D structure and render a geometric prior of the next target viewpoint, which the diffusion model refines into a high-quality video. To extend generation beyond the teacher’s fixed temporal window, we introduce a joint denoising regime where both view slots are initialized from noise during training, enabling temporally unbounded generation. We distill the model via Distribution Matching Distillation with Spatio-Temporal Self-Forcing, closing the train-inference exposure bias gap for both temporal and view-sequential autoregression. Extensive experiments on both synthetic and real-world data demonstrate that MV-Forcing produces geometrically consistent multi-view videos of dynamic scenes at arbitrary lengths and viewpoint counts using a single few-step student model.

ECCV 2026

Condensing Large-Scale Datasets Directly with Minimal Information Loss

Xinyi Shang, Peng Sun, Bei Shi, Zixuan Wang, Tao Lin

Recent advancements in scaling dataset distillation rely heav-ily on decoupled information extraction pipelines, comprising Squeeze,Recover, and Relabel stages. Despite their scalability to large-scaledatasets, these methods suffer from prohibitive computational overheadand poor cross-architecture generalization. In this paper, we reveal theroot cause of these bottlenecks: the implicit dual-compression process,from data to model and back to images, inherently induces severe infor-mation loss. Crucially, we empirically and theoretically demonstrate thatthis loss creates a distribution shift that fundamentally compromises thewidely adopted Relabel strategy, transforming the pre-trained modelinto an unreliable labeler that yields sub-optimal labels. To overcomethese critical flaws, we propose CIM, a novel, metric-driven frameworkthat abandons the flawed dual-compression paradigm. Instead, CIM ex-plicitly quantifies and minimizes the information gap between the originaland synthetic datasets. By directly aligning the data distributions, ourapproach ensures high-fidelity information condensation and inherentlysatisfies the prerequisites for effective relabeling. Extensive experimentsdemonstrate that CIM establishes a new state-of-the-art. Notably, itdistills ImageNet-1K at an IPC =10 in merely 80 minutes on a singleRTX-4090 GPU, achieving an unprecedented 48.7% Top-1 accuracy onResNet-18 and significantly outperforming previous SOTA approaches,such as NRR-DD and DELT, by 2.6% and 2.9%, respectively. Our codeis available at https://github.com/LINs-lab/CIM.

ECCV 2026

RL-AWB: Deep Reinforcement Learning for Auto White Balance Correction in Low-Light Night-time Scenes

Yuan-Kang Lee, Kuan-Lin Chen, Chia-Che Chang, Yu-Lun Liu

Nighttime color constancy still remains a challenging prob-lem in computational photography due to low-light noise and complexillumination conditions. We present RL-AWB, a novel framework com-bining statistical methods with deep reinforcement learning for night-time white balance. Our method begins with a statistical algorithmtailored for nighttime scenes, integrating salient gray pixel detectionwith novel illuminant estimation. Building on this foundation, we de-velop the first deep reinforcement learning approach for color constancythat leverages the statistical algorithm as its core, mimicking profes-sional AWB tuning experts by dynamically determining image-specificparameters at inference time, without requiring ground-truth illumi-nants or reference images. To further facilitate cross-sensor evaluation,we introduce the first multi-sensor nighttime dataset. Experiment re-sults demonstrate that our method achieves strong generalization capa-bility across low-light and well-illuminated images. Project page: https://ntuneillee.github.io/research/rl-awb/

ECCV 2026

MuSViT: A Foundation Vision Model for Sheet Music Representation

Carlos Penarrubia, Antonio Rios-Vila, Eliseo Fuentes-Martinez, Juan Martinez-Sevilla, Francisco Castellanos, María Alfaro-Contreras, Jorge Calvo-Zaragoza

Foundation models have transformed vision and languageprocessing by providing rich, reusable representations that transfer acrossdiverse tasks. Sheet music, as a visual encoding of musical language,lacks such a strong domain-specific backbone. We introduce MuSViT(Music Score Vision Transformer): the first foundation vision modelfor sheet music representation—a ViT encoder pre-trained via MaskedAutoencoders on 9.7 million pages from the International Music ScoreLibrary Project (IMSLP). To handle the complexity of real-world scores,we adopt a two-stage curriculum: a synthetic warm-up on typeset scoresfollowed by large-scale training on the full IMSLP corpus. We evaluateMuSViT on four downstream tasks—full-page and staff-level music scorerecognition, music symbol detection, and score difficulty classification—under two scenarios: linear probing (frozen encoder) and fine-tuning.Under linear probing, MuSViT consistently outperforms modern visionencoders, revealing that general-purpose representations, regardless ofscale, fall systematically short on the structured symbolic propertiesMuSViT PaliGemma 2 Qwen3-VL MuSViT SoTAMuSViT-Light Kosmos-2.5 DINOv3-7B MuSViT-LightFull-Page Music Full-Page MusicScore Recognition Score RecognitionScore Difficulty Staff-Level Music Score Difficulty Staff-Level MusicClassification Score Recognition Classification Score RecognitionMusic Symbol Music SymbolDetection DetectionFig. 2: MuSViT performance across four downstream tasks. Left: Linear probing(frozen encoder)—MuSViT (solid) consistently outperforms general-purpose vision en-coders (dashed), demonstrating superior representation quality. Right: Fine-tuning—MuSViT generally outperforms state-of-the-art methods (SoTA). Axes represent nor-malized performance on each task (higher is better); see Section 3 for detailed results.of musical notation. Under fine-tuning, MuSViT generally improvesupon task-specific state-of-the-art methods. An additional embedding-transcription consistency analysis reveals that MuSViT encodes sym-bolic musical structure directly in its representation space—unlike otherencoders, whose embeddings do not correlate with music notation con-tent. These results establish MuSViT as a foundation backbone for sheetmusic understanding.

ECCV 2026

Multi-label Instance-level Generalised Visual Grounding in Agriculture

Mohammadreza Haghighat, Alzayat Saleh, Mostafa Azghadi

Understanding field imagery such as detecting plants anddistinguishing individual crop and weed instances, is a central challengein precision agriculture. Despite progress in vision–language tasks likecaptioning and visual question answering, Visual Grounding (VG), local-ising language-referred objects, remains unexplored in agriculture. A keyreason is the lack of suitable benchmark datasets for evaluating groundingmodels in field conditions, where many plants look highly similar, appearat multiple scales, and the referred target may be absent from the image.To address these limitations, we introduce gRef-CW, the first dataset de-signed for generalised visual grounding in agriculture, including negativeexpressions. Benchmarking current state-of-the-art grounding models ongRef-CW reveals a substantial domain gap, highlighting their inabilityto ground instances of crops and weeds. Motivated by these findings, weintroduce Weed-VG, a modular framework that incorporates multi-labelhierarchical relevance scoring and interpolation-driven regression. Weed-VG advances instance-level visual grounding and provides a clear baselinefor developing VG methods in precision agriculture. Code and data areavailable at https://github.com/MHaghighat98/WeedVG-gRefCW.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.