ECCV 2026 Papers

Most advances in keypoint descriptions address monomodalsettings, where image variations arise from viewpoint, illumination, orcontrast changes. Multimodal scenarios involve images produced by fun-damentally different sensing processes, such as multispectral imaging,RGB-depth, satellite imagery, or medical imaging, causing the samestructures to appear differently. A common solution to cross-modal de-scription is to train descriptors for each modality pair, which requires re-training whenever the modalities change, or to train large models, whichincur a significant increase in runtime. Instead, we propose CrossFeat,a framework that enables an existing monomodal descriptor to operateacross modalities. Our method learns a crossing function in descriptorspace that maps features from one modality to a representation com-patible with another. To preserve the structural information capturedby the original descriptor, CrossFeat introduces a geometry–appearancedisentanglement such that only appearance is altered while the geomet-ric properties are preserved. Experiments across multiple domains anddatasets demonstrate improved performance in multimodal matching.Code available here: https://github.com/paulschneider01/CrossFeat

World–Action Models (WAMs) initialized from pre-trainedvideo generation backbones have demonstrated remarkable potential forrobot policy learning. However, existing approaches face two critical bot-tlenecks: joint reasoning over future visual dynamics and actions incurssubstantial inference overhead, and joint modeling entangles visual andmotion representations, making motion prediction dependent on videoforecasts. To address these issues, we introduce GigaWorld-Policy, anaction-centered WAM that learns 2D pixel–action dynamics while en-abling efficient action decoding with optional video generation. Specif-ically, the model predicts future action sequences conditioned on thecurrent observation and, during training, generates future videos condi-tioned on the predicted actions and the same observation. Supervisionfrom both action prediction and video generation provides richer learn-ing signals and encourages physically plausible actions through visual-dynamics constraints. With a causal design that prevents future-videotokens from influencing action tokens, explicit future-video generationis optional at inference time, enabling faster deployment. To supportthis paradigm, we curate a diverse, large-scale robot dataset to pre-train the model. Experiments on real-world robotic platforms show thatGigaWorld-Policy runs 9× faster than the WAM baseline Motus whileimproving task success rates by 7%; compared with π0.5 , it improvesperformance by 95% on RoboTwin 2.0.

Recent diffusion-based generative models have shown strongcapabilities in 3D scene understanding and affordance reasoning. How-ever, existing methods predominantly rely on deterministic global en-coders or fully stochastic diffusion backbones, which fail to disentanglestructural certainty from contextual uncertainty, a key requirement forreliable affordance-grasp prediction in cluttered, partially observed en-vironments. We propose a split-latent hierarchical diffusion frameworkthat explicitly decomposes the global scene representation into two com-plementary subspaces: a fixed latent encoding deterministic geometricand semantic priors, and a diffused latent capturing stochastic scene-level variations through a Global Conditional Diffusion guided by RGBcontext. To propagate global uncertainty to fine-grained reasoning, weintroduce a Local Conditional Diffusion that predicts dense, point-wiseaffordance and grasp fields conditioned jointly on both latents, producinggeometrically precise yet contextually diverse predictions. Furthermore,we incorporate Harmonic Grasp Field, a differential geometric regulariza-tion that enforces smooth, obstacle-aware grasp manifolds via harmonicpotential constraints. This unified formulation bridges deterministic ge-ometry with probabilistic reasoning, yielding physically consistent anduncertainty-aware grasp generation. Extensive experiments on cluttered3D affordance and grasping benchmarks demonstrate superior stability,generalization, and diversity compared to deterministic and single-latentdiffusion approaches. Project Website.

Learning robot manipulation policies from large-scale human videos is challenging due to the viewpoint mismatch between thirdperson observations and egocentric robot control. To this end, we introduce Exo2EgoPolicy, a geometry-aware framework that learns egocentric manipulation policies from monocular exocentric video via viewpointaligned pose translation. Unlike prior Exo→Ego approaches that rely on diffusion-based pixel synthesis and often produce temporally inconsistent predictions, our method operates directly on a structured pose manifold and explicitly models the geometric ambiguity inherent in monocular observation. We formulate exocentric-to-egocentric translation as a viewpoint-conditioned latent variable model on the SE(3) manifold that disentangles pose from camera transformation and captures the ambiguity induced by unknown camera extrinsics. Under bounded viewpoint motion, the formulation yields pose trajectories that are identifiable up to a global rigid transformation and supports stable sequential inference through temporal regularization. Beyond representation alignment, we show that for quasi-static manipulation tasks whose rewards depend primarily on relative hand–object geometry, pose captures the key task-relevant information required for policy learning. This enables policy co-training on pose-translated human demonstrations alongside limited robot teleoperation data within a unified human–robot state space. Empirically, Exo2EgoPolicy improves temporal consistency, cross-view alignment, and out-of-distribution policy transfer compared to pixelbased translation and other cross-view co-training baselines, achieving 20–30% higher task success on manipulation benchmarks. These results suggest that geometry-aligned pose representations provide a scalable foundation for cross-view policy learning from human video. Website

Creating 3D visual illusions, a single 3D mesh that revealsentirely different semantics from various viewing angles, is a fascinat-ing but tough challenge. Existing optimization-based methods are slowand can produce oversaturated colors. In contrast, naive stitching ap-proaches fail to produce geometrically coherent objects. This results invisible unnatural seams and semantic leaks. In this paper, we presenta fast and training-free framework for generating text-driven 3D visualillusions. Our approach decouples the generation into two stages. First,we propose a cross-space dual-branch denoising process. This processdynamically decodes 3D latents into voxel space for CLIP-guided orien-tation alignment and Signed Distance Field (SDF) blending, which en-sures seamless geometric fusion. Second, we introduce a view-conditionedtexture synthesis module that projects and aggregates view-specific 2Ddiffusion priors onto the fused geometry. Extensive experiments demon-strate that our method generates highly realistic, dual-semantic 3D illu-sions in just 3–5 minutes. It significantly outperforms existing methodsin geometric integrity, semantic recognizability, and efficiency. Projectpage: https://siang1105.github.io/JanusMesh.github.io/Target Junction view Target Junction view Target Junction viewview view view“A parrot”𝜃" 𝜃" 𝜃"Target Target Targetview view view𝜃! 𝜃! 𝜃!“A boat”❌ Severe over-saturation ❌ Visible backside leakage ✅ Dual-semantic coherency❌ Slow optimization (~40 min) ❌ Unnatural geometric seams ✅ Fast & Training-free (~3-5 min)(a) SDS-Based Method (b) Direct Concatenation (c) OursFig. 2: Comparison of 3D visual illusion generation methods. (a) SDS-BasedMethods suffer from severe over-saturation and slow optimization. (b) Direct Con-catenation exposes unnatural geometric seams and semantic leakage at target views(red arrows). (c) Our method creates a seamless, dual-semantic coherent 3D mesh.Unlike previous approaches, our method does not require training. It generates high-quality 3D visual illusions in just 3–5 minutes while completely preventing geometricinterference between the two semantics.

Whole slide image (WSI) analysis is central to computationalpathology, with multiple instance learning (MIL) emerging as the stan-dard pipeline for slide-level diagnosis. However, conventional approachesformulate WSI diagnosis as a flat classification task over discrete labels,contradicting the inherently hierarchical, coarse-to-fine nature of clinicalreasoning. Although recent hierarchical classifiers and vision–languagemodels (VLMs) have sought to address this structural gap, they eitherfail to capture semantic continuity between related diagnoses or sufferfrom unconstrained text generation that produces taxonomic hallucina-tions and parent–child label violations. To address these limitations, wepropose TaxoMIL, a taxonomy-constrained framework that reformulatesWSI diagnosis as a multi-granularity text generation task. TaxoMIL uti-lizes a dual-head Transformer decoder to generate coarse- and fine-leveldiagnostic text, and introduces taxonomy-guided objectives that explic-itly structure the label embedding space and strictly ground slide-level vi-sual representations within the clinical taxonomy. Extensive experimentsacross three diverse WSI datasets demonstrate that TaxoMIL consis-tently outperforms state-of-the-art MIL classifiers and VLM-based gen-erative methods, yielding accurate and hierarchy-aware diagnostic pre-dictions. The code is released at https://github.com/QuIIL/TaxoMIL..

Current multimodal large language models (MLLMs) havedemonstrated remarkable capabilities in short-form video understand-ing, yet translating long-form cinematic videos into detailed, temporallygrounded scripts remains a significant challenge. This paper introducesthe novel video-to-script (V2S) task, aiming to generate hierarchical,scene-by-scene scripts encompassing character actions, dialogues, expres-sions, and audio cues. To facilitate this, we construct a first-of-its-kindhuman-annotated benchmark and propose a temporally-aware hierarchi-cal evaluation framework. Furthermore, we present OmniScript, an 8B-parameter omni-modal (audio-visual) language model tailored for long-form narrative comprehension. OmniScript is trained via a progressivepipeline that leverages chain-of-thought supervised fine-tuning for plotand character reasoning, followed by reinforcement learning using tem-porally segmented rewards. Extensive experiments demonstrate that de-spite its parameter efficiency, OmniScript significantly outperforms largeropen-source models and achieves performance comparable to state-of-the-art proprietary models, including Gemini 3-Pro, in both temporallocalization and multi-field semantic accuracy. The code is available athttps://github.com/TencentARC/OmniScript.

We introduce a novel neural representation for maps between3D shapes based on flow-matching models, which is computationally ef-ficient and supports cross-representation shape matching without large-scale training or data-driven procedures. 3D shapes are represented asthe probability distribution induced by a continuous and invertible flowmapping from a fixed anchor distribution. Given a source and a targetshape, the composition of the inverse flow (source to anchor) with theforward flow (anchor to target), we map points between the two surfaces.By encoding the shapes with a pointwise task-tailored embedding, thisconstruction provides an invertible and modality-agnostic representationof maps between shapes across point clouds, meshes, signed distancefields (SDFs), and volumetric data. The resulting representation con-sistently achieves high coverage and accuracy across diverse benchmarksand challenging settings in shape matching. Beyond shape matching, ourframework shows promising results in other tasks, including UV mappingand registration of raw point cloud scans of human bodies.

Multimodal Large Language Models (MLLMs) have demon-strated remarkable abilities when analyzing images, yet translating thesecapabilities to few-shot image classification remains challenging. To bridgethis gap, we present DeCoDe, a simple yet effective technique that en-ables off-the-shelf MLLMs to act as strong few-shot classifiers withoutany additional training. Our approach builds on the idea of few-shotclassification as a set of pairwise image comparisons, decomposing thetask into a set of binary decisions. Given a query image and a supportimage from a candidate class, the MLLM is prompted to decide whetherthe two images depict the same class. The logit corresponding to an af-firmative response is then used as a similarity score to assign the queryimage to the most likely class. While this already yields good results,we show that providing additional high-level information, such as thedata domain, to the model further improves performance. Our evalu-ation provides an extensive analysis of various inference variants on asuite of twelve datasets, six established and six newly curated few-shotbenchmarks spanning across diverse domains. The results show that theproposed simple decomposition technique can turn off-the-shelf MLLMsinto powerful few-shot learners, significantly outperforming current state-of-the-art few-shot methods on both standard and novel domains. Codeis available at https://github.com/yunhanwang1105/DeCoDe.

Vision Language Models (VLMs) have shown promising plan-ning capabilities, yet their success remains confined to the text domain,leaving visual decision-making relatively underexplored. Addressing thisgap, we introduce Corrective Sequence Planning (CoSPlan) bench-mark, where VLMs must plan a sequence of visual actions from an initialscene to a target scene. CoSPlan evaluates models on their ability to imag-ine and execute a coherent set of visual steps required to reach the goal(Step Completion). To prevent any shortcuts that simply describethe final scene, we introduce an erroneous action in decision making,which must be detected (Error Detection) and corrected to reach thegoal, enabling a deeper understanding of the task. CoSPlan spans across4 tasks: maze navigation, block re-arrangement, image reconstruction,and object re-organization. Despite using advanced reasoning strategiessuch as Chain-of-Thought and Scene Graphs, VLMs struggle on CoSPlan,while still showing promising performance in the text domain. Addressingthis, we propose Scene Graph Incremental updates (SGI), a noveltraining-free method to transform images into ‘textual’ scene graphs, en-abling step-by-step reasoning through iterative scene graph refinement.SGI yields an average of ≃ 4.4% ↑ on CoSPlan w/ generalization onPlanBench and VQA. Link for solving puzzles on the project page.

Sparse autoencoders (SAEs) are increasingly used to extract interpretable concepts from vision and vision language models, yet existing evaluation methods largely rely on proxy metrics or qualitative inspection rather than measuring semantic correspondence. We present a human-grounded evaluation framework that quantifies alignment between SAE latents and human-annotated concepts, without requiring user studies, and validate this matching through targeted attribute perturbations. To enable this intervention-style evaluation in vision, we construct synCUB and synCOCO, synthetic benchmarks of paired images that differ in exactly one attribute. We introduce Fully-Binary Matching Pursuit (FBMP), a coalition-based matching procedure that supports many-to-one mappings between SAE latents and annotated concepts, and consistently outperforms one-to-one baselines. For functional validation, we propose a Targeted Attribute Perturbation Alignment Score (TAPAScore), which tests whether matched concepts respond selectively and in the expected direction under targeted image-level attribute perturbations. Under sanity checks, our matching and TAPAScore are the only evaluated metrics that reliably distinguish trained SAEs from untrained ones. Across SAEs trained on CLIP and DINOv2 embeddings, we find that increased overcompleteness can reduce perturbation alignment, indicating a reduction in interpretability. Our evaluation framework suggests that moderate dictionary sizes provide the best trade-off, yielding the most interpretable SAEs. Code and datasets are available at https://github.com/JonasKlotz/sae-concept-eval.

Deep neural networks remain vulnerable to backdoor at-tacks, in which adversaries embed hidden triggers during training tocause misclassification of any trigger-carrying input into an attacker-chosen target class, while maintaining normal behavior on clean inputs.Though recent fine-tuning-based defenses have shown promise in miti-gating backdoor attacks and are particularly practical in continual learn-ing and MLaaS scenarios, where backdoors are removed from pretrainedmodels using small clean datasets, we demonstrate that this defensiveparadigm remains fundamentally exploitable by an adaptive adversary.We study a threat model in which attackers anticipate fine-tuning-baseddefenses and craft backdoors whose resulting optimization signals areentangled with those of the clean task. Our approach leverages FisherInformation to identify task-critical neurons and optimizes a trigger gen-erator such that backdoor gradients align with benign gradients on theseneurons, ensuring that any attempt to suppress the backdoor inevitablyinterferes with clean task preservation. As a result, fine-tuning defensesfails to effectively reduce attack success rate without incurring substan-tial degradation in clean accuracy. Through extensive evaluation againstseven attacks and eight defenses, we show our method consistently showsa stronger effect than existing backdoor attacks across all settings. Ourwork reveals a critical vulnerability in fine-tuning-based defenses andhighlights the need for more robust defense mechanisms against adap-tive adversaries.

Video diffusion models have recently been applied with success to problems in content generation, novel view synthesis, and, more broadly, world simulation. Many applications in generation and transfer rely on conditioning these models, typically through perceptual, geometric, or simple semantic signals, fundamentally using them as generative renderers. At the same time, high-dimensional features obtained from large-scale self-supervised learning on images or point clouds are increasingly used as a general-purpose interface for vision models. The connection between the two has been explored for subject specific editing, aligning and training video diffusion models, but not in the role of a dense conditioning signal for pretrained video diffusion models. Features obtained through self-supervised learning like DINOv3, contain a lot of entangled information about style, lighting and semantics of the scene. This makes them great at reconstruction tasks but limits their generative capabilities. In this paper, we show how we can use the features for tasks such as video domain transfer and video-from-3D generation. We introduce a lightweight control architecture and training strategy that decouples appearance from other features that we wish to preserve, enabling robust control for appearance changes such as stylization and relighting. Furthermore, we show that low spatial resolution can be compensated by higher feature dimensionality, improving controllability in generative rendering from explicit spatial representations.

Advances in neural rendering have enabled high-fidelity multi-view reconstruction of 3D scenes. However, free-form non-rigid shapeediting remains a significant challenge. Point-based neural representationsare highly desirable for multi-view reconstruction because they lack fixedconnectivity, which does not constrain the learned surface topology tothat of the initialization. Yet this same property causes point-based rep-resentations to struggle with holes and surface discontinuities under largedeformations. To address this, we propose a novel self-supervised methodto enable point-based representations to adapt to large deformations with-out requiring ground truth multi-view images of deformed geometry. Thekey idea is to generate random deformations and to ensure consistencyin the predicted surface before and after deformation. In particular, thesurface prediction from the deformed point cloud should be the sameas the deformation applied to the surface prediction from the originalpoint cloud. We incorporate our approach into attention-based pointrepresentations, which differ from splatting-based point representationsin their use of a learned interpolation kernel between points as opposedto a Gaussian kernel around each point. This learned interpolation kernelcan learn to adapt to large deformations, without requiring addition orremoval of points. We show that our framework significantly enhancesits robustness to large deformations. Experiments on synthetic geometryediting benchmarks (Neural Editor, Objaverse) demonstrate that ourapproach outperforms existing point-based methods in zero-shot editingand significantly reduces artifacts. Furthermore, qualitative results on theDTU and Mip-NeRF 360 datasets demonstrate our method’s effectivenesson real-world scenes.

Recent point-based intrinsic decomposition and inverse ren-dering methods have advanced the modelling of the shading and albedoof 3D scenes. However, we identify a fundamental limitation: these meth-ods suffer from a misattribution issue, where individual primitives learnincorrect appearance features despite producing correct aggregated ren-derings. We show that the root cause lies in volume rendering, whichaggregates translucent primitives along each ray and only supervises thefinal colour, preventing direct supervision of individual primitive fea-tures. To address this, we propose Intrinsic PAPR, a robust intrinsic de-composition framework which leverages Proximity Attention Point Ren-dering (PAPR) to enable direct per-point supervision. Unlike volumerendering approaches, PAPR eliminates translucent primitives and di-rectly predicts appearance at ray-surface intersections, enabling accuratesupervision to the feature of each individual point. Our method incor-porates a 2D albedo prior adapted with conditional Implicit MaximumLikelihood Estimation (cIMLE) to handle monocular ambiguities, andemploys a space carving loss to ensure multi-view consistency. Exten-sive evaluations on synthetic and real-world datasets demonstrate thatIntrinsic PAPR outperforms point-based inverse rendering, NeRF-basedintrinsic decomposition, and diffusion-based PBR methods in novel viewsynthesis and albedo estimation while resolving the misattribution issue.

Recent advances in Image-to-Video (I2V) generation allow asingle image to be animated into a convincing video under text guidance,raising serious copyright and privacy risks. We propose Anti-Prompt,an image protection approach that injects imperceptible perturbationsinto an image, inducing visible inconsistencies and structural failures intext-guided I2V generation. Our method is motivated by a simple em-pirical observation: when text guidance is removed from modern I2Vmodels, generation quality degrades markedly, not only in motion re-alism but also in subject preservation, structural coherence, and tem-poral consistency. Building on this insight, Anti-Prompt exploits themodel’s reliance on textual guidance by attenuating text-conditioned in-teractions during denoising while strengthening visual-only pathways.To further systematically evaluate protection effectiveness, we introducea Video-LLM–assisted evaluation protocol that provides interpretable,frame-grounded analyses of generation artifacts and inconsistencies. Ex-periments on two representative I2V architectures demonstrate that ourmethod achieves strong protection performance while improving effi-ciency and cross-model transferability.

Workflow generation in visual creation systems such as Com-fyUI demands not only syntactic accuracy but also expert-level reason-ing over modular compositions. Existing large language model (LLM)approaches often treat this as a direct text-to-JSON generation task,struggling with structural brittleness and lacking the experiential knowl-edge required for effective design. We argue that successful workflowgeneration requires modeling knowledge itself, including its structure,hierarchy, and reasoning dynamics. To this end, we propose a knowledge-centric framework that learns to invert, inject, and infer with knowl-edge across multiple abstraction levels. We first perform knowledge in-version to distill hierarchical representations, ranging from full pseudo-codes and skeletons to high-level strategies, from large collections of real-world workflows. We then conduct knowledge injection through super-vised fine-tuning, teaching the model to reason from task descriptionsto strategies and from strategies to executable structures. During infer-ence, the model performs reversible reasoning to synthesize executableworkflows, augmented by self-refinement for structural coherence. Ex-tensive experiments demonstrate that our method produces workflowswith richer node diversity, more coherent structures, and higher execu-tion success rates than existing systems, establishing a new foundationfor knowledge-driven, agentic workflow generation.

Measuring structured object understanding in vision founda-tion models remains challenging due to inconsistent evaluation protocolsand limited part-level supervision. Semantic correspondence (SC) eval-uates this capability by testing whether object parts can be matchedacross instances and categories under large variations in appearance,viewpoint, and geometry. To enable a systematic SC evaluation, we in-troduce SOCO, a new benchmark for Semantic Object Correspondencethat introduces a taxonomy of correspondence types and provides con-sistent, functionally meaningful keypoint annotations across 100 cate-gories and over 1M correspondence pairs. In addition, SOCO includeskeypoint language descriptions, enabling the evaluation of large vision-language models (LVLMs) and their fine-grained part-level understand-ing. Comprehensive experiments reveal that (i) vision foundation back-bones encode strong semantic structure but transfer correspondencespoorly across related categories and only partially capture object-partposition, (ii) LVLMs are stronger at text-prompted part localizationthan at visual-reference cross-image matching, exposing a gap betweenlanguage-grounded localization and fine-grained visual correspondence,and (iii) correspondence performance predicts dense downstream tasks—segmentation, tracking, 3D pose estimation, and 3D detection—morestrongly than ImageNet classification. Together, these findings positionSOCO as a benchmark for structured, part-level representation qualityin vision and multimodal foundation models.

Whole-slide images (WSIs) are widely used for computa-tional cancer prognosis. However, current research primarily focuses onin-domain performance and fails to generalize across clinical centers. Thislimitation stems from their reliance on pixel-derived representations thatare highly susceptible to domain-specific artifacts caused by stainingprotocols and scanner hardware. We hypothesize that high-level pathol-ogy semantics, such as tumor grade and micro-environmental architec-ture, provide a domain-invariant semantic representation that mirrorsthe robust diagnostic logic of human pathologists. Therefore, we proposea Semantic-Anchored Evidential Fusion Survival (SAEFS) framework,where SAEFS derives semantic anchors from WSIs via Visual QuestionAnswering (VQA), employs a dual-stream WSI evidence extraction archi-tecture, uses Dirichlet-based Subjective Logic to model uncertainty, andfuses semantic and visual evidence through a cautious conjunction rule toavoid overconfident fusion from correlated sources. Trained exclusively onone source domain and evaluated zero-shot across four unseen domains,SAEFS consistently outperforms state-of-the-art models both in predic-tion accuracy and reliability, improving the average C-index by 10.2%.Quantitative analyses further show that VQA-derived semantic featuresexhibit significantly lower cross-center divergence than pixel-derived fea-tures, highlighting their robustness for cross-center clinical applications.The code is available at https://github.com/YuchengXing99/SAEFS.