ECCV 2026 Papers

We introduce EmbedCopilot-Bench, a multimodal bench-mark designed to evaluate large vision-language models (LVLMs) as as-sistants for embedded development. Compared with pure software pro-gramming, embedded programming inherently involves hardware, mak-ing it well-suited to benefit from LVLMs’ visual capabilities. However, tothe best of our knowledge, no existing benchmark comprehensively eval-uates how effectively LVLMs can serve as embedded copilots. Our bench-mark is developed to fill this gap. Built on real-world embedded develop-ment videos, we constructed 216 annotated multimodal QA triplets span-ning heterogeneous hardware platforms and peripheral modules, cov-ering hardware operation, software configuration, and code generationtasks. To assess model performance, we pair a rubric-driven LLM-as-a-Judge protocol with execution-centric evaluation in Wokwi platformand hardware-in-the-loop Execution Success Rate (ESR), jointly cap-turing semantic quality and functional correctness. Experiments on arange of closed- and open-source LVLMs show that visual context sig-nificantly boosts performance, especially for hardware-related tasks, butthat models remain far from reliable embedded copilots. ESR analysesand a pin-localization case study reveal frequent near-miss failures causedby brittle grounding and limited structured reasoning over board layouts.EmbedCopilot-Bench provides a challenging, realistic testbed for futuremethods that more tightly integrate perception, circuit understanding,and executable code synthesis for real-world embedded development.

Generalization is a central challenge in autonomous driv-ing, as real-world deployment requires robust performance under unseenscenarios, sensor domains, and environmental conditions. Recent world-model-based planning methods have shown strong capabilities in sceneunderstanding and multi-modal future prediction, yet their generaliza-tion across datasets and sensor configurations remains limited. In addi-tion, their loosely coupled planning paradigm often leads to poor video-trajectory consistency during visual imagination. To overcome these limi-tations, we propose DriveVA, a novel autonomous driving world modelthat jointly decodes future visual forecasts and action sequences in ashared latent generative process. DriveVA inherits rich priors on mo-tion dynamics and physical plausibility from well-pretrained large-scalevideo generation models to capture continuous spatiotemporal evolutionand causal interaction patterns. To this end, DriveVA employs a DiT-based decoder to jointly predict future action sequences (trajectories)and videos, enabling tighter alignment between planning and scene evolu-tion. We also introduce a video continuation strategy to strengthen long-duration rollout consistency. DriveVA achieves an impressive PDM-based planning performance of 90.9 PDM score on the NAVSIM bench-mark. Extensive experiments also demonstrate the zero-shot capabilityand cross-domain generalization of DriveVA, which reduces average L2error and collision rate by 78.9% and 83.3% on nuScenes and 52.5%and 52.4% on the Bench2Drive built on CARLA v2 compared with thestate-of-the-art world-model-based planner.

Camera-controlled video diffusion models (VDMs) have re-cently emerged as powerful world models, enabling users to explore 3Dscenes through flexible, user-defined camera trajectories. Nevertheless,current VDMs typically rely on implicit spatiotemporal representationswithout explicit 3D geometric grounding. Such geometry-agnostic mod-eling often leads to issues including geometrically implausible struc-tures and cross-view spatial inconsistencies. To alleviate this, we presentDreamWorld, a new recipe of world model that novelly bridges the strongspatial structure priors of 3D foundation models with the high-fidelitygenerative capabilities of video diffusion models for geometry-consistent3D scene generation. Specifically, given the input image and camera tra-jectory, DreamWorld first learns a geometry video diffusion model topredict compact geometry features for the target novel views, function-ing as explicit structure pivots to reflect the underlying 3D spatial layout.To achieve this, we introduce a distillation paradigm that transfers high-level structural knowledge from a pretrained 3D foundation model to thediffusion model, thereby enabling it to produce geometrically consistentand spatially coherent features. Conditioned on such geometry features,another appearance video diffusion model is then utilized to synthesizethe final video, ensuring improved geometric plausibility and cross-viewconsistency while maintaining high visual fidelity. Extensive experimentsdemonstrate that DreamWorld outperforms existing methods in visualquality, 3D consistency, and camera controllability. Our project page isavailable at https://yanghb22-fdu.github.io/DreamWorld.

Reconstructing physics-based 3D assets—geometry, mate-rials, and illumination—from multi-view images is a core problem incomputer graphics and vision, and a prerequisite for realistic relightingand editing. Physics-based inverse rendering offers an accurate image-formation model, but is severely underconstrained: without strong pri-ors, illumination is baked into materials, and reconstructions generalizepoorly to novel views and lighting. Data-driven diffusion models, in con-trast, predict visually plausible materials, yet their predictions rarelysatisfy the rendering equation and are not directly usable for physics-based rendering. We bridge these two paradigms rather than replacingeither. Our key idea is to treat the predictions of a state-of-the-art dif-fusion model not as target material values but as a similarity kernel foroptimization: we introduce a regularization loss that penalizes deviationsin the optimized material over surface regions where the diffusion predic-tions are near-constant, while leaving the optimization free to match theinput images. Built on this regularizer, our end-to-end pipeline jointlyreconstructs geometry, materials, and illumination, yielding high-qualityassets that drop into standard rendering pipelines and relight faithfully.On the Synthetic4Relight, Stanford-ORB, and DTC-Synthetic datasets,our method significantly outperforms state-of-the-art baselines in bothreconstruction accuracy and relighting quality.

Large vision-language models with CLIP as a core compo-nent have achieved remarkable progress across a wide range of tasks,yet they remain highly vulnerable to adversarial attacks. Existing ad-versarial fine-tuning methods typically optimize CLIP under a singlefixed perturbation strength, resulting in weak robustness generalizationand semantic instability. To address this limitation, we propose Path-Consistent Fine-Tuning (PACO), a new framework for unsupervised ad-versarial fine-tuning. Rather than optimizing solely on adversarial exam-ples, PACO adopts a two-stage procedure that regularizes representa-tions on continuous local paths along adversarial directions. Specifically,it first anchors adversarial embeddings to a clean reference to preventsevere semantic drift. Building upon this, it constructs a local path be-tween the clean sample and the adversarial anchor, regularizing inter-mediate representations to encourage a linear transition, which yields amore stable visual embedding. Extensive experiments show that PACOachieves a superior robustness-accuracy trade-off: it not only mitigatesthe clean-performance degradation common to prior methods, but alsodelivers superior robustness across diverse attack settings and perturba-tion strengths. Our code is available at https://github.com/Trusted-LLM/PACO.

Visual reasoning in immersive environments is critical forVR, AR, and robotics, yet is fundamentally challenged by partial ob-servability. Existing Vision-Language Models (VLMs) are ill-suited forthis task, as their rigid “look first, think later” paradigm decouples per-ception from reasoning. They either process distorted, low signal-to-noisepanoramas or reason over a fixed set of pre-determined views, preventingdynamic, task-driven exploration. We reframe immersive visual reason-ing as an iterative process of probabilistic belief updating. We introduceDiffusion-Based Immersive Visual Reasoning (DIVR), a novel frameworkthat models reasoning as a denoising diffusion process, progressively re-fining a latent reasoning hypothesis from a state of high uncertainty toa denoised low-uncertainty state. DIVR comprises two tightly coupledmodules: a Latent Hypothesis Refiner (LHR) that uses observed visualevidence to denoise the current hypothesis, and a Reasoning-Guided Ob-server (RGO) that uses the denoised hypothesis to actively select the nextmost informative viewport. This creates a closed-loop system where theevolving reasoning hypothesis continuously guides perception, distinctfrom the rigid, open-loop paradigm of prior work. Extensive experimentson multiple 360° VQA benchmarks demonstrate that DIVR establishesa new state-of-the-art, significantly outperforming strong baselines, par-ticularly on complex questions that require dynamic, multi-step evidencegathering.

Most digital videos are stored in 8-bit low dynamic range(LDR) formats, where much of the original high dynamic range (HDR)scene radiance is lost due to saturation and quantization. This loss ofhighlight and shadow detail precludes mapping accurate luminance toHDR displays and limits meaningful re-exposure in post-production work-flows. Although techniques have been proposed to convert LDR imagesto HDR through dynamic range expansion, they struggle to restore re-alistic detail in the over- and underexposed regions. To address this, wepresent DiffHDR, a framework that formulates LDR-to-HDR conversionas a generative radiance inpainting task within the latent space of a videodiffusion model. By operating in Log-Gamma color space, DiffHDR lever-ages spatio-temporal generative priors from a pretrained video diffusionmodel to synthesize plausible HDR radiance in over- and underexposedregions while recovering the continuous scene radiance of the quantizedpixels. Our framework further enables controllable LDR-to-HDR videoconversion guided by text prompts or reference images. To address thescarcity of paired HDR video data, we develop a pipeline that synthesizeshigh-quality HDR video training data from static HDRI maps. Extensiveexperiments demonstrate that DiffHDR significantly outperforms state-of-the-art approaches in radiance fidelity and temporal stability, produc-ing realistic HDR videos with considerable latitude for re-exposure.

Despite the exceptional reasoning capabilities of MultimodalLarge Language Models (MLLMs), their adaptation into universal em-bedding models is significantly impeded by task conflict. To addressthis, we propose TSEmbed, a universal multimodal embedding frameworkthat synergizes Mixture-of-Experts (MoE) with Low-Rank Adaptation(LoRA) to explicitly disentangle conflicting task objectives. Moreover,we introduce Expert-Aware Negative Sampling (EANS), a novel strategythat leverages expert routing distributions as an intrinsic proxy for se-mantic similarity. By dynamically prioritizing informative hard negativesthat share expert activation patterns with the query, EANS effectivelysharpens the model’s discriminative power and refines embedding bound-aries. To ensure training stability, we further devise a two-stage learningparadigm that solidifies expert specialization before optimizing repre-sentations via EANS. TSEmbed achieves state-of-the-art performance onboth the Massive Multimodal Embedding Benchmark (MMEB) and real-world industrial production datasets, laying a foundation for task-levelscaling in universal multimodal embeddings.

Federated parameter-efficient fine-tuning (PEFT) enables scalable adaptation of large pre-trained backbones under communication constraints. In heterogeneous federated settings, personalization is commonly realized through gated residual mixing between shared and clientspecific branches. However, this parameterization is inherently scale nonidentifiable: only the product of residual magnitude and mixing weight determines the functional contribution. When private residuals are optimized locally, their scales may drift across clients and communication rounds, rendering mixing coefficients an unreliable measure of personalization strength. We address this limitation with FedSDG, a structuredecoupled gating framework for federated PEFT. FedSDG maintains separate shared and private LoRA branches and introduces projection-level scalar gates for depth-aware personalization. Crucially, we propose Dynamic Alignment, a backbone-anchored calibration mechanism that stabilizes private residual scale prior to gated mixing, thereby improving gate identifiability without additional communication. Extensive experiments demonstrate consistent personalization improvements and stable depth-wise adaptation across diverse non-IID regimes.

Multi-person pose estimation (MPPE), which involves detecting body joint positions (keypoints) for every person in an image, is a fundamental task in computer vision. Despite recent advances, no transformer-based model currently achieves real-time performance. This work addresses the latency challenge by introducing DETRPose, the first family of real-time, end-to-end transformer models for multi-person 2D pose estimation. DETRPose significantly enhances the GroupPose decoder, enabling real-time inference. For training, a novel denoising keypoint technique is proposed to accelerate convergence. The varifocal loss is also extended for keypoints, termed Keypoint Similarity VariFocal loss, to improve query quality. Extensive evaluation demonstrates that DETRPose models achieve accuracy comparable to or exceeding that of leading alternatives while requiring five to ten times fewer training epochs. DETRPose-S matches the accuracy of YOLOv8-Pose-X and YOLO11- Pose-X on the COCO dataset (67.0 vs 67.3 and 67.2 in AP) with 81% fewer parameters (11.5M vs 69.4M and 58.8M) and 52% faster inference speed (2.39ms vs 5.23ms and 4.93ms). On the CrowdPose dataset, DETRPose-X has 13.1× fewer FLOPs (232.3G vs 3048.1G) and only 2% fewer precision (75.1 vs 76.6 in AP) than ED-Pose-SwinL-5S. On the OCHuman dataset, DETRPose-S surpasses all previous models, showing the robustness of DETRPose on out-of-distribution datasets. Code is available at https://github.com/SebastianJanampa/DETRPose

Recent radiance field methods represent scenes with 2D prim-itives that offer surface alignment and efficient rasterization, from Gaus-sian disks to triangles, yet all rely on convex boundaries: curved and con-cave structures demand excessive primitives. We introduce DeformableTriangle Splatting, which augments each triangle with K control pointsper edge, each parameterized by a single learnable scalar displacementthat shifts the boundary inward or outward, enabling non-convex shaperepresentation while preserving the three base vertices that define the 3Dplane. To render these non-convex primitives differentiably, we designa rasterization pipeline in the triangle’s barycentric coordinate space,ensuring view-consistent rendering. A winding number test determineswhether each pixel lies inside the deformed primitive, and a windowfunction controlled by two learnable parameters, sharpness and cornersmoothness, together with a per-primitive scalar opacity, produces thesmooth opacity transition from interior to boundary. Validation is donein a variety of real-world scenes, outperforming recent works based onnon-volumetric primitives in terms of visual quality and versatility whilestill achieving competitive rendering efficiency.Project page: orioljim1.github.io/detris

Recent advances in generative video models have signifi-cantly improved visual realism in video object removal, yet evaluationprotocols still focus on masked-region fidelity, treating removal as lo-cal inpainting. In real scenes, object removal is a causal intervention:eliminating an object also requires removing its induced physical ef-fects, such as shadows, reflections, illumination changes, translucency,and dynamic traces. Existing benchmarks lack aligned clean referencesor remain limited to simplified synthetic settings, preventing systematicevaluation of causal consistency. We introduce BeyondMasks, a pairedbenchmark for causally consistent video object removal, consisting oftemporally aligned synthetic and real-world video pairs with clean back-ground references. The dataset spans diverse photometric, geometric,volumetric, and dynamic interactions, and supports both mask-basedand instruction-driven editing. We further propose CORE, a structuredvision–language model-based evaluation protocol that jointly measuresobject disappearance and after-effect consistency, aligning more closelywith human judgments than existing metrics. Benchmarking state-of-the-art methods reveals systematic failures in removing secondary physicaleffects despite high masked-region fidelity, exposing a gap between visualplausibility and causal correctness. BeyondMasks reframes video objectremoval as causal scene consistency rather than local reconstruction andprovides a unified framework for its evaluation.

Vision-Language-Action (VLA) models have made signifi-cant strides in embodied intelligence by integrating the powerful repre-sentations of pre-trained Vision-Language Models (VLMs). However, themassive parameter scale of VLAs imposes a heavy computational burden,and these models exhibit extreme sensitivity to parameter pruning. Cur-rent paradigms often treat the resulting performance degradation as in-evitable, relying on fine-tuning or low-rank corrections to recover efficacy.We challenge this convention by questioning whether the removed param-eters are truly redundant if VLA pruning necessitates performance recov-ery to be effective, or if this paradigm masks the indiscriminate pruningof critical parameters. We revisit parameter redundancy through the lensof VLM-to-VLA adaptation, first quantifying the spatial distribution ofparameter divergence during adaptation to reveal structured patternsacross different modules. Subsequently, we introduce controlled pruningas a diagnostic probe: by comparing the direct impact of removing dif-ferent parameter subsets on VLA performance without any fine-tuning,we establish a causal link between adaptation-induced divergence signalsand functional contributions. Based on the discovered modular hetero-geneities, we design a multi-module joint pruning scheme. Evaluationson the LIBERO benchmark demonstrate that our approach reduces theparameters of OpenVLA and π0.5 by 12%–30% while maintaining approx-imately 90% of the original performance without any post-pruning recov-ery. In contrast, existing parameter pruning criteria result in total perfor-mance collapse when evaluated under the same recovery-free constraints.Our study reveals the parameter evolution mechanism in VLA adapta-tion and provides a new path for deploying efficient, robust robotic poli-cies in resource-constrained environments. Code is available at https://github.com/Niannnnnn/VLA_Parameter_Redundancy_VLM2VLA.

High-performance Self-Supervised Learning (SSL) encodersdeployed as Encoder-as-a-Service (EaaS) are highly vulnerable to ad-vanced contrastive model stealing attacks, such as RDA and ContSteal.Traditional defenses often stack complex modules or rely on heuristicnoise, struggling to balance privacy and utility. To address this, we pi-oneer the use of active perturbations for EaaS protection by proposingUniTrap, a simple yet effective single-module defense specifically tai-lored against contrastive-based stealing paradigms. Instead of mere dis-tance maximization, UniTrap explicitly optimizes global feature entropyvia a Gaussian potential function. This generates a stealthy perturba-tion that strictly enforces an isotropic, uniform distribution on the unithypersphere—acting as “one trap” to fundamentally block contrastivealignment. We theoretically demonstrate that this perfect uniformitycauses the attacker’s informative gradients to cancel out in the tangentspace, inducing gradient vanishing and permanently stalling surrogateoptimization. Extensive experiments confirm that our method achievesa state-of-the-art privacy-utility Pareto frontier across various bench-marks.

Promptable foundation segmentation models exhibit impres-sive capabilities across diverse tasks, yet their performance deterioratessubstantially on medical images due to domain and sub-domain shifts.Online test-time adaptation (OTTA) helps alleviate these shifts, but ex-isting methods remain limited in effectiveness and efficiency. To thisend, we introduce Semantic-Preserving Dual-Perturbation Adaptation(SPDA), a resource-efficient OTTA framework tailored for promptablemedical segmentation. SPDA constructs diverse and robust learning sig-nals by enforcing feature and prediction consistency across two semantic-preserving perturbations: (i) a domain-adaptive low-frequency radial-spectrum perturbation that simulates task-irrelevant intra-sub-domainvariations without out-of-domain distortions, and (ii) a prompt under-sampling perturbation that reduces prompt density while preservingtarget semantics. These consistency constraints drive the optimizationof a lightweight post-encoder adapter, facilitating efficient adaptationwith minimal computational overhead. Extensive experiments on varied2D/3D medical data demonstrate that SPDA consistently outperformsstate-of-the-art OTTA baselines under box and point prompts, whileusing substantially fewer resources. Ablation studies validate each com-ponent and indicate the suitability for real-time clinical deployment.

Long-form narrative-to-film generation requires shot-level con-trollability and cross-clip consistency in both visual identity and charac-ter behavior—requirements that remain difficult to satisfy with currentprompt-based workflows. A core reason existing workflows remain brit-tle is the lack of a structured intermediate layer between scripts andvideo models, especially when screenplays are underspecified at key cin-ematic decision points. We introduce a structured orchestration layer forfilm-oriented script-to-video generation, implemented as a multi-agentframework that operates between scripts and off-the-shelf video genera-tors. The layer is centered on FilmDSL, a film-oriented domain-specificlanguage that makes cinematic constraints explicit, including shot andcamera directives, asset and continuity requirements, and persona cues,so that agents coordinate through a shared structured specification forplanning, generation, critique, and repair. Specifically, a generation agentconstructs asset packs and storyboard keyframes that anchor composi-tion before clip-by-clip synthesis, while a critic agent produces structuredQA signals and triggers targeted refinement without retraining the basemodel. Experiments on TV-style segments show improved controllabilityand consistency over text-only and reference-only baselines.

Large multimodal models have achieved strong reasoning oncomplex visual tasks, but their inference efficiency is often restricted bylong chains of thought. A promising solution is to pair a small draftmodel with a large target model, enabling cooperative inference employ-ing a routing signal that adaptively routes queries to either the draft ortarget model based on their difficulties for optimal efficiency and accu-racy. Yet, the remaining bottleneck is to establish a reliable query dif-ficulty signal under multimodal settings. Existing approaches designedfor language models either rely on post-hoc token probabilities, whichfall short in multimodal scenarios, or depend on supervised fine-tuning,which is a data-sensitive strategy. Both paradigms perform routing onlyafter a complete output, and ignore whether the target model can ac-tually solve the routed instances. To address this, we propose PRP,a Proactive Routing Paradigm that enables early decision-making byjointly evaluating the competence of both the draft and target models.Our Draft Rating Learning (DRL) equips the draft model with an inter-nal confidence estimator, while Joint Rating Learning (JRL) predictshow well the target model can handle a given query, thereby prioritiz-ing the allocation of samples it excels at rather than the hardest ones.These ratings enable fine-grained, instance-level Proactive Routingand substantially accelerate inference without compromising overall per-formance. Extensive experiments across multiple multimodal reasoningbenchmarks validate our effectiveness and efficiency.

Diffusion Transformers have become the dominant paradigmin generative AI, but their high computational costs severely hinderreal-time applications. Prediction-based feature caching is widely usedto accelerate diffusion transformers; however, as the number of stepsincreases, the deviation between its predictions and the reference full-compute trajectory gradually grows. An intuitive idea is to use an onlineregression model to dynamically correct this deviation, but it faces theissue of label data being unavailable during the acceleration process.This paper presents a statistical observation that the residuals betweenthe features of full computation steps using caching methods and refer-ence full-compute trajectory locally exhibit a zero-mean Gaussian dis-tribution. By treating the features of full computation steps as noisyobservations of reference features, the data acquisition problem is re-solved. Based on this observation, a plug-and-play GP-Refiner correctionframework is proposed. This method utilizes Gaussian Process Regres-sion for correction and, leveraging the properties of GPR, introducesan uncertainty-adaptive computation strategy that triggers necessaryfull-computation calibration by monitoring the posterior variance in realtime. Experiments demonstrate significant improvements across differentmodels when combined with various state-of-the-art methods. Integrat-ing the proposed framework with TaylorSeer reduces the computationalload by 19.3% while improving PSNR by 0.9 dB and reducing LPIPSfrom 0.46 to 0.29. Code is available in https://github.com/Aredstone/GP-Refiner.

Transferring articulated motion from monocular videos torigged 3D characters is challenging due to pose ambiguity in 2D obser-vations and morphological differences between source and target. Ex-isting approaches often follow a reconstruct-then-retarget paradigm, ty-ing transfer quality to intermediate 3D reconstruction and limiting ap-plicability to categories with parametric templates. We propose Mor-phGS, a framework that formulates motion retargeting as a target-drivenanalysis-by-synthesis problem, directly optimizing target morphology andpose through image-space supervision. A rig-coupled morphology pa-rameterization factorizes character identity from time-varying joint rota-tions, while dense 2D-3D correspondences and synthesized views providecomplementary structural and multi-view guidance. Experiments on syn-thetic benchmarks and real-world videos show consistent improvementsover baselines. Project page: https://xodus777.github.io/MorphGS/