ECCV 2026 Papers

Text-to-image models produce images that align well withnatural language prompts, but compositional generation has long beena central challenge. Models often struggle to satisfy multiple conceptswithin a single prompt, frequently omitting some concepts and resultingin partial success. Such failures highlight the difficulty of jointly op-timizing multiple concepts during reward optimization, where compet-ing concepts can interfere with one another. To address this limitation,we propose Correlation-Weighted Multi-Reward Optimization (CMO),a framework that leverages the correlation structure among concept re-wards to adaptively weight each attribute concept in optimization. Byaccounting for interactions among concepts, CMO balances competingreward signals and emphasizes concepts that are partially satisfied yetinconsistently generated across samples, improving compositional gener-ation. Specifically, we decompose multi-concept prompts into pre-definedconcept groups (e.g., objects, attributes, and relations) and obtain re-ward signals from dedicated reward models for each concept. We thenadaptively reweight these rewards, assigning higher weights to conflictingor hard-to-satisfy concepts using correlation-based difficulty estimation.By focusing optimization on the most challenging concepts within eachgroup, CMO encourages the model to consistently satisfy all requestedattributes simultaneously. We apply our approach to train state-of-the-art diffusion models, SD3.5 and FLUX.1-dev, and demonstrate consis-tent improvements on challenging multi-concept benchmarks, includingConceptMix, GenEval 2, and T2I-CompBench.

As audio-visual generative models evolve into world simula-tors, cross-modal synchronization stands as a critical proxy for assessingthe consistency of world dynamics and causality in generated content.However, existing evaluation metrics presume structural correctness, re-ducing synchronization to mere temporal alignment. Consequently, theyfail on generative outputs, especially when exhibiting structural hallu-cinations and asymmetric cross-modal relations, which currently man-date expert human annotation to assess synchronization. Thisdependency introduces a critical paradox: human evaluators rely on rel-ative, reference-dependent comparisons, whereas automated metrics re-quire reference-free, absolute scalars. We resolve this paradox by propos-ing a framework that distills relative human perception into a continu-ous, globally consistent metric. First, we introduce SynthSync, a datasetof generative failures ranked via pairwise human annotations. Second,we adapt the Omni-LLM equipped with a continuous latent projectionto translate relative human rankings into continuous absolute values.Third, we propose Real-Valued Group Relative Policy Optimization (R-GRPO) to internalize the global causal structure of synchronization vialistwise score distributions. Empirically, our metric achieves state-of-the-art human preference alignment. We leverage this estimator to establisha standardized benchmark, advancing AV-Gen assessment from low-levelsignal correlation to visually grounded causality.

Cellular vehicle-to-everything (C-V2X) enables cooperativeperception, prediction, and planning beyond the field of view of individualagents. However, existing datasets often overlook the complexities of real-world deployment, such as limited communication bandwidth and its dy-namics, heterogeneous sensing modalities, and scalability beyond a singlecooperative partner. In this paper, we introduce CooperScene, a high-fidelity cooperative autonomy dataset with real-world C-V2X communica-tion characterization. The dataset is organized into diverse scenes, includ-ing intersections, highway ramps, and parking lots. These scenes involvethree connected and autonomous vehicles (CAVs) and one infrastructureroadside unit (RSU), all equipped with multi-modal sensors and commer-cial off-the-shelf C-V2X communication radios. All scenes are annotatedwith globally consistent 3D labels at 10 Hz, totaling 344K objects across59K frames, underpinned by tight sensor- and agent-synchronization,centimeter-level localization and spatial alignment, precise cross-modalitycalibration, and 3GPP-standard-compliant C-V2X communication. Coop-erScene establishes a rigorous benchmark for evaluating multi-agentscaling and actual performance in real-world deployable settings. Projectwebsite for data and benchmark: https://cisl.ucr.edu/CooperScene.

Physics-based Human-Scene Interaction (HSI) imitation learn-ing is crucial for embodied intelligence as it bridges the gap betweenkinematic 3D motions and real-world dynamics. However, most exist-ing methods focus on simplified scene settings, leaving complex envi-ronments largely unexplored, which limits their applicability in real-world scenarios. In this paper, we focus on HSI mimicry in complexenvironments. Under this complex setting, we observe an inherent trade-off between successfully performing interaction and maintaining natu-ral, physically plausible motions. To address this challenge, we proposeComplexMimic, a framework that reconstructs diverse HSI by inter-preting imperfect MoCap data. First, we introduce a Dual Flow Strat-egy, which learns two complementary experts: an imitation expert foraccurate motion tracking and an interaction expert for collision-awareadaptation in complex scenes. Second, naive multi-expert distillation,which treats all experts equally, often under-samples challenging be-haviors, limiting effective learning. To mitigate this issue, we proposea difficulty-aware distillation strategy that adaptively weights super-vision and prioritizes hard-yet-learnable trajectories guided by failurestatistics and learning progress signals. Extensive experiments on threebenchmark datasets demonstrate that our approach outperforms cur-rent state-of-the-art methods. Our implementation is available at https://github.com/LuPan23/ComplexMimic.

Pose-free feed-forward 3D Gaussian Splatting enables novelview synthesis from uncalibrated multi-view images. Although more viewsshould improve performance, existing methods often degrade with dense-view inputs because global aggregation spreads attention over many to-kens, and naive voxel fusion averages many Gaussians into overly smoothrepresentations. We present AVSplat, a framework that turns additionalviews into reliable signals for both aggregation and representation. Be-fore global attention, each view performs a single lightweight interactionwith a small set of Assist Views chosen for relevance and diversity, andthe cached features provide a focused scene context that stabilizes corre-spondence. For representation, we use adaptive temperature-aware voxelfusion that sharpens attribution under high occupancy, guided by oc-cupancy and point confidence. Crucially, AVSplat restores positive viewscaling where performance remains stable or improves as more inputviews are added, instead of degrading in the dense-view regime. Abla-tions show that Assist View Preconditioning is primarily responsible forpreventing dense-view degradation, while Occupancy-guided Voxel Fu-sion contributes most of the single-point image-quality gains.

Existing red-teaming studies on GUI agents face two funda-mental limitations: adversarial perturbations require white-box accessunavailable in commercial deployments, while prompt injection is increas-ingly neutralized by stronger safety alignment. To study robustness undera more practical threat model, we propose Semantic-level UI ElementInjection, a black-box red-teaming paradigm that overlays safety-alignedand harmless UI elements onto screenshots to misdirect the agent’s visualgrounding. Our method couples a modular Editor–Overlapper–Victimpipeline with iterative search that samples multiple candidate edits, keepsthe best cumulative overlay, and adapts future prompt strategies basedon previous failures. Experiments across 19 victim models spanning 8model families show that strategic optimization substantially outperformsrandom injection (3.5–6.9× on the most robust victims) and transfersnear-perfectly across architectures, confirming model-agnostic visual-semantic vulnerabilities. After the first successful attack, the victim stillclicks the attacker-controlled icon in over 15% of subsequent independenttrials versus below 1% for random injection, establishing that strategicallyplaced icons act as persistent attractors that causally redirect groundingrather than introducing incidental clutter. Public code is available athttps://github.com/HashTAG00002/UI-Injection.

Autonomous personal mobility benefits greatly from the abil-ity to predict executable spatial goals from language and front view imagealone. In this study, we focus on absolute target position prediction froma language instruction and front view image, which requires referring ex-pression disambiguation and absolute target localization without depthcues. To address this, we propose AiSCREAM, a language-conditionedtarget localization model based on cross-view vision-language reason-ing. AiSCREAM constructs cross-view geometric semantic alignment be-tween the front and an Aerial Semantic view, which is an overhead-viewimage synthesized from the front image via a text-conditioned image-to-image generation model. By preserving appearance-level scene charac-teristics, the alignment can leverage language-conditioned semantic cuesand enable reliable absolute target localization from a single image. Toevaluate AiSCREAM, we constructed DRAMATiST, a benchmark thatintroduces road and traffic diversity. The experimental results demon-strated that AiSCREAM achieved a mean absolute position error of4.03 m on DRAMATiST, which outperformed baseline methods and hu-man performance. The project page is available at https://ai-scream-project-page.vercel.app/ .

Vision-and-Language Navigation (VLN) requires an embod-ied agent to ground complex natural-language instructions into long-horizon navigation in unseen environments. While Vision-Language Mod-els (VLMs) offer strong 2D semantic understanding, current VLN sys-tems remain constrained by limited spatial perception, 2D–3D repre-sentation mismatch, and monocular scale ambiguity. In this paper, wepropose AgentVLN, a novel and efficient embodied navigation frameworkthat can be deployed on edge computing platforms. We formulate VLNas a Partially Observable Semi-Markov Decision Process (POSMDP)and introduce a VLM-as-Brain paradigm that decouples high-levelsemantic reasoning from perception and planning via a plug-and-playskill library. To resolve multi-level representation inconsistency, we de-sign a cross-space representation mapping that projects perception-layer3D topological waypoints into the image plane, yielding pixel-alignedvisual prompts for the VLM. Building on this bridge, we integrate acontext-aware self-correction and active exploration strategy to recoverfrom occlusions and suppress error accumulation over long trajectories.To further address the spatial ambiguity of instructions in unstructuredenvironments, we propose a Query-Driven Perceptual Chain-of-Thought(QD-PCoT) scheme, enabling the agent to actively query and acquiregeometric depth information according to task demands. Finally, we con-struct AgentVLN-Instruct, a large-scale instruction-tuning dataset withdynamic stage routing conditioned on target visibility. Extensive experi-ments show that AgentVLN consistently outperforms prior state-of-the-art (SOTA) methods on long-horizon VLN benchmarks, offering a prac-tical paradigm for lightweight deployment of next-generation embodiednavigation models. Code: https://github.com/Allenxinn/AgentVLN.

Human demonstrations are widely considered the corner-stone of end-to-end (E2E) autonomous driving despite human demon-stration’s scarcity for long-tail and safety-critical scenarios. Nonetheless,current E2E autonomous driving (AD) training paradigms continue torely on human demonstrations. Imitation learning (IL) requires humandemonstrations for training, whereas reinforcement learning (RL) hasemerged as a promising alternative to reduce this dependency. How-ever, most existing RL methods for E2E AD still rely implicitly on hu-man demonstrations. A pure rewards-based RL method can overcomethe need for human demonstrations, but general RL policy gradientmethods suffer from the cold-start problem. In this paper, we proposeZTRS (Zero-human demonstration end-to-end autonomous driving withTRajectory Scorer) — a complete RL-based E2E planning paradigmtrained solely on real-world images and rule-based rewards, entirely with-out human demonstration. Through our proposed Exhaustive PolicyOptimization (EPO), a policy gradient variant tailored for enumer-able trajectory actions and dense supervision, ZTRS enables the modelto generalize better to long-tail driving scenarios. We demonstrate thisgeneralization through our SOTA performance against IL approaches onboth long-tail Navhard and closed-loop HUGSIM datasets. Project page:https://zhenxinli.net/ZTRS/.

Rolling shutter (RS) cameras are widely used in consumerdevices, but their row-wise exposure causes distortions under motion,making geometric 3D vision problems dependent on both camera in-trinsics and readout time ratio. Existing RS calibration methods relyon calibration targets or specialised hardware, limiting their use in un-constrained settings. We present the first self -calibration method forRS cameras that directly estimates camera intrinsics and the readouttime ratio from image sequences, without requiring calibration targets.The method is implemented as a self-calibrating bundle adjustment(BA), which critically depends on the RS imaging model. We com-bine two known complementary models. The first formulates RS imag-ing as continuous-time trajectory estimation under a row-wise pose rep-resentation. The second interprets RS images as temporally distortedglobal shutter (GS) images and requires to estimate correction fields.The combination is non-trivial and results in a unified dual-projectionmodel, in which each 3D point is simultaneously constrained at both row-dependent and reference timestamps along a shared continuous trajec-tory, enforcing stronger geometric and temporal consistency. Extensivesimulations analyse the applicability of several implementations undervarying conditions, and real data experiments demonstrate the accuracy,robustness, and practical effectiveness of the proposed approach.

Multimodal Large Language Models (MLLMs) have demon-strated strong image-level visual understanding and reasoning, yet theirpixel-level perception across both images and videos remains limited.Foundation segmentation models such as the SAM series produce high-quality masks, but they rely on low-level visual prompts and cannotnatively interpret complex conversational instructions. Existing segmen-tation MLLMs narrow this gap, but are usually specialized for eitherimages or videos and rarely support both textual and visual promptsin one interface. We introduce X2SAM, a unified segmentation MLLMthat extends any-segmentation capabilities from images to videos. Givenconversational instructions and visual prompts, X2SAM couples an LLMwith a Mask Memory module that stores guided vision features for tem-porally consistent video mask generation. The same formulation sup-ports generic, open-vocabulary, referring, reasoning, grounded conversa-tion generation, interactive, and visual grounded segmentation across im-age and video inputs. We further introduce the Video Visual Grounded(V-VGD) segmentation benchmark, which evaluates whether a modelcan segment object tracks in videos from interactive visual prompts.With a unified joint training strategy over heterogeneous image and videodatasets, X2SAM delivers strong video segmentation performance, re-mains competitive on image segmentation benchmarks, and preservesgeneral image and video chat ability. Code is available at https://github.com/wanghao9610/X2SAM.

Generating realistic cast shadows for inserted foreground ob-jects is a crucial yet challenging problem in image composition, wheremaintaining geometric consistency between objects and their shadows incomplex scenes remains difficult due to the ill-posed nature of shadowformation. To address this challenge, we propose VSDiffusion, a visibility-constrained two-stage framework that narrows the solution space by in-corporating visibility priors. In Stage I, a coarse shadow mask is pre-dicted to localize plausible shadow regions. In Stage II, conditional dif-fusion guided by lighting and depth cues estimated from the compositeimage is used to generate accurate shadows. Within VSDiffusion, visibil-ity priors are injected through two complementary pathways: (1) a vis-ibility control branch with shadow-gated cross-attention that providesmulti-scale structural guidance, and (2) a learned soft prior map thatreweights the training loss in error-prone regions to encourage geometriccorrection. In addition, we introduce a high-frequency guided enhance-ment module to sharpen shadow boundaries and improve texture inter-action with the background. Extensive experiments on the widely usedDESOBAv2 benchmark demonstrate that VSDiffusion produces geomet-rically consistent shadows in complex scenes and establishes new state-of-the-art results across most evaluation metrics. The code is availableat https://github.com/Jadelingli/VSDiffusion.

Vision-language models (VLMs) have demonstrated remark-able capabilities in understanding and reasoning about visual content,but significant challenges persist in tasks requiring cross-viewpoint under-standing and spatial reasoning. We identify a critical limitation: currentVLMs excel primarily at egocentric spatial reasoning (from the cam-era’s perspective) but fail to generalize to allocentric viewpoints whenrequired to adopt another entity’s spatial frame of reference. We introduceViewSpatial-Bench, a dedicated multi-perspective spatial localizationbenchmark covering both camera-perspective and human perspective rea-soning, supported by an automated 3D annotation pipeline that generatesprecise directional labels. Comprehensive evaluation of diverse VLMs onViewSpatial-Bench reveals a significant performance disparity: modelsdemonstrate reasonable performance on camera-perspective tasks butexhibit reduced accuracy when reasoning from a human viewpoint. Byfine-tuning VLMs on our multi-perspective spatial dataset, we achievean overall performance improvement of 46.24% across tasks, confirmingthe strong latent spatial reasoning capacity of VLMs under targetedsupervision. Our work establishes a crucial benchmark for spatial intel-ligence in embodied AI systems and provides empirical evidence thatmodeling 3D spatial relationships enhances VLMs’ corresponding spatialcomprehension capabilities.Our benchmark is available at HuggingFace.

Motion, speech, and sound effects are fundamental elementsof human-centric videos, yet their heterogeneous temporal characteris-tics make joint generation highly challenging. Existing audio-video gen-eration models often fail to maintain consistent alignment across thesemodalities, leading to noticeable mismatches between motion, speech,and environmental sounds. We present Unison, a unified framework thatexplicitly promotes coherence across the motion, speech, and sound modal-ities. Within the audio stream, Unison employs a semantic-guided har-monization strategy that decouples the generation of speech and sound-effect components. Leveraging bidirectional audio cross-attention andsemantic-conditioned gating for semantic-driven adaptive recomposition,this approach effectively mitigates speech dominance and enhances acous-tic clarity. For audio–motion synchronization, we propose a bidirectionalcross-modal forcing strategy where the cleaner modality guides the nois-ier one through decoupled denoising schedules, reinforced by a progres-sive stabilization strategy. Extensive experiments demonstrate that Uni-son achieves state-of-the-art performance in both audio perceptual qual-ity and cross-modal synchronization, highlighting the importance of ex-plicit multimodal harmonization in human-centric video generation.

Traditional Shot Boundary Detection (SBD) inherently strug-gles with complex transitions by formulating the task around isolated cutpoints, frequently yielding corrupted video shots. We address this fun-damental limitation by formalizing the Shot Transition Detection (STD)task. Rather than searching for ambiguous points, STD explicitly detectsthe continuous temporal segments of transitions. To tackle this, we pro-pose TransVLM, a Vision-Language Model (VLM) framework for STD.Unlike regular VLMs that predominantly rely on spatial semantics andstruggle with fine-grained inter-shot dynamics, our method explicitly in-jects optical flow as a critical motion prior at the input stage. Through asimple yet effective feature-fusion strategy, TransVLM directly processesconcatenated color and motion representations, significantly enhancingits temporal awareness without incurring any additional visual tokenoverhead on the language backbone. To overcome the severe class im-balance in public data, we design a scalable data engine to synthesizediverse transition videos for robust training, alongside a comprehensivebenchmark for STD. Extensive experiments demonstrate that TransVLMachieves superior overall performance, outperforming traditional heuris-tic methods, specialized spatiotemporal networks, and top-tier VLMs.

Reconstructing 3D scene structures from sparse, low-overlapobservations remains a fundamental challenge in autonomous driving.Recent state-of-the-art frameworks achieve promising results by incor-porating voxel-based Gaussians, but incur substantial computational re-dundancy due to a uniform volumetric processing strategy. To bridgethe gap between the efficiency of pixel-based Gaussian methods and thestructural completeness of voxel-based Gaussian approaches, we proposeFocusGS, a simple yet effective framework that shifts the paradigmfrom global densification to targeted structural completion. Our centralinsight is that structural completion should be decoupled from determin-istic regions, with computation concentrated exclusively on areas exhibit-ing geometric ambiguity. Specifically, FocusGS addresses the localizationchallenge by deriving a 3D Geometric Ambiguity Manifold to accuratelyisolate localized areas prone to occlusion and high geometric uncertainty.To overcome the subsequent manifold completion challenge, we design alightweight targeted structure completion module that selectively instan-tiates and optimizes continuous Gaussian queries strictly within this un-structured, sparse topological subspace. Extensive experiments demon-strate that FocusGS achieves a superior efficiency-quality trade-off, ad-vancing state-of-the-art performance on driving-centric benchmarks whilenaturally reducing the total number of Gaussians by ∼74% and decreas-ing rendering time by ∼34%.

Simulation provides a low-cost, scalable pathway to largescale robotic manipulation data collection. However, existing 3D scene generation methods can rarely be applied directly to manipulation data synthesis, as their generated scenes often lack instance-level interactivity and physical plausibility. Focusing on tabletop manipulation, we propose TabletopGen, a training-free and automated tabletop scene generation and interactive simulation engine. Starting from text or a single image, we first obtain independent 3D object models via generative instance extraction. Second, we introduce a novel pose and scale alignment approach that recovers a collision-free scene layout using a Differentiable Rotation Optimizer and a Top-View Spatial Alignment mechanism. Finally, we assemble the generated scene in a physics simulator with collision geometry, yielding a stable, interactable environment for synthesizing multimodal manipulation data. Extensive experiments and user studies demonstrate that TabletopGen achieves state-of-the-art performance in visual fidelity, layout accuracy, and physical plausibility. Furthermore, we validate the executability of the collected trajectories on a real robotic arm via zeroshot real-to-sim-to-real policy transfer, indicating that TabletopGen can serve as a reliable data engine for robotic manipulation data synthesis.

Diffusion transformer (DiT) architectures show great poten-tial for real-world image super-resolution (Real-ISR). However, theircomputationally expensive iterative sampling necessitates one-step dis-tillation. Existing one-step distillation methods struggle with Real-ISRon DiT. They suffer from fundamental trajectory mismatch and gen-erate severe grid-like periodic artifacts. To tackle these challenges, wepropose StrSR, a novel one-step adversarial distillation framework fea-turing spectral and trajectory regularization. Specifically, we proposean asymmetric discriminative distillation architecture to bridge the tra-jectory gap. Additionally, we design a frequency distribution matchingstrategy to effectively suppress DiT-specific periodic artifacts caused byhigh-frequency spectral leakage. Extensive experiments demonstrate thatStrSR achieves state-of-the-art performance in Real-ISR, across bothquantitative metrics and visual perception. The code and models will bereleased at https://github.com/jkwang28/StrSR.

Semi-supervised few-shot learning (SSFSL) resembles real-world applications such as “auto-annotation”, as it aims to learn a modelfrom a few labeled and abundant unlabeled task-specific examples to an-notate the unlabeled ones. Despite the availability of powerful open-sourceVision-Language Models (VLMs) and open-world data, existing SSFSLliterature largely neglects these resources. In contrast, the related areafew-shot learning (FSL) has already exploited them to boost performance.Arguably, to solve real-world auto-annotation, SSFSL should leverage suchopen resources. To bridge this gap, we explore established SSL methodsto finetune a VLM. Unexpectedly, they significantly underperform FSLbaselines that do not use unlabeled data. Our in-depth analysis revealsthe root cause of failure: VLMs produce “flat” distributions of softmaxprobabilities, resulting in zero utilization of unlabeled data and weak su-pervision signals. To address this challenge, we propose an embarrassinglysimple solution that uses temperatures to sharpen the softmax output,which not only increases the confidence scores of pseudo-labels to improvethe utilization of unlabeled data, but also strengthens training supervisionfor effective finetuning. Furthermore, we exploit task-relevant open data,e.g., those retrieved from VLMs’ publicly available pretraining set. Tomitigate the imbalance and domain gaps in retrieved data, we employa stage-wise training strategy. Building on the successful finetuning ofVLMs and the exploitation of open data, we present a simple yet effectiveSSFSL method, Stage-Wise Finetuning with Temperatures (SWIFT).Across five benchmarks, SWIFT outperforms recent FSL and SSL meth-ods by ∼5 accuracy points. SWIFT even rivals supervised learning, whichfinetunes a VLM assuming unlabeled data having ground-truth labels!