ECCV 2026 Papers

Reinforcement learning for end-to-end autonomous drivingin closed-loop simulations is gaining increasing attention, yet most sim-ulators differ substantially from real-world conditions, leading to a sig-nificant sim2real gap. Recent methods use scene reconstruction to buildphotorealistic simulators, improving sensor realism but remaining con-strained by the training data distribution, which limits their ability torender novel trajectories and corner cases. To address this, we proposeReconDreamer-RL, a framework that integrates video diffusion priorsinto scene reconstruction for reinforcement learning, enabling more re-alistic and diverse closed-loop autonomous driving training. Specifically,we introduce ReconSimulator, which combines video diffusion priors forappearance modeling with a kinematic model for physical modeling, re-constructing driving scenarios from real-world data and narrowing thesim2real gap. To cover more corner cases, we propose the Dynamic Ad-versary Agent (DAA), which adjusts surrounding vehicles’ trajectoriesrelative to the ego vehicle to generate challenging scenarios such as cut-ins. Finally, we introduce the Cousin Trajectory Generator (CTG) tomitigate the bias of training trajectories toward simple straight-line mo-tion. Experiments show that ReconDreamer-RL improves end-to-end au-tonomous driving training and outperforms imitation learning methodswith a 5× reduction in Collision Ratio.

Recent di_x001B_usion-based super-resolution (SR) methods oftenemploy large text-to-image (T2I) backbones as generic priors. While thisyields strong perceptual quality, restoration performance can becomeoverdependent on text-driven semantics. T2I pipelines typically rely onclassi_x001C_er-free guidance (CFG), doubling the number of function evalu-ations at inference time. To address these limitations, we propose theReference-to-Image Aware Latent Di_x001B_usion Model for Image Super-Resolution (ReAL), a purely Reference-to-Image (R2I) model condi-tioned only on the LR input and a retrieved reference. The ReferenceFusion Module encodes the reference once, caches its key_x0015_value tensors,and injects them into the denoising model's self-attention layers at everytimestep, providing strong feature-level guidance for texture and struc-ture recovery. The R2I design substantially reduces dependence on T2Ipriors and improves semantic consistency, achieving state-of-the-art per-formance compared to T2I-based SR baselines.

Cross-identity character animation aims to drive a targetidentity (from a reference image) to follow the motion of a source char-acter (from a driving video). The core challenge lies in the inherent en-tanglement of two essential capabilities: cross-identity spatial mapping-aligning position, scale, and skeletal proportions between the referenceand the driving pose-and subsequent motion control-refining joint articu-lation, volumetric consistency, and view coherence during generation. Inthis paper, we introduce Reference-Aware Structural Alignment(RASA), a novel framework that systematically disentangles spatialmapping from motion control by injecting structured priors into a Diffu-sion Transformer (DiT). Our approach operates in two complementarystages. First, a Spatial Prior Calibrator (SPC) fuses the referenceidentity with the driving pose to generate an initial noise latent thatis spatially grounded—it ensures the target character is correctly po-sitioned, scaled, and proportionally aligned with the driving skeleton.This resolves cross-identity spatial mismatches at the very start of gen-eration. Second, to achieve identity-agnostic motion control, we proposean Inherent Motional Guider (IMG). Moving beyond appearance-biased 2D keypoints, IMG encodes shape-agnostic SMPL articulationparameters into a semantic motion vector. Injected into the interme-diate layers of the DiT, this vector serves as complementary guidancethat works in tandem with the base pose condition, providing anatomi-cally consistent joint articulation and view-aware volumetric refinement.To rigorously evaluate this challenging task, we curate CIM-Bench, ahigh-quality benchmark with rigorous manual curation. Extensive exper-iments demonstrate that RASA significantly outperforms state-of-the-artmethods in both motion fidelity and visual quality. Our work establishesa new paradigm for cross-identity animation, showing that disentan-gled spatial and motional priors are key to achieving robust and con-sistent character animation. Our project page is at https://hidream-ai.github.io/RASA/.

Audio-Visual Question Answering (AVQA) extends classical VQA by requiring joint reasoning over video and synchronized audio. However, many AVQA systems rely on deeply stacked layers of selfand cross attention across text, video, and audio. Such sequential stacking may incur loss of information such as subtle inter-modal cues over the layers, causing errors to accumulate across sequential attention layers during the fusion. We introduce Q-TriM which performs multi-modal fusion in a shallow and parallel manner instead of a deep and sequential manner. For Q-TriM, we propose a novel framework for attention operation incorporating video and audio conditioned on text. As a result, we obtain not only standard cross attention outputs but also Tri-Modal Attention representations in which Query, Key, and Value come from distinct modalities. These attention representations are combined in parallel at a single stage, thus avoiding the multi-modal fusion with deep stacks in order to mitigate error accumulation and depth-induced issues. QTriM achieves state-of-the-art performance on three AVQA benchmarks, including substantial gains on MUSIC-AVQA-R, which demonstrates its robustness and out-of-distribution generalization. Code is available at https://github.com/Sunghun95/Q-TriM

Token pruning has emerged as an essential technique for enabling efficient and resilient inference for foundation models, showing great potential in accelerating the Reinforcement Learning (RL) optimization of Vision–Language–Action (VLA) Models. However, our indepth analysis indicates that directly transferring prior pruning strategies to RL is non-trivial. Inference-time pruning introduces a train–test discrepancy, whereas dynamically adjusting the pruning configuration leads to inconsistent policy rollout and improvement. Therefore, we propose PARL-VLA, a pruning-aware RL framework that enforces explorationexploitation pruning consistency by recording the rollout pruning configuration and replaying it during policy updates, enabling end-to-end co-adaptation between token pruning and policy optimization. PARLVLA further trains across a spectrum of token budgets to learn a single policy that remains reliable under different information budgets at test time. On LIBERO and LIBERO-Plus, PARL-VLA discards 55% of visual tokens, speeds up rollout policy forward by about 1.3× and policy updates by about 1.7×, and improves LIBERO-Plus robustness by up to 6.6 points (3.2 on average) without sacrificing in-distribution success. On RoboTwin2.0, PARL-VLA achieves 46.9% real-robot success over 405 trials, improving over RLinf at 37.0%.

Grounding has become a fundamental capability of vision-language models (VLMs). Most existing VLMs point by generating coor-dinates as part of their text output, which requires learning a complicatedcoordinate system and results in a high token count. Instead, we pro-pose a more intuitive pointing mechanism that directly selects the visualtokens that contain the target concept. Our model generates a specialpointing token that cross-attends to the input image or video tokens andselects the appropriate one. To make this model more fine-grained, we fol-low these pointing tokens with an additional special token that selects afine-grained subpatch within the initially selected region, and then a thirdtoken that specifies a location within that subpatch. We further showthat performance improves by generating points sequentially in a consis-tent order, encoding the relative position of the previously selected point,and including a special no-more-points class when selecting visual to-kens. Using this method, we set a new state-of-the-art on image pointing(70.7% on PointBench), set a new state-of-the-art for fully open modelson GUI pointing (61.1% on ScreenSpotPro), substantially improve VLMvideo tracking (62.5 on J &F vs 56.7 for Molmo2 on Molmo2Track), andimprove video pointing (59.1% human preference win rate vs. Molmo2).We additionally show that our method significantly improves learningefficiency and discuss the qualitative differences that emerge with thisdesign change. Our model weights, new datasets, and source code areavailable at https://allenai.org/blog/molmopoint.

Weakly supervised semantic segmentation aims to achievepixel-level predictions using image-level labels. Existing methods typi-cally entangle semantic recognition and object localization, which oftenleads models to focus exclusively on sparse discriminative regions. Al-though foundation models show immense potential, many approachesstill follow the tightly coupled optimization paradigm, struggling to ef-fectively alleviate pseudo-label noise and often relying on time-consumingmulti-stage retraining or unstable end-to-end joint optimization. To ad-dress the above challenges, we present ModuSeg, a training-free weaklysupervised semantic segmentation framework centered on explicitly de-coupling object discovery and semantic assignment. Specifically, we in-tegrate a general mask proposer to extract geometric proposals withreliable boundaries, while leveraging semantic foundation models to con-struct an offline feature bank, transforming segmentation into a non-parametric feature retrieval process. Furthermore, we propose seman-tic boundary purification and soft-masked feature aggregation strate-gies to effectively mitigate boundary ambiguity and quantization errors,thereby extracting high-quality category prototypes. Extensive experi-ments demonstrate that the proposed decoupled architecture better pre-serves fine boundaries without parameter fine-tuning and achieves highlycompetitive performance on standard benchmark datasets. Code is avail-able at https://github.com/Autumnair007/ModuSeg.

Recent advances in Diffusion Transformers (DiTs) have en-abled high-quality joint audio-video generation, producing videos withsynchronized audio within a single model. However, existing control-lable generation frameworks are typically restricted to video-only con-trol. This restricts comprehensive controllability and often leads to sub-optimal cross-modal alignment. To bridge this gap, we present MM-Control, which enables users to perform Multi-Modal Control in jointaudio-video generation. MMControl introduces a dual-stream conditionalinjection mechanism. It incorporates both visual and acoustic controlsignals—including reference images, reference audio, depth maps, andpose sequences—into a joint generation process. These conditions are in-jected through bypass branches into a joint audio-video Diffusion Trans-former, enabling the model to simultaneously generate identity-consistentvideo and timbre-consistent audio under structural constraints. Further-more, we introduce modality-specific guidance scaling, which allows usersto independently and dynamically adjust the influence strength of eachvisual and acoustic condition at inference time. Extensive experimentsdemonstrate that MMControl achieves fine-grained, composable controlover character identity, voice timbre, body pose, and depth-guided scenestructure in joint audio-video generation.

We introduce a novel self-supervised learning framework thatautomatically learns representations from input computer-aided design(CAD) models for downstream tasks, including part classification, mod-eling segmentation, and machining feature recognition. To train our net-work, we construct a large-scale, unlabeled dataset of boundary repre-sentation (BRep) models. The success of our algorithm relies on two keycomponents. The first is a masked graph autoencoder that reconstructsrandomly masked geometries and attributes of BReps for representa-tion learning to enhance the generalization. The second is a hierarchi-cal graph Transformer architecture that elegantly fuses global and locallearning by a cross-scale mutual attention block to model long-range ge-ometric dependencies and a graph neural network block to aggregatelocal topological information. After training the autoencoder, we replaceits decoder with a task-specific network trained on a small amount oflabeled data for downstream tasks. We conduct experiments on varioustasks and achieve high performance, even with a small amount of labeleddata, demonstrating the practicality and generalizability of our model.Compared to other methods, our model performs significantly better ondownstream tasks with the same amount of training data, particularlywhen the training data is very limited.

High-fidelity simulation of mmWave radar signals for dynamichuman motion is valuable for developing radar-based human sensing mod-els; yet collecting accurately labeled measurements for a specific deploy-ment site remains expensive. We present HybridSim, a physics–learninghybrid simulator that synthesizes mmWave radar signals from dynamichuman meshes under a fixed indoor room configuration, explicitly de-coupling propagation into two components. To parameterize the humansubject, we use a tri-plane representation to extract human features anda Graph Convolutional Network to stabilize optimization and mitigategradient instability. The direct signal path is modeled via an inverse-rendering formulation with a microfacet BRDF to capture primary surfacereflections. In parallel, the indirect path is approximated by combining3D Gaussian Splatting with a virtual-receiver geometry to fit and repro-duce site-specific multipath interference patterns, achieving substantiallylower computational cost than explicit full ray tracing. Experiments ina fixed-room setting show improved agreement with a physically basedreference and consistent gains on downstream radar-based human sensingtasks when using HybridSim for site-specific data augmentation.

Interactive traffic simulation is a vital world model for au-tonomous driving. A central challenge in long-horizon simulation is mod-eling sustained multi-agent interactions, which is further exacerbatedby dynamic token cardinality as agents continuously enter and exit thescene. In this work, we propose that the solution lies in the synergybetween the architectural inductive biases and statistical priors of large-scale sequence models, e.g., Large Language Models (LLMs). Our prob-ing experiments reveal that the transferability of attention mechanismsand the distributional consistency between motion tokens and naturallanguage enable small-scale, heavily frozen LLMs to rapidly adapt to traf-fic modeling. Building on this insight, we introduce RosettaSim, a unifiedframework that projects scene topology, agent states, and spawning in-tents into a structured autoregressive stream with variable length, achiev-ing both strong short-term accuracy and stable long-horizon simulationfidelity. Furthermore, evaluating extended rollouts presents yet anotherhurdle, as one-to-one agent correspondence inevitably fades over time.To address this, we introduce Retrieval-based Traffic Evaluation (RTE),which retrieves semantically similar real-world scenarios as context-awarereference anchors. Experiments on the Waymo Open Sim Agent Chal-lenge (WOSAC) demonstrate that RosettaSim achieves state-of-the-artperformance in both short- and long-term simulation. Furthermore, RTEexhibits a stronger correlation with standard metrics (r = 0.83) than ex-isting approaches (r = 0.74), indicating improved alignment with long-horizon simulation fidelity.

Visual document retrieval requires rapidly locating relevantpages from large multi-modal corpora in response to user queries. Whilerecent methods powered by Multi-modal Large Language Models (MLLMs)show competitive accuracy, they suffer from prohibitive computationalcosts by applying intensive MLLM encoding to every single page. Mean-while, we observe that user queries are typically keyword-anchored, con-taining semantically rich words that are expected to appear directly inthe visible text of relevant pages, offering an efficient cue for quickly nar-rowing down candidate pages. Building on this insight, we propose Light-STAR, an efficient framework that decomposes visual document retrievalinto: 1) LLM-free Visual Selection, which utilizes content-grounded queryencoding to focus on informative words and employs LLM-free visual em-beddings to produce a high-recall candidate set; and 2) Vision-adaptiveSemantic Refinement, which further performs fine-grained semantic match-ing exclusively on these top candidates via adaptive region-wise featurefusion to effectively combine textual and layout cues, optimized through ahardness-aware contrastive objective. Experimental results demonstratethat LightSTAR achieves state-of-the-art retrieval accuracy while reduc-ing end-to-end latency by several-fold, offering a highly practical solutionto the accuracy-efficiency trade-off in visual document retrieval. Code isavailable at https://github.com/bokufa/LightSTAR.

Computational pathology needs whole-slide image (WSI) foun-dation models that transfer across diverse clinical tasks, yet current ap-proaches remain largely slide-centric, often depend on private data andexpensive paired-report supervision, and do not explicitly model relation-ships among multiple slides from the same patient. We present MOOZY,a patient-first pathology foundation model in which the patient case, notthe individual slide, is the core unit of representation. MOOZY explicitlymodels dependencies across all slides from the same patient via a casetransformer during pretraining, combining multi-stage self-supervisionwith scaled low-cost task supervision. In Stage 1, we pretrain a vision-only slide encoder on 77,134 public slide feature grids using masked self-distillation. In Stage 2, we align these representations with clinical se-mantics using a case transformer and multi-task supervision over 333tasks from 56 public datasets, including 205 classification and 128 sur-vival tasks across four endpoints. Across sixteen held-out tasks, MOOZYimproves macro weighted F1, balanced accuracy, and macro weightedROC-AUC relative to PRISM by +4.19%, +7.93%, and +6.95%, re-spectively. MOOZY is also parameter efficient with 85.77M parameters,14× smaller than GigaPath. These results suggest that patient-level pre-training yields transferable embeddings, providing a path toward scalablepatient-first histopathology foundation models.

Transfer-based adversarial attacks craft adversarial examplesusing surrogate models to mislead black-box victim models. Beyond per-turbation generation, transferability is fundamentally governed by thecoupling of initialization, surrogate adaptation, and gradient dynam-ics. We revisit this challenge from a Bilevel-Minimax perspective andinstantiate it in BMAT (Bilevel-Minimax Adversarial Transfer). Thebilevel formulation captures the dependency between initialization andperturbation, while the inner minimax problem promotes surrogate ro-bustness for cross-architecture generalization. Algorithmically, we designan integrated bottom-up solver that combines a Soft Weight Modulatorand an Implicit Gradient Approximator for ternary coupling interaction.We further provide theoretical insights into the optimization dynamicsof the proposed bilevel-minimax framework. Extensive experiments onclassification and segmentation benchmarks show that BMAT surpasses10+ strong baselines across 30+ victim models, improving both intra-and cross-architecture transfer, and yielding up to 2× mIoU reduction.Code is available at https://github.com/callous-youth/BMAT.

Counting serves as a simple but powerful test of a Large Vision-Language Model’s (LVLM) reasoning; it forces the model to identify each individual object and then add them all up. In this study, we investigate how LVLMs implement counting using controlled synthetic and real-world benchmarks, combined with mechanistic analyses. Our results show that LVLMs display a human-like counting behavior, with precise performance on small numerosities and noisy estimation for larger quantities. We introduce two novel interpretability methods, Visual Activation Patching and HeadLens, and use them to uncover a structured “counting circuit” that is largely shared across a variety of visual reasoning tasks. Building on these insights, we propose a lightweight intervention strategy that exploits simple and abundantly available synthetic images to fine-tune arbitrary pretrained LVLMs exclusively on counting. Despite the narrow scope of this fine-tuning, the intervention not only enhances counting accuracy on in-distribution synthetic data, but also yields an average improvement of +8.36% on out-of-distribution counting benchmarks and an average gain of +1.54% on complex, general visual reasoning tasks for Qwen2.5-VL. These findings highlight the central, influential role of counting in visual reasoning and suggest a potential pathway for improving overall visual reasoning capabilities through targeted enhancement of counting mechanisms.

Scene reconstruction with 3D Gaussian Splatting (3DGS) has become common, however deployment remains painful as the uncompressed file sizes can be massive. Current 3DGS compression systems combine multiple strategies for file size reduction, which can obscure where gains come from and limit component reuse across training pipelines. To make the gains more transparent, we propose KISS-GS, a modular compression pipeline named after the principle of keeping things simple, designed to decouple compression entirely from training. Given a 3DGS scene reconstructed with vanilla 3DGS, we are able to reduce it through compaction by 15.7x using a combination of state-ofthe-art pruning schemes. Then we encode it into an image-based format designed for simple, ubiquitous decoding. With the SOG-XT format, we propose a novel extension to Self-Organizing Gaussians with two main contributions: (i) Self-organizing 2D Codebooks and (ii) Parallel Representative Assignment Smoothing (PRAS), which leverages the symmetry of quaternion and scale parameterizations to produce 2D attribute grids more amenable to encoding. This encoding reduces scene size by 6.6x. We show that optional encoding-aware fine-tuning yields a further 2.2x. Across standard 3DGS benchmarks, our simple and modular approach thus achieves a total of 85× to 319× reductions in the size of the scene over uncompressed vanilla 3DGS, setting new benchmarks for real-world scenes and surpassing tightly integrated methods in rate-distortion. Decoding relies solely on web-native image formats, and the modular design makes each stage easy to combine with future advances in reconstruction and compaction. Code and project page: https://fraunhoferhhi.github.io/KISS-GS/

Customized Portrait Generation (CPG) technologies havebeen widely used to generate high-fidelity person images given an inputimage indicating the identity and a text prompt indicating the requirededits. Yet these methods pose significant privacy risks by spreading fakevisual information. Against such risks, each public generator should beable to suppress its generation ability for a particular person when re-quested. Therefore, in this work we investigate the identity unlearningproblem for CPG. Since there are no previous methods in this field, wepropose a simple baseline that updates the image encoder by minimizingidentity similarity between generated and input images for target iden-tities to be unlearned, while maximizing it for identities to be retained.However, we find such a global perturbation in the feature space harmsthe fidelity of generated images for other identities to be retained. Tosolve this problem, we propose a novel method IREU, which first locatesidentity-related features in an offline manner and then only performsfeature perturbations on them. The experimental results show that ourproposed method IREU achieves better identity unlearning performancefor target identities to be unlearned, and also keeps high fidelity for otheridentities to be retained. In addition, our unlearned image encoder isgeneralizable across different generators with the same encoder withoutfine-tuning, which is friendly for deployment in practice.

Continual vision–language learning is increasingly required in real deploymentswhere data, skills, and objectives drift, yet models still struggle to learn new tasks withouterasing old ones and to act reliably in task-agnostic settings with limited memory. We ad-dress this gap by turning cross-task conflict into an instance-level signal that drives how themodel adapts over time. We propose an interference-aware adapter routing framework thatestimates, for each sample, a metric-consistent projection energy to quantify potential inter-ference, routes to the least interfering experts with capacity balancing, enlarges inter-expertseparation via principal-angle packing, and grows LoRA rank only along the residual princi-pal direction when current experts are insufficient. Across classification, structured conceptmatching, generative VQA, and retrieval, our approach improves over strong baselines. Anal-yses show that the conflict signal reliably predicts downstream forgetting, packing widensprincipal angles over time, and rank growth is sparse yet beneficial—especially in mid–highlayers.

Monocular video depth estimation requires temporal con-sistency, geometric accuracy, and generalization across diverse scenar-ios—yet existing methods struggle to achieve all three simultaneously.Discriminative models excel at per-frame accuracy but suffer from tem-poral drift due to limited context windows, while generative methodsimprove consistency and generalization at the cost of extensive train-ing data (10M+ samples) and lack of geometric precision. In responseto these issues, we introduce ICDepth, a framework that adapts pre-trained text-to-video diffusion transformers for video depth estimationvia In-Context Conditioning (ICC), leveraging their rich spatial-temporalpriors. To address key challenges in transferring ICC from generationto dense prediction, we propose: (1) SAND-Attention, which ensuresprecise spatial-temporal alignment via shared RoPE and enforces uni-directional attention to prevent noise contamination; (2) SRFM, whichinjects DINOv2 semantic and resolution priors to enhance geometric pre-cision. ICDepth achieves state-of-the-art results on multiple benchmarkswith remarkable data efficiency, trained on only 0.8M frames (6–13×less than competing generative methods), while demonstrating strongzero-shot generalization to diverse domains.