ECCV 2026 论文列表

Vision-Language Navigation (VLN) for Unmanned AerialVehicles (UAVs) demands complex visual interpretation and continuouscontrol in dynamic 3D environments. Existing hierarchical approachesrely on dense oracle guidance or auxiliary object detectors, creating se-mantic gaps and limiting genuine autonomy. We propose AeroVLA, aminimalist end-to-end Vision-Language-Action framework mapping rawvisual observations and fuzzy linguistic instructions directly to contin-uous physical control signals. First, we introduce a streamlined dual-view perception strategy that reduces visual redundancy while preserv-ing essential cues for forward navigation and precise grounding, whichadditionally facilitates future simulation-to-reality transfer. To reclaimgenuine autonomy, we deploy a fuzzy directional prompting mechanismderived solely from onboard sensors, completely eliminating the depen-dency on dense oracle guidance. Ultimately, we formulate a unified con-trol space that integrates continuous 3-Degree-of-Freedom (3-DoF) kine-matic commands with an intrinsic landing signal, freeing the agent fromexternal object detectors for precision landing. Extensive experiments onthe TravelUAV benchmark demonstrate that AeroVLA achieves state-of-the-art performance in seen environments. Furthermore, it exhibitssuperior generalization in unseen scenarios by achieving nearly threetimes the success rate of leading baselines, validating that a minimal-ist, autonomy-centric paradigm captures more robust visual-motor rep-resentations than complex modular systems. Code is available at: https://github.com/XuPeng23/AeroVLA

Modern autonomous driving systems increasingly rely ontightly coupled Camera–LiDAR fusion pipelines to achieve robust sceneunderstanding. While this multi-modal redundancy is designed to counterindependent single-sensor failures, it implicitly assumes that physicalobjects cannot simultaneously deceive both modalities. In this paper, weexpose the vulnerability of this assumption by proposing Agent-OBJ,a generative framework that synthesizes physically plausible 3D adver-saries via a prompt-driven agent. Unlike prior works that mainly focus onsensor-specific perturbations, our pipeline instantiates a physically plau-sible 3D adversary by generating a canonical pedestrian geometry from abase prompt and then modulating its shape via an additional semanticprompt, while controlling appearance with a pretrained multi-view per-sonalization model conditioned on multi-view images and a style promptto produce view-consistent appearance. To simultaneously compromiseboth 2D monocular and 3D fusion detectors, the agent dynamically op-timizes the generated adversary using confidence-score feedback fromboth detectors. Meanwhile, to ensure stealthiness, we regularize the gen-erated 3D point cloud by enforcing similarity to a pedestrian featurebank, aligning it with the distribution of natural objects and makingit visually and geometrically indistinguishable from benign instances.Furthermore, we incorporate a multi-view consistency constraint duringoptimization to promote cross-view robustness, ensuring the adversaryremains effective under diverse viewpoints in real-world driving scenes.Extensive experiments on nuScenes demonstrate high attack success ratesagainst both monocular and fusion detectors while preserving strongperceptual plausibility.

Video-based embodied world models provide an appealingsubstrate for robotic manipulation by predicting future states, yet cur-rent approaches remain limited by a fundamental entanglement: accu-rately modeling dynamics typically requires low-level temporal reason-ing, while producing high-resolution frames demands expansive visualsynthesis according to high-level semantics. This entanglement results inslow inference speed for iterative planning or too coarse predictions toretain contact-rich details. To solve this dilemma, we present Disentan-gled Video Generation World Model (DVG-WM), an efficient frameworkthat explicitly decomposes world modeling into dynamics learn-ing and visual synthesis. Conditioned on an initial observation anda language instruction, our model first generates a plausible sequence ofintermediate visual states to preview the physical interaction and refinesthem to obtain high-fidelity videos. Furthermore, an efficient cascadingmechanism is proposed, where DVG-WM leverages flow matching to di-rectly map the dynamics to video latents, and introduces a latent degra-dation mechanism to enable the capability of regenerating contact-richdetails. Experiments on LIBERO and real-world platforms demonstrateimproved video quality with up to 3.97 × acceleration, validating thatdisentangled video generation can be an efficient embodied world modelfor robotic manipulation.

Detecting and localizing objects in space is a fundamentalcomputer vision problem. While much progress has been made to solve2D object detection, 3D object localization is much less explored and farfrom solved, especially for open-world categories. To address this researchchallenge, we propose Boxer, an algorithm to estimate static 3D boundingboxes (3DBBs) from 2D open-vocabulary object detections, posed images,and optional depth represented either as a sparse point cloud or densedepth. At its core is BoxerNet, a transformer-based network that lifts2D bounding box (2DBB) proposals into 3D, followed by multi-viewfusion and geometric filtering to produce globally consistent de-duplicated3DBBs in metric world space. Boxer leverages the power of existing 2DBBdetection algorithms (e.g., DETIC [52], OWLv2 [29], SAM3 [8]) to localizeobjects in 2D. This allows the main BoxerNet model to focus on liftingto 3D rather than detecting, ultimately reducing the demand for costlyannotated 3DBB training data. Extending the CuTR [20] formulation, weincorporate an aleatoric uncertainty for robust regression, a mean depthpatch encoding to support sparse depth inputs, and large-scale trainingwith over 1.2 million unique 3DBBs. BoxerNet outperforms state-of-the-art baselines in open-world 3DBB lifting, including CuTR in egocentricsettings without dense depth (0.532 vs. 0.010 mAP) and on CA-1M withdense depth available (0.412 vs. 0.250 mAP). Project page and code areavailable at https://facebookresearch.github.io/boxer.

Vision-Language Navigation (VLN) requires an agent to navigate through 3D environments by following natural language instructions. While recent Video Large Language Models (Video-LLMs) have largely advanced VLN, they remain highly susceptible to State Drift in long scenarios. In these cases, the agent’s internal state drifts away from the true task execution state, leading to aimless wandering and failure to execute essential maneuvers in the instruction. We attribute this failure to two distinct cognitive deficits: Progress Drift, where the agent fails to distinguish completed sub-goals from remaining ones, and Memory Drift, where the agent’s history representations degrade, making it lose track of visited landmarks. In this paper, we propose a Dual-Anchoring Framework that explicitly anchors the instruction progress and history representations. First, to address progress drift, we introduce Instruction Progress Anchoring, which supervises the agent to generate structured text tokens that delineate completed versus remaining sub-goals. Second, to mitigate memory drift, we propose Memory Landmark Anchoring, which utilizes a Landmark-Centric World Model to retrospectively predict object-centric embeddings extracted by the Segment Anything Model, compelling the agent to explicitly verify past observations and preserve distinct representations of visited landmarks. Facilitating this framework, we curate two extensive datasets: 3.6 million samples with explicit progress descriptions, and 937k grounded landmark data for retrospective verification. Extensive experiments in both simulation and real-world environments demonstrate the superiority of our method, achieving a 15.2% improvement in Success Rate and a remarkable 24.7% gain on long-horizon trajectories. To facilitate further research, we will release our code, data generation pipelines, and the collected datasets.

Traffic elements such as traffic lights and road signs play a fundamental role in human driving decisions and should naturally influence end-to-end driving performance. However, existing end-to-end driving research predominantly focuses on dynamic road participants (e.g., vehicles and pedestrians), while the role of traffic elements remains largely unexplored. To date, the community lacks a systematic study quantifying how traffic elements affect end-to-end driving models. This gap stems from two main challenges: first, existing public datasets rarely provide structured annotations for traffic elements; second, modern end-to-end driving systems vary widely in architectures and training paradigms, making conclusions drawn from a single method difficult to generalize. In this work, we present the first systematic investigation of traffic element awareness for end-to-end autonomous driving. We begin by constructing a unified research infrastructure by augmenting multiple public driving datasets with comprehensive traffic element annotations. To ensure broad applicability across diverse model families, we intentionally adopt a minimal and universal integration design, allowing traffic element signals to be incorporated into existing pipelines in a plug-and-play manner with negligible architectural modification. We evaluate this design across a wide spectrum of modern paradigms, including perception–prediction-planning pipelines, vision-language-action models (VLA), regression-based planners, diffusion-based policies, and trajectory scoring frameworks. Experiments are conducted on multiple widely used benchmarks, including nuScenes, NAVSIM-v1, NAVSIM-v2, and Bench2Drive. Across all paradigms and datasets, our simple integration consistently improves driving performance, demonstrating that traffic element awareness provides a robust and generalizable signal for end-to-end driving systems. Notably, on the challenging NAVSIM-v2 benchmark, our approach significantly boosts the performance of stateof-the-art architectures and data pipelines, establishing a new state of the art.

Humans seamlessly integrate egocentric perceptions with exocentric perspectives to comprehend dynamic spatial positioning. For Embodied AI, mastering this ego-exo spatial reasoning is a fundamental prerequisite for real-world interaction. Although Multimodal Large Language Models (MLLMs) have demonstrated significant progress in general visual reasoning, their capacity for ego-exo spatial reasoning in real-world environments remains largely unexplored. To investigate whether current MLLMs exhibit such human-like spatial cognition, we introduce SEERBench, a rigorous benchmark comprising 1,198 highquality, human-annotated question-answer pairs across 35 diverse realworld Ego-Exo scenarios. SEERBench evaluates models through 10 distinct tasks structured into three progressive cognitive levels: Spatial Perception, Spatial Imagination, and Ego-Exo Collaboration. Extensive evaluations of 19 representative MLLMs reveal a substantial performance gap: even the best-performing frontier models trail human experts by a margin of 46.8%, highlighting severe limitations in current multimodal foundations. To address this, we propose SEER-Map, a training-free, tool-augmented baseline. By explicitly constructing top-down Bird’s-Eye View (BEV) spatial topologies to assist in ego-exo geometric alignment, SEER-Map functions as an Oracle-informed probing baseline lower the threshold of spatial understanding. Codes and data are available here.

Learned Image Signal Processing (ISP) pipelines offer powerful end-to-end performance but are critically dependent on large-scale paired raw-to-sRGB datasets. This reliance on costly-to-acquire paired data remains a significant bottleneck. To address this challenge, we introduce a novel, unsupervised training framework based on Optimal Transport capable of training arbitrary ISP architectures in both unpaired and paired modes. We are the first to successfully apply Unbalanced Optimal Transport (UOT) for this complex, cross-domain translation task. Our UOT-based framework provides robustness to outliers in the target sRGB data, allowing it to discount atypical samples that would be prohibitively costly to map. A key component of our framework is a novel “committee of expert discriminators,” a hybrid adversarial regularizer. This committee guides the optimal transport mapping by providing specialized, targeted gradients to correct specific ISP failure modes, including color fidelity, structural artifacts, and frequency-domain realism. To demonstrate the effectiveness of our approach, we retrained existing state-of-the-art ISP architectures using our paired and unpaired setups. Our experiments show that while our framework, when trained in paired mode, improves upon the original paired methods across all metrics, our unpaired mode concurrently achieves quantitative and qualitative performance that provides a competitive alternative to the original pairedtrained counterparts. The code and pre-trained models are available at: https://github.com/gosha20777/EGUOT-ISP.git.

Despite remarkable progress in multimodal understanding,current MLLMs still exhibit limitations in video text understanding, par-ticularly when semantics emerge through the integration of temporallydistributed textual cues across multiple frames. This perception challengefundamentally differs from static image text understanding, yet existingdatasets fail to capture: the vast majority of questions remain answerablefrom single frames, inadequately reflecting real-world video text compre-hension demands. To address this, we present ViTexQA, a large-scalevideo-text QA dataset, and FrameThinker for robust multi-frame tem-poral reasoning. We build ViTexQA via a quality-controlled Chain-of-Thought (CoT) annotation pipeline boosted with temporal constraints;all its QA pairs demand cross-frame text fusion to solve, enforcing truetemporal reliance. FrameThinker adopts two-stage training for explicittemporal modeling: CoT-Guided Supervised Fine-Tuning (SFT) gen-erates frame-aware reasoning chains, followed by Temporally-groundedReinforcement Learning (RL) optimized with multi-frame coherence re-wards. Evaluations show our method outperforms SOTA baselines onViTexQA, lifting ROUGE-L by 6.3%.

Current Video Large Language Models (Video LLMs) excel in question answering (QA) but largely operate as black boxes, providing textual answers without verifiable visual grounding. Existing explainability efforts rely on textual rationales or sparse bounding boxes, which struggle to capture complex video dynamics such as occlusions and nonrigid deformations. We propose Evidence-Backed Video Question Answering (E-VQA), a novel task requiring models to jointly output a semantic answer and precise spatio-temporal evidence: temporal segments and dense, tracked object segmentation masklets. To support this, we introduce ST-Evidence, the first human-verified benchmark for both discriminative and generative pixel-level grounding. Evaluations of state-of-the-art models reveal a critical decoupling between QA accuracy and true visual perception that scaling alone fails to bridge. To address this, we develop scalable, automated generation pipelines to create ST-Evidence-Instruct, a 160k-scale dataset bridging highlevel reasoning with fine-grained grounding. Fine-tuning grounded Video LLMs on this data yields substantial gains over the corresponding sizematched UniPixel baselines (e.g., +27.2 t-mean and +13.8 J&F on a 7B model), establishing a robust baseline for explainable, evidencebacked video understanding. Code and data are available at https: //github.com/SalesforceAIResearch/EVQA.

Point cloud registration critically depends on local featuresthat are both distinctive and robust to arbitrary 3D rotations. Existinglearning-based methods typically approximate rotation invariance viafragile local reference frames or extensive data augmentation, providingonly empirical invariance and often degrading under unseen rotationaltransformations. In this paper, we propose SHReg, a strictly rotation-equivariant point cloud registration framework grounded in the repre-sentation theory of SO(3). By representing local geometric features asirreducible representations of SO(3), SHReg guarantees exact equivari-ance under arbitrary rotations without relying on local reference frames.Built upon a spherical-harmonics-based equivariant backbone, SHRegjointly learns rotation-invariant descriptors for robust correspondencematching and rotation-equivariant features that preserve fine-grained ori-entation information. The preserved equivariant structure enables eachcorrespondence to directly hypothesize a rigid transformation, reducingreliance on large-scale hypothesis sampling in conventional RANSAC-based pipelines and leading to improved robustness under challengingrotational variations. Extensive experiments on 3DMatch, 3DLoMatch,and KITTI demonstrate that SHReg consistently outperforms state-of-the-art methods in registration accuracy, particularly under large rota-tional perturbations.

Continual learning methods for vision-language models are developed on benchmarks where each new task introduces entirely new domain knowledge. Real-world task sequences are more natural: they routinely share visual concepts, language patterns, and even training samples across stages. However, existing mixture-of-expert methods that assign one expert per task with fixed routing can split similar inputs across different experts and degrade performance. We introduce Semantic Overlap-aware Continual Learning (SOCiaL), a simple framework designed to identify and leverage shared structure across tasks. SOCiaL employs a Gaussian mixture model to estimate contextual similarity, generate synthetic replay samples, and guide expert routing. When task contexts are highly similar, their adapters are consolidated and the router is updated accordingly. To study this more realistic continual learning scenario, we also present UCIT-O, a new benchmark with three protocols that progressively increase semantic similarity across tasks. Across both disjoint and overlapping benchmarks, SOCiaL consistently outperforms existing methods, achieving 7.35, 8.04, and 3.07–9.77 points above the strongest baseline on CoIN, UCIT, and UCIT-O while reducing deployed adapters by up to threefold.

Machine unlearning in Vertical Federated Learning (VFL)has attracted growing interest, yet existing methods certify forgettingsolely using output-level metrics. We challenge these works by intro-ducing Mirage, a representation-level auditing framework built fromfour complementary diagnostics. Mirage combines linear probe recovery(LPR), centered kernel alignment (CKA), feature separability scoring,and layer-wise recovery analysis to assess what a representation actuallyretains. Extensive experiments across seven datasets and seven baselinemethods following recent VFL unlearning protocols reveal three key find-ings. (1) Forgetting gap: methods that pass output-level certification stillretain substantial class structure in their representations, with LPR ex-ceeding the retrained baseline by up to 15.4 points. CKA shows thatthese models remain structurally closer to the original than to the re-trained reference, and separability scores indicate persistent geometricdiscrimination. (2) Unlearning trilemma: no existing method simultane-ously achieves high utility, output-level forgetting, and representation-level forgetting. (3) Class-sample asymmetry: class-level forgetting leavesstrong representational traces (LPR exceeding 96% on several datasets),whereas sample-level forgetting is indistinguishable from chance (LPR≈ 50%). Layer-wise analysis further shows that residual class informationpersists across network depths. These findings call for representation-aware evaluation standards in federated unlearning research. Code ispublicly available at https://github.com/YuZhenyuLindy/Mirage.

Referring expression comprehension (REC) aims to localize the object in an image described by natural language. In Weakly supervised REC (WREC), existing approaches primarily operate on anchorlevel visual representations. Even when enriched with auxiliary cues, relational interactions remain implicitly encoded within individual anchor features. The resulting visual representation remains flat and unary-only, limiting its ability to align with the structured nature of language. In this work, we propose a Structured Visual Compositional Representation (SVCR) learning framework for WREC. Rather than implicitly encoding relations within unary anchors, the proposed SVCR explicitly models both unary object embeddings and pairwise relational embeddings, forming a structured visual representation space. We further introduce a compositional alignment mechanism that matches unary and pairwise visual representations with their corresponding textual embeddings in a unified manner, enabling compositional visual–textual matching under weak supervision. Extensive experiments on RefCOCO, RefCOCO+, and RefCOCOg show that the proposed SVCR achieves state-of-the-art performance. These results demonstrate the effectiveness of explicit structured visual representations and visual-textual alignment for WREC.1

Assessing AI-generated images remains a signi_x001C_cant chal-lenge because both visual quality and prompt-image alignment are vital.Existing metrics prioritize visual quality, but the exploration of semanticalignment with user prompts remains limited. To address this issue, wepropose a novel cyclic mutual information maximization framework forAI-generated image quality assessment (CMIM-AIGIQA). Unlike exist-ing methods, our framework focuses on promoting e_x001B_ective multimodalinformation fusion in the deep feature domain, which has cyclic mutualinformation maximization phases. In the forward phase, we maximize themutual information between text and image features to generate re_x001C_ned,cross-aware representations. The re_x001C_ned features are then integratedthrough a fusion network. In the backward phase, the mutual informationbetween the joint embedding and every single modality is maximized toensure that the joint embedding retains critical semantic and visual cues.Experiments on three datasets demonstrate that our method e_x001B_ectivelybridges the gap between visual quality and prompt-image alignment. Thecode is available at https://github.com/Buka-Xing/CMIM-AIGIQA.

All-in-one image restoration aims to handle diverse degra-dations within a unified framework. While Mixture-of-Experts (MoE)architectures offer a scalable path, existing designs predominantly relyon vanilla end-to-end fidelity-oriented experts that lack task-adaptivegenerative priors, limiting their ability to recover realistic structuresand textures in complex scenarios. In this paper, we first investigatecomplementary task-adaptive generative priors: Masked Auto-Regressive(MAR) experts are tailored for local degradations (e.g., shadow) by re-constructing semantic structures via clean local backgrounds; conversely,Diffusion experts excel at global degradations (e.g., noise) by recoveringtextural details through iterative denoising. Motivated by these insights,we propose Mixture-of-Synergy-Experts (MoSE), a novel framework thatdeeply integrates these complementary generative priors into the MoEarchitecture and synergizes MAR and Diffusion experts with vanillabackbones at both image and feature levels. MoSE is orchestrated by aDINOv3-guided semantic-aware router that adaptively regulates not onlyexpert coordination but also generation control: mask sizes are adjustedto handle varying scales of local semantic corruption, while iterationsteps are modulated according to degradation intensity. By synergistically"weaving" complementary experts, our approach achieves a controllablefidelity-perception trade-off. Comprehensive experiments demonstrate theeffectiveness of MoSE, achieving competitive performance across variousall-in-one image restoration benchmarks.

Real-time autoregressive (AR) video diffusion has progressedrapidly. Existing approaches typically distill pretrained bidirectional videofoundation models into few-step causal students; however, naive distilla-tion often collapses due to architectural mismatch. To obtain stable ini-tialization, prior methods rely on ordinary differential equation (ODE)trajectory matching, which incurs substantial computation and can in-troduce errors by forcing the student to imitate global trajectories. Inthis paper, we bypass trajectory matching stage by pretraining a robustcausal AR model that equips the student for direct few-step distillation.Moreover, to bridge the gap between pretraining and distillation, we pro-pose Error Forcing, an error-aware and parallelizable AR video diffu-sion framework. During training, Error Forcing injects controlled residualerrors into the conditioning context, approximating the inference-timehistory distribution. This enables the causal model to learn from de-graded yet clean contexts, improving robustness to accumulated errorswithout sacrificing parallel training throughput. With this initialization,we can perform direct distillation, and the student and teacher/criticnaturally operate under consistent conditioning distributions. Extensiveexperiments show that our method outperforms existing baselines in gen-eration quality during both the pre-training and distillation stages.

Eye tracking (ET) plays a critical role in augmented andvirtual reality applications. However, rapidly deploying high-accuracy,on-device gaze estimation for new products remains challenging becausehardware configurations (e.g., camera placement, camera pose, and il-lumination) often change across device generations. Visual foundationmodels (VFMs) excel on natural-image benchmarks and offer a promis-ing path to rapid training and deployment; yet, we find that off-the-shelfVFMs still struggle to reach high accuracy on specialized near-eye in-frared images. To close this gap, we introduce DistillGaze, a frameworkthat distills a VFM using labeled synthetic data and unlabeled real datafor rapid, high-accuracy on-device gaze estimation. DistillGaze proceedsin two stages. First, we adapt a VFM into a domain-specialized teacherusing synthetic gaze labels and unlabeled real images. Synthetic dataprovide scalable, high-quality gaze supervision, while unlabeled real databridges the synthetic-to-real domain gap. Second, we train an on-devicestudent from both teacher guidance and self-training. Evaluated on alarge-scale crowd-sourced dataset spanning more than 2,000 participants,DistillGaze reduces median gaze error by 58.6% relative to synthetic-only baselines while maintaining a lightweight 256K-parameter modelsuitable for real-time on-device deployment. More broadly, DistillGazeoffers an efficient path to training and deploying ET models that adaptto hardware changes, and a recipe for combining synthetic supervisionwith unlabeled real data in on-device regression tasks.

Partial-label learning (PLL), which refers to the classifica-tion task where each training instance is associated with a set of candidate-labels, among which only one is correct. While recent advances in deepPLL primarily focus on designing sophisticated disambiguation strate-gies, the role of PLL-specific data augmentation has been less explored.Most methods often treat data augmentation as a generic plug-in, over-looking its potential interaction with the label disambiguation processunique to PLL. To address this gap, we propose a local-ambiguity-awareaugmentation framework that consists of label identification and modelregularization modules and accommodates a wide range of PLL algo-rithms. Within this framework, a PLL-specific augmentation strategybased on a mixing-renormalization process is applied to partial-labeldata. Theoretically, we show that our augmentation can make the PLLdataset easier by reducing the local ambiguity degree, and it induces mul-tiple implicit regularizations during training thereby facilitating highlyeffective disambiguation. Extensive experiments on both benchmark andreal-world datasets demonstrate that our approach consistently improvesthe generalization performance of state-of-the-art PLL algorithms.