ECCV 2026 论文列表

Latent world models roll observations forward accurately onfactual transitions, but their predictive latents are typically entangledand weakly aligned with the underlying causal factors; as a result, theydegrade under interventions and on counterfactual queries. We addressthis with CausalVAE, a plug-in structural module that attaches to di-verse encoder–transition backbones and organizes their latents througha learned directed acyclic graph (DAG) over causal factors, withoutaltering the base prediction architecture. Across four benchmarks andeight backbones, the plug-in largely preserves factual retrieval while im-proving intervention-aware counterfactual retrieval on most backbone–benchmark pairs; the gains are benchmark- and backbone-dependentrather than universal. Improvements are largest on the Physics bench-mark: for a graph-network backbone trained with a negative-log-likelihoodobjective, counterfactual H@1 (CF-H@1) rises from 11.0 to 41.0, withseveral other backbones gaining +10 to +30 CF-H@1 points. A causalanalysis further shows that the learned structure recovers meaningfulfirst-order physical interaction trends, supporting the interpretability ofthe latent causal structure.

Imitation learning enables robots to acquire complex manipulation skills from expert demonstrations, but offline-trained policies struggle to generalize beyond their training distribution, where small errors compound into severe distribution shift. Dataset Aggregation (DAgger) mitigates this by collecting human corrections during deployment, yet conventional DAgger operates entirely in the real world with a oneto-one ratio between expert effort and training trajectories makes comprehensive out-of-distribution (OOD) coverage impractical. Digital Twin (DT) techniques can scale data generation, but when applied to DAgger data without modification, they ignore the spatial and temporal failure patterns that DAgger rollouts reveal, leaving policies vulnerable precisely where and when they fail most. We propose Twin-DAgger, which treats each real-world intervention as simultaneously a spatial probe identifying failure-prone workspace regions and a temporal prior marking critical deviation moments. Twin-DAgger exploits these signals through two targeted augmentation mechanisms: difficulty-based stratified sampling concentrates synthetic data generation on high-failure-rate regions, and intervention-focused trajectory warping perturbs trajectories at each intervention onset to simulate a distribution of near-failure states while preserving the expert’s corrective recovery. On real-world manipulation tasks, Twin-DAgger matches conventional DAgger performance using only 15% human interventions and outperforms failure-agnostic DT baselines by 20% success rate under equal data budgets, demonstrating that generating data where and when the policy fails, rather than generating more data uniformly, is the critical driver of sample-efficient robustness.

Learning efficient and expressive visual representation haslong been the pursuit of computer vision research. While Vision Trans-formers (ViTs) gradually replace traditional Convolutional Neural Net-works (CNNs) as more scalable vision learners, their applications areplagued by the quadratic complexity of the self-attention mechanism. Toaddress the challenge, we introduce a new linear-time sequence modelingmethod Test-Time Training (TTT) into vision and propose Vision-TTT,which treats visual sequences as datasets and compresses the visual to-ken sequences in a novel self-supervised learning manner. By incorporat-ing the dual-dataset strategy and Conv2d-based dataset preprocessing,Vision-TTT effectively extends vanilla TTT to model 2D visual cor-relations with global receptive fields. Extensive experiments show thatVittt-T/S/B achieve 77.7%, 81.8%, 82.7% Top-1 accuracy on ImageNetclassification and also greatly outperform their counterparts on down-stream tasks. At 1280 × 1280 resolution, Vittt-T reduces FLOPs by79.4% and runs 4.72× faster with 88.9% less memory than DeiT-T.These results demonstrate the expressiveness and efficiency of Vision-TTT as an alternative for the next-generation visual backbone. Codesare available at https://github.com/imKQv/Vision-TTT.

Collaborative perception serves as a pivotal solution to en-hance the perception capability of individual agents in autonomous driv-ing, where a core challenge lies in seeking reliable evidence to quantifyand weight the contribution of each participating agent. Existing meth-ods typically rely on a confidence map (co-trained with the detectionhead), which is, however, inherently correlated with the detection re-sults and thus fails to provide unbiased physical evidence. Furthermore,how to deeply integrate evidence into the cooperative fusion process re-mains an open question. To address these issues, this paper first pro-poses uncertainty map, a physically grounded and unambiguous met-ric for evaluating perception quality. This map is directly supervisedby real-time sensor signals (i.e., LiDAR point density), ensuring de-coupling from detection noise and thereby providing physical scenario-aware evidence for weighting agent contribution. Based on this map,we develop the Uncertainty-Enhanced Collaborative Perception (UECP)framework, centered on the Uncertainty-Aware Pyramid Fusion (UAPF)module. UAPF uses a coarse-to-fine strategy, with two key components:Uncertainty-Weighted Downsampling (UWD) for high-fidelity featurepreservation, and Uncertainty-Guided Residual Fusion (UGRF) to rein-force ego features, suppressing noise and ensuring robust fusion. Exten-sive experiments on real-world datasets show UECP outperforms SOTAmethods in effectiveness and robustness by embedding the uncertaintymap into fusion. Code will be publicly available.

Despite remarkable advancements in Vision-and-LanguageNavigation (VLN), a significant sim-to-real gap remains. Most meth-ods are trained and evaluated in simplified or physics-limited simulators,which ignore the feedback loop between decision making and physical ex-ecution, resulting in unstable navigation performance. To bridge this gap,we propose Hi-Nav, a top-down hierarchical navigation framework thatdecomposes VLN into three controllable levels. The high-level moduleleverages a large language model to split long-horizon instructions intosub-tasks. At the middle level, an Interest Score Occupancy Map (ISOM)integrates instruction-relevant semantics with geometric traversability toperceive the surrounding environment, after which waypoint reasoning isperformed for route refinement. ISOM construction and waypoint reason-ing are jointly modeled within a unified multi-task learning framework,where a Q-Former fuses RGB-D observations and navigation instructionsinto shared query representations for efficient multi-task learning, aug-mented with learnable temporal embeddings to mitigate long-horizon for-getting. Finally, low-level behaviors are executed through ROS, enablingobstacle avoidance. We introduce a novel waypoint reasoning datasetfor comprehensive training and build a ROS/Gazebo VLN benchmarkto evaluate methods in a realistic simulation setting, which significantlyreduces the difficulty of sim-to-real transfer. Extensive experiments con-ducted in both Gazebo and the real world demonstrate strong perfor-mance, achieving up to 80% Success Rate (SR) with 0.65 m NavigationError (NE). Our code will be released at https://hiway-page.github.io/.

Monitoring canopy height change is essential for understand-ing carbon sinks and forest dynamics. Remote sensing enables consis-tent, large-scale observations of such changes, increasingly integratedwith deep learning architectures such as Geospatial Foundation Models(GFMs). However, existing methods and datasets frame the problem asbinary change detection, which overlooks both the continuous nature ofchange, especially for vegetation, and the inherent uncertainty in labels.We present the Canopy Height Change (CHC) dataset, providing 3 mresolution continuous canopy height differences and associated spatiallyresolved uncertainties across 10 598 km2 of northern and western Spain.The dataset is paired with a co-located time series of PlanetScope satel-lite imagery. Based on the dataset, we introduce the task of uncertainty-aware change regression, associated metrics and strategies for fine-tuningGFMs. Furthermore, we evaluate state-of-the-art GFMs and highlightpromising directions and remaining challenges for advancing continuouscanopy height change estimation.

We present a novel implementation of Visual SimultaneousLocalisation and Mapping (VSLAM), whose computation is performedcollaboratively between a sensor-based front-end utilizing efficient in-pixel compute, and a PC-based back-end that sends pose estimationback to the sensor to improve feature matching. The front-end performsthe tasks of point-feature detection and tracking at up to 2000 Hz, whilethe PC back-end conducts the less immediate, higher level tasks, of map-ping and localisation at ≈ 200 Hz. This division of tasks is somewhatreminiscent of biological vision, in which the retina performs low levelcomputations, while the brain handles slower integrative functions. Onlysparse data is exchanged between these two ends, such as feature loca-tions, descriptor data, and sensor pose estimations. Essentially, each sideonly sends data which the other side requires to perform its respectivetask. This is a radical departure from the traditional visual pipeline,where dense image frames are streamed from sensor to PC. By compar-ison, our collaborative compute pipeline is extremely efficient, demon-strating a 10× improvement to speed and latency, > 1000× bandwidthand ∼ 20× reduction in power consumption per processed frame.

In this work, we introduce FSU-QA, a VQA dataset for au-tonomous driving scenarios designed to advance research on ForesightIntelligence—the ability to anticipate and reason about complex, long-horizon futures. Unlike existing benchmarks that mainly focus on imme-diate perception or reactive planning, FSU-QA evaluates future-orienteddriving understanding through multi-agent-aware and rule-grounded coun-terfactual QA. Rather than holding surrounding agents fixed or target-ing geometric path generation, our benchmark requires models to infersemantic future outcomes from front-view historical observations andpast ego trajectories. A comprehensive evaluation on the accompanyingFSU-Bench reveals that state-of-the-art VLMs still face significant chal-lenges in anticipating future events. Furthermore, beyond model perfor-mance, we examine whether WM-generated predictions remain seman-tically consistent by using VLM-based proxy judges, and validate thisevaluation protocol through shuffled control experiments. Fine-tuningmodels on FSU-QA leads to substantial improvements in foresight under-standing, demonstrating the dataset’s effectiveness and offering a prin-cipled foundation for future research.

Humans paint images incrementally: they plan a global lay-out, sketch a coarse draft, inspect, and refine details, and most impor-tantly, each step is grounded in the evolving visual states. However, canunified multimodal models trained on text-image interleaved datasetsalso imagine the chain of intermediate states? In this paper, we introduceprocess-driven image generation, a multi-step paradigm that decom-poses synthesis into an interleaved reasoning trajectory of thoughtsand actions. Rather than generating images in a single step, our ap-proach unfolds across multiple iterations, each consisting of 4 stages: tex-tual planning, visual drafting, textual reflection, and visual refinement.The textual reasoning explicitly conditions how the visual state shouldevolve, while the generated visual intermediate in turn constrains andgrounds the next round of textual reasoning. A core challenge of process-driven generation stems from the ambiguity of intermediate states: howcan models evaluate each partially-complete image? We address thisthrough dense, step-wise supervision that maintains two complementaryconstraints: for the visual intermediate states, we enforce the spatial andsemantic consistency; for the textual intermediate states, we preserve theprior visual knowledge while enabling the model to identify and correctprompt-violating elements. This makes the generation process explicit,interpretable, and directly supervisable. To validate proposed method,we conduct experiments under various text-to-image benchmarks.

Real-world intelligent systems often require both distributedcollaboration across data-isolated clients and continual adaptation toevolving tasks. This setting naturally gives rise to Federated Class In-cremental Learning (FCIL), which combines Federated Learning (FL)and Continual Learning (CL). However, their combination introducestwo coupled sources of interference: spatial interference from heteroge-neous clients and temporal interference from sequential tasks, jointlyleading to Spatial_x0015_Temporal Catastrophic Forgetting (ST-CF). Exist-ing approaches typically address spatial and temporal interference withseparate mechanisms, often incurring additional client-side computationor communication, while leaving directional interactions among updatesduring aggregation unregulated. In this paper, we reinterpret FCIL as auni_x001C_ed multi-task learning problem, where both client and task updatesare represented as adaptation vectors in a shared parameter space. Basedon this view, we proposeSurgery & Merge Sum ( ), a purely server-sideframework that performs geometric surgery on adaptation vectors duringaggregation. Spatial Sum mitigates client-level interference within eachSumround, while causal online temporal removes cross-task interferenceover time without additional client-side computation, communication, ormemory beyond standard federated training. Empirically, Sum achievesup to 22% improvement over prior FCIL methods across diverse visionand language benchmarks while remaining robust to unreliable clientsand maintaining computational e_x001E_ciency.

Generative models have achieved remarkable progress, yetapplying them to satellite imagery remains challenging. Unlike natu-ral imagery, satellite scenes are structured by spatially complex andsemantically distinct geometries. Prior work addresses this complexityby adapting natural image frameworks using dense rasters or sparseprompts, trading off annotation cost and fidelity while breaking com-patibility with vector primitives commonly used to represent geographicinformation. We introduce TerraDiT-Ω, a unified spatial control frame-work that generates satellite imagery directly from any native geospatialprimitive. By jointly leveraging precise annotations (polygons, polylines)and coarser ones (bounding boxes, points), the model supports control-lable layouts across varying annotation budgets, broadening applicabilityto design tasks such as urban planning while remaining naturally com-patible with end-to-end GeoAI workflows. To effectively leverage theseprimitives during generation, we propose Geometry-Aware Local Atten-tion, a conditioning mechanism that injects explicit geometric cues intothe attention space. Across all conditioning formats, our approach con-sistently outperforms both dense-control and sparse-control baselines.Furthermore, this flexibility enables controllable synthetic data augmen-tation using a single generative model, improving downstream perfor-mance on land-cover segmentation, object detection, road graph extrac-tion, and scene classification. Code, data, and weights are available at

Autoregressive multimodal large language models (MLLMs)enable 3D generation but struggle to scale to high-resolution shapesdue to inadequate 3D tokenizations. Compact set-based representationsdiscard deterministic spatial ordering, leading to ambiguous sequenceprediction, while uniform or octree-based voxel grids preserve orderingat the cost of severe redundancy and excessively long sequences. Thisstructural trade-off limits stable and efficient autoregressive 3D gener-ation. We present SuperVoxelGPT, a representation-first frameworkthat resolves this tension through adaptive and deterministically orderedsupervoxel tokenization. Given a prompt, we first predict a coarse geo-metric saliency distribution and construct a shape-adaptive supervoxelpartition using saliency-guided centroidal Voronoi tessellation, allocatingfine-grained cells to complex regions and larger cells to smooth regions.Conditioned on this prompt and ordered supervoxel layout, we introducea SuperVoxelVAE and fine-tune a pretrained MLLM to autoregressivelygenerate supervoxel tokens. Experiments using Trellis-500K data showthat SuperVoxelGPT reduces token sequence length to 12.8% of uniformvoxel tokenization while achieving state-of-the-art generation quality andan average 10× speedup over prior methods.

While there are many techniques (e.g., QR codes) that con-vey information via visual patterns, some applications would benefit fromusing imperceptible patterns. We present a method for designing subtlecode-conveying patterns that can be printed on transparent sticker paperand applied to real-world surfaces. An image of a scene with an encodedsticker can be captured and the sticker localized and decoded. Thus,our stickers, while mostly unobtrusive to the human eye, are detectableand decodable to our model. We jointly optimize the encoder, localizer,and decoder end-to-end, balancing imperceptibility and accuracy. Ourmethod encodes 100-bit secrets and experimental results show that itis robust to human placement errors, imaging conditions, multiple stick-ers, occlusions, and wrinkles in stickers. Importantly, this method is easyto deploy and applicable in situations such as object tracking for aug-mented reality, conveyance of information about tourist sites, and subtleidentification of manufactured goods or their components.

The ultimate goal of spatial intelligence is to enable embod-ied agents to actively interact with the physical world. However, exist-ing benchmarks adopt a “Spectator View”—relying on single images orpassive video—neglecting the necessity of resolving geometric ambigu-ity through active viewpoint transformation. To address this, we presentWalkerBench, a global-scale interactive benchmark built on real-worldstreet-view and map data spanning 161 cities, with two categories ofhierarchically structured spatial tasks. Our empirical evaluation revealsa critical “representation misalignment”: VLMs’ one-dimensional linearcontext structures are fundamentally incompatible with the topologicalnature of 3D environments, causing inevitable spatial forgetting duringnavigation. To overcome this, we propose Spatial-IDE, which breaks fromlinear dialogue history via two mechanisms: (1) State Externalization,transforming implicit observations into an Explicit Topological Memory(ETM); and (2) Cognitive Decoupling, disentangling visual understand-ing into Goal-Directed Perception and Spatial Reasoning, letting theVLM focus on pure high-level decision-making. Spatial-IDE achieves sub-stantial improvements on WalkerBench and successfully generalizes zero-shot to real-world urban navigation on the Unitree humanoid robot. Theproject is available on https://github.com/lalayang123456- ctrl/WalkerBench.

An embodied kitchen assistant must do more than recog-nize food in isolated frames. It must track ingredient states over timeand integrate visual observations with recipe and nutritional knowledgeto support constraint-aware decision-making. We formalize this capa-bility as Embodied Nutrition Management: perceiving nutrition-relevantevents, maintaining a persistent food state, and using it for knowledge-grounded planning. Existing benchmarks evaluate static food under-standing or embodied cooking actions, but do not measure whether anagent can continuously update and use nutrition-relevant states in dy-namic kitchens. To fill this gap, we introduce NutriBench-Kitchen,a benchmark containing 1,500 manually verified question–answer pairsfrom 160 cooking videos. It covers five task families: Ingredient Entry,Memory Management, Recipe Query, Long-Term Planning, and Short-Term Planning, spanning food-state construction, maintenance, knowl-edge retrieval, and decision-making across different planning horizons.Evaluations of proprietary and open-source large vision-language modelsreveal a substantial gap from human performance, particularly in quan-titative ingredient estimation, long-term state tracking, and reasoningunder interacting constraints. We further introduce Nutri-Vgent, a di-agnostic long-video agent with separate episodic, food-state, and recipememories. Its consistent improvements demonstrate the value of explicitstate representations and structured memory for nutrition management.Together, NutriBench-Kitchen and Nutri-Vgent provide a testbed forstudying persistent state tracking and knowledge-grounded reasoning indynamic kitchens. Code is available at https://github.com/V1ol1n/NutriBench-Kitchen.

The emergence of Multimodal Large Language Models hasenabled a new generative paradigm for multimodal emotion recognition(MER), where emotional interpretations are produced through multi-modal token-level reasoning. In view of multimodal inputs, existing stud-ies commonly observe that reallocating attention from generated tokensto modality tokens during inference consistently improves performance.However, in this work, we reveal a counterintuitive phenomenon: such astrategy only benefits two-modality settings, but degrades performancein three- or more-modality scenarios, where the opposite reallocation di-rection instead leads to improvements. To understand this phenomenon,through comprehensive analysis, we show that with more modalities, thecomplex interactions between modalities make the model hard to paymore attention to tokens with more information, especially in deeperlayers, leading to improper attention allocation and the observed contra-dictory behavior. Based on these insights, we propose a training-freeattention rectification method that leverages structured shallow-layerattention as a prior to regularize entangled final-layer attention dur-ing inference, without introducing additional parameters or modifyingthe backbone model. Extensive experiments on nine datasets in MER-UniBench demonstrate that our method achieves state-of-the-art per-formance, consistently outperforming both reallocation directions acrossdiverse multimodal scenarios. Our code is available at https://github.com/yzl77/ReAR.

Human trajectory forecasting is crucial for safe navigationin crowded environments, requiring models that balance accuracy withcomputational efficiency. Efficiently modeling social interactions is key toperformance in dense crowds. Yet, most recent methods rely on attentionmechanisms, which are effective at capturing complex dependencies, butincur quadratic computational costs that scale poorly with the grow-ing number of neighbors. Recently, Selective State-Space Models haveprovided a linear-time alternative; however, their inherently sequentialdesign is misaligned with the unstructured and dynamic nature of socialinteractions. To address this challenge, we propose Social-Mamba, a fore-casting architecture that reformulates social interactions as structuredsequential processes. At its core is the Cycle Mamba block, a novel mod-ule that enables continuous bidirectional information flow. Social-Mambaorganizes agents on an egocentric grid and introduces social triplet fac-torization, which decomposes interactions into temporal, egocentric, andgoal-centric scans. These are dynamically integrated through a learnablesocial gate and global scan to generate accurate and efficient trajectorypredictions. Extensive experiments on five trajectory forecasting bench-marks show that Social-Mamba achieves state-of-the-art accuracy whileoffering superior parameter efficiency and computational scalability. Fur-thermore, embedding Social-Mamba into a flow-matching framework fur-ther enhances both accuracy and efficiency, establishing it as a flexibleand robust foundation for future trajectory forecasting research. Thecode is publicly available: https://github.com/vita-epfl/Social-Mamba.

Recent co-speech gesture generation methods often overlookcultural differences, limiting their effectiveness in human–agent inter-action. Moreover, culture-conditioned models are rarely evaluated un-der speaker-disjoint splits, so apparent “cultural” behavior may be con-founded with speaker-specific gesturing style. We introduce SICAGE, amodular framework for culture-aware co-speech gesture generation thatconditions motion synthesis models on speaker-independent cultural rep-resentations. SICAGE learns these representations from audio and textby treating each speaker as a separate domain while imposing invari-ance across speakers. This encourages representations to remain culture-discriminative while reducing dependence on speaker identity. The re-sulting cultural embeddings condition a multimodal generator to pro-duce culturally appropriate gestures. We instantiate this idea with twodomain generalization approaches: adversarial learning and Fishr reg-ularization. We further introduce ALaDiT, a real-time diffusion-basedgesture generator designed to efficiently incorporate the learned culturalembeddings. To validate our method, we built TED4C-L, a 106-hourmultimodal dataset of 764 TED speakers from four cultural groups. Ex-periments show that SICAGE improves motion realism, diversity, beatsynchronization, semantic relevance, and cultural consistency.

Modern Visual Place Recognition (VPR) methods excel onstandard benchmarks yet remain brittle in feature-poor environments.By treating each query image in isolation, they discard the sequentialcontext in any real trajectory. We formalize a task that exploits thiscontext: given a query sequence, localize the final image against an un-ordered reference database—which, unlike sequence-to-sequence meth-ods, requires no sequential structure in the database. We propose TRAIL(TRajectory-Aware Image Localization), a principled framework basedon Conditional Random Fields (CRF) that combines learned functionsfor visual similarity and for camera-motion consistency, refining a dis-tribution over candidate references as each query arrives. A lightweightpost-processing layer atop any pre-trained VPR backbone, TRAIL im-proves a state-of-the-art baseline by up to 8.3 percentage points on ourprimary benchmark, transfers to unseen datasets without retraining, anddelivers its largest gains where visual cues are scarce.