ECCV 2026 Papers

Vision-Language Models (VLMs) improve generalization andinterpretability in autonomous driving but suffer from efficiency issuesdue to long visual token sequences, particularly in standard multi-viewsettings. Existing token pruning methods employ fixed pruning rate al-location and static importance metrics, ignoring dynamic inter-view im-portance differences and the evolving information importance during in-ference. Our analysis reveals that multi-view VLMs inherently encodetask-related view priors in deeper layers and exhibit dynamic informa-tion requirements. Motivated by these findings, we propose MVPruner,a two-stage adaptive token pruning method that aligns pruning behaviorwith the model’s dynamic information requirements. The first stage al-locates pruning budgets based on the information diversity of each view,and retains tokens with consistent contribution across stages, ensuringsemantic representational capacity. The second stage allocates budgetsand selects tokens guided by instruction text to guarantee task align-ment. Experimental results on four benchmarks demonstrate the supe-rior performance of our method. For example, DriveMM equipped withMVPruner achieves 87.3% reduction in FLOPs, 4.97× speedup in pre-filling phase while retaining 98.5% accuracy on DriveLM benchmark.

High-fidelity 3D generative modeling increasingly relies onthe latent diffusion paradigm, where the reconstruction quality of theunderlying 3D VAE becomes a primary bottleneck. Existing approacheslargely follow two paradigms: sparse voxel-based representations achievestrong reconstruction quality but incur significant memory and compu-tational overhead, while set-based representations are compact and con-tinuous yet typically lag in fidelity due to latent sparsity and excessiveglobal smoothness. We propose MSVS-VAE, a hierarchical set-based VAEthat closes this fidelity gap without sacrificing compactness. Our keyidea is to progressively densify anchored VecSet latents via hierarchicalpoint-shuffle upsampling, increasing spatial capacity for fine-grained ge-ometry modeling. To efficiently decode from the densified hierarchy, wereplace global cross-attention with AVS-Conv, a geometry-aware localaggregation operator operating within local neighborhoods rather thanthe exhaustive latent set. We further introduce multi-scale query decod-ing to fuse coarse-to-fine latent features, where coarse scales providestable global context, and fine scales refine localized geometry, reducingartifacts from overly local receptive fields. Extensive experiments on Ob-javerse, ABO, and in-the-wild benchmarks demonstrate that MSVS-VAEconsistently outperforms prior set-based and voxel-based VAEs, deliver-ing ∼10× faster decoding than prior set-based methods and ∼10× highercompactness than voxel-based baselines.

Converting static 3D meshes into interactable articulated as-sets is crucial for embodied AI and robotic simulation. However, exist-ing zero-shot pipelines struggle with complex assets due to a criticallack of physical grounding. Specifically, ungrounded Vision-LanguageModels (VLMs) frequently suffer from kinematic hallucinations, whileunconstrained joint estimation inevitably leads to catastrophic meshinter-penetration during physical simulation. To bridge this gap, we pro-pose MotionAnymesh, an automated zero-shot framework that seam-lessly transforms unstructured static meshes into simulation-ready dig-ital twins. Our method features a kinematic-aware part segmentationmodule that grounds VLM reasoning with explicit SP4D physical priors,effectively eradicating kinematic hallucinations. Furthermore, we intro-duce a geometry-physics joint estimation pipeline that combines robusttype-aware initialization with physics-constrained trajectory optimiza-tion to rigorously guarantee collision-free articulation. Extensive experi-ments demonstrate that MotionAnymesh significantly outperforms state-of-the-art baselines in both geometric precision and dynamic physicalexecutability, providing highly reliable assets for downstream applica-tions. Our project page is available at https://xwb0117.github.io/Motionanymesh/

Personalized object detection aims to adapt a general-purposedetector to recognize user-specific instances from only a few examples.Lightweight models often struggle in this setting due to their weak se-mantic priors, while large vision-language models (VLMs) offer strongobject-level understanding but are too computationally demanding forreal-time or on-device applications. We introduce MOCHA (Multi-modalObjects-aware Cross-arcHitecture Alignment), a distillation frameworkthat transfers multimodal region-level knowledge from a frozen VLMteacher into a lightweight vision-only detector. MOCHA extracts fusedvisual and textual teacher’s embeddings and uses them to guide stu-dent training through a dual-objective loss that enforces accurate localalignment and global relational consistency across regions. This processenables efficient transfer of semantics without the need for teacher modi-fications or textual input at inference. MOCHA consistently outperformsprior baselines across four personalized detection benchmarks under strictfew-shot regimes, yielding a +10.1 average improvement, with minimalinference cost.

Generative novel view synthesis faces a fundamental dilemma:geometric priors provide spatial alignment but become sparse and inaccu-rate under view changes, while appearance priors offer visual fidelity butlack geometric correspondence. Existing methods either propagate geo-metric errors throughout generation or suffer from signal conflicts whenfusing both statically. We introduce MoCam, which employs structureddenoising dynamics to orchestrate a coordinated progression from geom-etry to appearance within the diffusion process. MoCam first leveragesgeometric priors in early stages to anchor coarse structures and toleratetheir incompleteness, then switches to appearance priors in later stagesto actively correct geometric errors and refine details. This design natu-rally unifies static and dynamic view synthesis by temporally decouplinggeometric alignment and appearance refinement within the diffusion pro-cess. Experiments demonstrate that MoCam significantly outperformsprior methods, particularly when point clouds contain severe holes ordistortions, achieving robust geometry-appearance disentanglement.

Multimodal Large Language Models (MLLMs) have demon-strated promising spatial reasoning capabilities, while these abilities re-main underexplored in the emerging visual modality of panoramic im-agery. The full 360◦ ×180◦ field of view of panoramas essentially supportscomplex global multi-step reasoning, which is also the fundamental ad-vantage of panoramas in applications such as embodied intelligence. How-ever, existing panoramic benchmarks largely focus on simplistic queriesthat rely on local cues or single-/few-step reasoning, thereby ignoring thefundamental advantage of panoramas and failing to fully exploit theirpotential. To address this gap, we introduce OmniCoT , a panoramicspatial reasoning suite designed to enable MLLMs to use global evi-dence and perform multi-step inference across viewpoints. It includesOmniCoT-B (6.7K data) for evaluation, which measures both answeraccuracy and reasoning quality, OmniCoT-Real (1K data) as a man-ually annotated real-world subset to quantify the Sim-to-Real gap. Fortraining, OmniCoT-T (14.3K data) is purpose-built with structuredstepwise Chain-of-Thought annotations that explicitly link intermediatereasoning steps to panoramic evidence. Based on OmniCoT-T, we intro-duce OmniCoT-R1 and adopt a two-stage training strategy tailoredto the geometrically complex panoramic space, where Supervised Fine-Tuning (SFT) anchors reasoning to panoramic evidence (e.g., bearings,proximity) and GRPO penalizes geometrically incoherent paths to con-solidate global 360◦ spatial consistency. Through OmniCoT, we aim torecalibrate the difficulty of panoramic spatial reasoning to bet-ter align with the intrinsic capabilities of panoramic imagery ,thereby fostering meaningful progress in this research area.

Autoregressive (AR) models deliver high-quality image gen-eration but suffer from severe inference latency due to sequential decod-ing. While speculative decoding (SD) successfully accelerates large lan-guage models, we demonstrate that state-of-the-art methods like EAGLE-3 provide limited gains for visual AR models. We attribute this to theintrinsic nature of visual tokens: unlike text, their probability distribu-tions are flat and non-discriminative, which hinders drafter–target align-ment and causes verification to collapse under sampling. To bridge thisgap, we propose LISA, a Locality-Informed Speculative framework forAutoregressive image generation. LISA overcomes alignment barriers viatwo key components: (i) Locality-Informed Distillation, which alignsthe drafter by prioritizing supervision on structured uncertainty; and(ii) Geometry-Aware Soft Verification, which leverages embeddingproximity and target confidence to enable more permissive yet safe to-ken acceptance. Experiments across T2I and C2I tasks demonstrate thatLISA improves the speed–quality trade-off, achieving up to 4.12× accel-eration and outperforming recent competitive methods.

Large vision–language models (LVLMs) exhibit strong reason-ing ability but suffer from visual forgetting during long-horizon decoding,where attention progressively drifts away from visual evidence. Existingmethods largely treat this issue as a late-stage attention decay prob-lem or attempt to mitigate it through heuristic reminders or post-hocattention lifting. Through systematic empirical analysis, we find thatperformance degradation under visual forgetting is largely driven bytwo overlooked factors: early-stage attention decay disrupts evidenceacquisition, and attention concentration on a subset of task-irrelevantvisual sink tokens. Motivated by these insights, we propose LASER, apost-training framework that regulates both the visual attention tra-jectory and intra-visual token attention distribution during reasoning.Technically, LASER introduces two complementary rewards: a VisualGrounding Reward, which encourages the model to maintain attentionon semantically salient visual tokens throughout decoding, and a SinkSuppression Reward, which penalizes excessive attention concentration onvisual sink tokens. Together, these rewards preserve early-stage groundingwhile preventing attention collapse onto uninformative regions. Exten-sive experiments on eight benchmark datasets demonstrate that LASERconsistently outperforms strong baselines, validating attention-awaretraining as an effective remedy for visual forgetting. The code is availableat https://github.com/KeViNYuAn0314/LASER.

Recent Vision-Language Models (VLMs) have achieved re-markable success in visual understanding, driven by the growing avail-ability of high-quality image-text pairs. However, the performance ofVLMs often degrades in the video domain due to the essential needfor temporal understanding and the scarcity of large-scale annotatedvideo data. In this work, we focus on marine video understanding, whichbrings further challenges: first, it requires substantial domain expertise;and video VLMs usually struggle with localizing and interpreting criti-cal information from marine videos, as the informative events are typ-ically sparse, unpredictable, and unevenly distributed. To address thesechallenges, we carefully curate the first event-centric marine video un-derstanding dataset called MarineEVT, which features 20K multi-task,video-level visual question-answering pairs spanning multiple dimensionsof marine understanding and analysis. Meanwhile, based on MarineEVT,we decompose marine video understanding as an Event-centric VisualTool-integrated Reasoning process (EVT-R1 for short), where we lever-age powerful visual tools to drive the model to localize and interpretcritical information aligned with visual questions and human intent. Todemonstrate its effectiveness, we compare EVT-R1 against 11 SOTAVLMs in different settings. EVT-R1 outperforms the top open-sourceand top commercial models by 5.22 and 11.09, respectively. MarineEVTand EVT-R1 lay the foundation for ecological discovery and marine ed-ucation, fostering the development of VLMs capable of interpreting ma-rine dynamics, reasoning about ecological interactions, and supportingsustainable ocean video understanding and analysis.

Vision-Language-Action (VLA) models have emerged as apromising paradigm for building general-purpose robotic agents. How-ever, the VLA landscape remains highly fragmented and complex: asexisting approaches vary substantially in architectures, training data,embodiment configurations, and benchmark-specific engineering. In thiswork, we introduce StarVLA-α, a simple yet strong baseline designedto study VLA design choices under controlled conditions. StarVLA-αdeliberately minimizes architectural and pipeline complexity to reduceexperimental confounders and enable systematic analysis. Specifically,we re-evaluate several key design axes, including action modeling strate-gies, robot-specific pretraining, and interface engineering. Across uni-fied multi-benchmark training on LIBERO, SimplerEnv, RoboTwin, andRoboCasa, the same simple baseline remains highly competitive, indi-cating that a strong VLM backbone combined with minimal design isalready sufficient to achieve strong performance without relying on ad-ditional architectural complexity or engineering tricks. Notably, our sin-gle generalist model outperforms π0.5 by 20% on the public real-worldRoboChallenge benchmark. We expect StarVLA-α to serve as a solidstarting point for future research in the VLA regime. Code will be re-leased at https://github.com/starVLA/starVLA.

While Multimodal Large Language Models (MLLMs) ex-cel in visual understanding, the quadratic complexity imposed by densehigh-resolution visual tokens poses significant challenges for edge-devicedeployment. Current training-free pruning strategies predominantly de-pend on attention weights. However, viewed from the frequency domain,the self-attention mechanism functions as a low-pass filter, causing thediscard of essential high-frequency signals (e.g., textures and edges) andresulting in feature over-smoothing. In this paper, we present Spatial-Window Integrated Frequency-aware Token Pruning (SWIFT), a seam-less plug-and-play framework designed for efficient token compression.SWIFT leverages two novel components: a Frequency-Aware Indicator(FAI), which identifies fine-grained details by estimating high-frequencyresiduals through matrix factorization, and a Spatial-Window Integra-tion (SWI) module, which prevents spatial structure collapse and posi-tional bias via localized retention. Extensive evaluations on Qwen2.5VL-3B and LLaVA benchmarks show that under a practical 50% compressionratio, SWIFT achieves a 2.10× speedup with near-lossless performance.Under an extreme 75% compression stress test, SWIFT yields an averageperformance drop of 5.1% across 8 benchmarks, while delivering a 2.66×speedup and 69.07% KV cache reduction. SWIFT exhibits superior ro-bustness in detail-intensive tasks compared to SOTA methods, providinga potent lightweight solution for on-device MLLM applications. Our codeis available at https://github.com/damo-lgl/SWIFT.

6-DoF LiDAR-based localization is a fundamental capabilityfor autonomous systems operating in large-scale outdoor environments.Many deep-learning-based localization methods have achieved promisingperformance so far. However, as one of the always-on modules competingfor limited on-board computational resources, the localization module isexpected to consume only a small portion of the overall compute bud-get. Most existing learning-based methods are still too heavy for thispurpose. In contrast, binary neural networks (BNNs) offer an appealingsolution, but the 1-bit compression causes severe information loss andperformance drop. In this paper, we address this challenge by propos-ing Binarized LiDAR-based Localization (BiLoc), the first binary neuralnetwork framework for 6-DoF LiDAR localization. Specifically, we rein-terpret the training of BNNs from the perspective of the information-bottleneck principle, aiming at retaining minimal yet sufficient represen-tations for pose estimation while suppressing redundant variations. Andwe introduce an auxiliary objective that adaptively regulates informationretention in the binary encoder, effectively mitigating the informationloss caused by binarization. This auxiliary objective provides additionaloptimization signals that compensate for the limited representationalcapacity and the gradient mismatch inherent in BNNs. Extensive exper-iments on large-scale outdoor LiDAR datasets demonstrate that BiLocestablishes a new state of the art for LiDAR localization with BNNs.

Tool-augmented vision–language models (VLMs) can solve multi-modal, multi-step tasks by calling external tools, yet they remain fragile in prac-tice. Existing works have two common gaps. Supervised fine-tuning (SFT) is builtmostly on successful trajectories and offers little signal for recovery after tool fail-ures, while sparse trajectory-level RL rewards provide limited guidance on whichstep failed and how to repair it. We introduce ReGRPO (Reflection-augmentedGroup Relative Policy Optimization), a framework that learns reflection-guidedcorrection in tool-using agents. ReGRPO starts with a structured reflective dataengine: we execute near-miss actions to collect grounded failure observations,then build Reflection-of-Thought triplets (ErrorType, Evidence, FixPlan) pairedwith corrected actions for warm-start SFT. We then optimize reflection tokens andcorrective actions jointly within local trajectories using group-relative advantages,and include a reflection-cost term to reduce unnecessary reflection. Experimentson GTA and GAIA show that, under the same backbone and tool suite, ReGRPOconsistently outperforms strong open-source baselines and achieves the best resultsamong the compared open-source controllers. Code and RoT data are available athttps://github.com/showlab/ReGRPO.

Vision-Language-Action (VLA) models have advancedrapidly, yet most gains still come from larger backbones, larger embod-ied datasets, or expensive iterative decoders, while action representa-tion remains under-explored. We argue that action space design is aprimary bottleneck and present LEAP (Latent-Enhanced ActionPrototyping)-VLA, a two-stage framework that learns a structuredaction latent space from demonstrations via Multi-level Soft ResidualQuantization (MSRQ), then trains a lightweight VLM-external aligner topredict actions by prototype-aware soft selection in that space. This de-sign replaces discrete code assignment and multi-step denoising with fullydifferentiable single-pass latent prediction, preserving pretrained vision-language alignment without modifying the VLM. Empirically, LEAP-VLA outperforms the state-of-the-art methods with substantially fewertrainable parameters and no embodied pretraining, showing that a well-structured continuous action latent space can offset model scale and datarequirements.

On-device learning is essential for personalization, privacy,and long-term adaptation in resource-constrained environments. Achiev-ing this requires efficient learning, both fine-tuning existing models andcontinually acquiring new tasks without catastrophic forgetting. Yet bothsettings are constrained by high memory cost of storing activations dur-ing backpropagation. Existing activation compression methods reducethis cost but rely on repeated low-rank decompositions, introducing com-putational overhead. Also, such methods have not been explored forcontinual learning. We propose LANCE (Low-rank Activation Compres-sion), a framework that performs one-shot higher-order Singular ValueDecomposition (SVD) to obtain a reusable low-rank subspace for ac-tivation projection. This eliminates repeated decompositions, reducingboth memory and computation. Moreover, fixed low-rank subspaces fur-ther enable on-device continual learning by allocating tasks to orthogo-nal subspaces without storing large task-specific matrices. Experimentsshow that LANCE reduces activation storage by 25× to 380× depend-ing on the architecture, while maintaining accuracy comparable to fullbackpropagation on CIFAR-10/100, Oxford-IIIT Pets, Flowers102, andCUB-200 datasets. On continual learning benchmarks (Split CIFAR-100,Split MiniImageNet, 5-Datasets), it performs competitively with orthog-onal gradient projection methods at a fraction of the memory cost. Theseresults position LANCE as a practical and scalable solution for efficientfine-tuning and continual learning on edge devices.3

The intrinsic link between facial motion and speech is oftenoverlooked in generative modeling, where talking head synthesis andtext-to-speech (TTS) are typically addressed as separate tasks. Thispaper introduces JAM-Flow, a unified framework to simultaneously syn-thesize and condition on both facial motion and speech. Our approachleverages flow matching and a novel Multi-Modal Diffusion Transformerarchitecture, integrating specialized Motion-DiT and Audio-DiT modules.These are coupled via selective joint attention layers and incorporate keyarchitectural choices, such as temporally aligned positional embeddingsand localized joint attention masking, to enable effective cross-modalinteraction while preserving modality-specific strengths. By analyzing andleveraging pretrained representation embeddings, JAM-Flow is designedfor efficient, near real-time sampling. Trained with an inpainting-styleobjective, JAM-Flow supports a wide array of conditioning inputs (in-cluding text, reference audio, and reference motion) facilitating taskssuch as synchronized talking head generation from text, audio-drivenanimation, and much more, within a single, coherent model. JAM-Flowsignificantly advances multi-modal generative modeling by providing apractical solution for holistic audio-visual synthesis. Project website

Unsupervised physical parameter estimation from video lacksa common benchmark: existing methods evaluate on non-overlappingsynthetic data, the sole real-world dataset is restricted to single-bodysystems, and no established protocol addresses governing-equation identi-fication. This work introduces IRIS, a high-fidelity benchmark comprising240 real-world videos captured at 4K resolution and 60 fps, spanning bothsingle- and multi-body dynamics with independently measured ground-truth parameters and uncertainty estimates. Each dynamical systemis recorded under controlled laboratory conditions and paired with itsgoverning equations, enabling principled evaluation. A standardized evalu-ation protocol is defined encompassing parameter accuracy, identifiability,extrapolation, robustness, and governing-equation selection. Multiplebaselines are evaluated, including a multi-step physics loss formulationand four complementary equation-identification strategies (VLM tempo-ral reasoning, describe-then-classify prompting, CNN-based classification,and path-based labelling), establishing reference performance across allIRIS scenarios and exposing systematic failure modes that motivate futureresearch. The dataset, annotations, evaluation toolkit, and all baselineimplementations are publicly released.

Di!usion models have emerged as a dominant paradigm ingenerative modeling, enabling high-fidelity sampling from complex datadistributions. Despite impressive capabilities, controlling di!usion mod-els to produce outputs aligned with user intent remains an open chal-lenge, especially when balancing global coherence with local precision.Existing control mechanisms vary in the granularity of their conditioningsignals. For example, textual prompts guide generation globally throughhigh-level semantics, while ControlNet-like approaches secure precise lo-cal structure via dense conditions. In this work, we introduce Histogram-constrained Image Generation (HIG), a novel control mechanism thatfalls into the middle ground of control granularity. Our framework en-forces user-specified distributional constraints (e.g., color histograms orlatent token distributions) during the generation process with exact pre-cision. We model such control as an optimal transport (OT) problemand apply explicit guidance transformations during sampling, therebydriving the di!usion trajectory to align with the desired histogram. Wedemonstrate the versatility of HIG across diverse applications, includingconstrained generation via color/latent histograms and high-capacity in-formation embedding through histogram-level encoding. Our findings un-derscore the promise of distributional control, a flexible and interpretablecontrol scheme that is fully compatible with existing control mechanisms,diversifying the hybrid strategies for controllable image generation. Ourproject page is available at: https://maps-research.github.io/hig/.

Robotic imitation learning faces a fundamental trade-off be-tween modeling long-horizon dependencies and enabling fine-grained closed-loop control. Existing fixed-frequency action chunking approaches strug-gle to achieve both. Building on this insight, we propose HiPolicy, ahierarchical multi-frequency action chunking framework that jointly pre-dicts action sequences at different frequencies to capture both coarsehigh-level plans and precise reactive motions. We extract and fuse hi-erarchical features from history observations aligned to each frequencyfor multi-frequency chunk generation, and introduce an entropy-guidedexecution mechanism that adaptively balances long-horizon planningwith fine-grained control based on action uncertainty. Experiments ondiverse simulated benchmarks and real-world manipulation tasks showthat HiPolicy can be seamlessly integrated into existing 2D and 3D gen-erative policies, delivering consistent improvements in performance whilesignificantly enhancing execution efficiency.