paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

ECCV 2026

General Self-Calibration with Varying Intrinsics

Norio Kosaka, Timothy Duff, Tomas Pajdla, Akihiro Sugimoto

We address multiview self-calibration from image correspon-dences when camera intrinsic parameters vary across views. While Kruppaequations constrain an implicit matrix encoding of the camera intrinsics(called the dual image of the absolute conic; DIAC), practical calibrationpriors are expressed directly in intrinsic-parameter space. We formulatesuch priors algebraically and map them into DIAC space, yielding explicitconstraints that integrate directly with Kruppa relations. Unlike priorself-calibration methods that focus on specific varying-intrinsic regimes(e.g. shared focal length, fixed aspect, or zero skew), we provide a unifiedalgebraic framework that handles arbitrary intrinsic priors expressed aspolynomial constraints in the space of intrinsics, yielding a more flex-ible formulation of varying-intrinsic self-calibration. Since Kruppa sys-tems are projective and often algebraically dependent, we algorithmicallyconstruct locally independent square subsystems via Jacobian analysis toassess solvability and algebraic complexity under varying intrinsics. Ex-periments demonstrate parity with a few previous classical approachesin shared-focal settings as well as shared-principal-point settings andenable stable estimation for varying-intrinsic configurations previouslyconsidered unsolved, validated on synthetic and real dynamic-intrinsicsequences. Code is available at https://github.com/Rowing0914/Self-Calibration-Varying-Intrinsics.

ECCV 2026

Holistic Optimal Label Selection for Robust Prompt Learning under Partial Labels

Yaqi Zhao, Haoliang Sun, Yating Wang, Yongshun Gong, Yilong Yin

Prompt learning has gained significant attention as a parameter-efficient approach for adapting large pre-trained vision-language modelsto downstream tasks. However, when only partial labels are available, itsperformance is often limited by label ambiguity and insufficient supervi-sory information. To address this issue, we propose Holistic Optimal La-bel Selection (HopS), leveraging the generalization ability of pre-trainedfeature encoders through two complementary strategies. First, we designa local density-based filter that selects the top frequent labels from thenearest neighbors’ candidate sets and uses the softmax scores to identifythe most plausible label, capturing structural regularities in the featurespace. Second, we introduce a global selection objective based on optimaltransport that maps the uniform sampling distribution to the candidatelabel distributions across a batch. By minimizing the expected trans-port cost, it can determine the most likely label assignments. These twostrategies work together to provide robust label selection from both lo-cal and global perspectives. Extensive experiments on eight benchmarkdatasets show that HopS consistently improves performance under par-tial supervision and outperforms all baselines. Those results highlightthe merit of holistic label selection and offer a practical solution forprompt learning in weakly supervised settings. The code is available athttps://github.com/Qizhoay/HopS.

ECCV 2026

HieDG: A Hierarchical Discrete Geometry-Guided Framework for Multi-Animal Tracking

Chenxun Deng, Zhongde Zhang, Ye Yuan, Chengyang Zhang, Yifan Zhang, Bohao Chen, Hongying Yan, Hang Zhou, Hua Han, Xi Chen

Multi-animal tracking (MAT) is critical for wildlife monitor-ing and behavioral analysis, yet remains challenging due to uniform ap-pearance, high density, and irregular motion. Existing methods typicallyfollow heuristic- or query-based paradigms: the former relies on hand-crafted geometric associations without end-to-end optimization, whereasthe latter enables joint optimization but relies heavily on appearanceembeddings. In such conditions, continuous geometric embeddings canbe unstable, as small coordinate perturbations may disproportionatelyalter cross-frame attention weights, degrading identity association perfor-mance. To address this limitation, we propose HieDG, a HierarchicalDiscrete Geometry-guided tracking framework that reformulates geo-metric dynamics as structured discrete representations within a query-based tracker. Instead of directly using raw geometric signals, HieDGemploys a two-stage residual codebook to discretize position, scale, andvelocity cues, transforming unstable continuous geometry into struc-tured, stable discrete tokens. These tokens are aligned with visual em-beddings and integrated into the tracking queries to enhance identityconsistency. Extensive experiments on animal-specific benchmarks (Ani-malTrack, BFT, and BuckTales) demonstrate state-of-the-art associationperformance with significant improvements in HOTA, AssA, and IDF1.Additional evaluations on generic multi-object tracking benchmarks, in-cluding DanceTrack and SportsMOT, show competitive performance,indicating the broader applicability of discretized geometric modelingbeyond animal-specific scenarios.

ECCV 2026

Orthogonal Knowledge Refreshing for Domain-Incremental Object Detection

Aoting Zhang, Dongbao Yang, Chang Liu, Xiaopeng Hong, Can Ma, Yu Zhou

Domain-incremental object detection (DIOD) requires mod-els to continually adapt to new domains while preserving prior knowl-edge. Recently, parameter-efficient fine-tuning offers a promising avenue,wherein a pre-trained model is frozen and a small number of learnable pa-rameters are injected for downstream tasks. However, these methods riskoverwriting critical past knowledge, triggering inter-domain interferenceand performance degradation. To address this challenge, we propose Or-thogonal Knowledge Refreshing (OKR), a simple yet effective frameworkfor DIOD. OKR incrementally constructs independent domain-specificsubspaces via dedicated low-rank branches for each domain, which areseamlessly fused for a holistic decision, enabling conflict-free capacity ex-pansion without domain selection during inference. To minimize knowl-edge interference during fusion, we present a gradient-based orthogo-nal refreshing strategy that projects gradient updates of new domainsonto the orthogonal complement of the fused historical subspace, sup-porting continual adaptation without forgetting. Moreover, to mitigatesemantic fragmentation across domains, we enforce topology-aware con-sistency, aligning the semantic structures of old and new domains. Ex-tensive experiments validate the superiority of OKR, outperforming thebest exemplar-free method by significant margins of +5.6% and +6.5%mAP on the Pascal VOC and BDD100K series, respectively.

ECCV 2026

GTR: Guide-Then-Refine Token Compression for Training-Free Acceleration of Video-LLMs

Sijin Zhou, Wei Feng, Zhuang Qi, Zhonghua Wang, Lie Ju, Xiang An, Mehrtash Harandi, Zongyuan Ge

Video large language models (Video-LLMs) have achieved re-markable progress in video understanding tasks, but their heavy compu-tational overhead during inference severely limits deployment in resource-constrained scenarios. This highlights the urgent need for efficient infer-ence acceleration frameworks tailored to Video-LLMs. Existing accelera-tion methods typically apply a homogeneous processing approach to allvideo frames, disregarding the fact that each frame within a video clipconveys distinct visual information. Additionally, many of them under-utilize text queries, or add text guidance only through costly pre-trainingor intrusive LLM modifications, underexploiting the text-video align-ment that is critical in long-form video understanding where redundantvisual content is common. To address this limitation, we propose Guide-Then-Refine (GTR), a plug-and-play inference acceleration frameworkfor Video-LLMs that realizes token compression through a two-stagemechanism. First, for all tokens in each video frame, we compute bothglobal scores and text-aware scores. Based on the sum of these scores forall tokens in a frame, we dynamically adjust the token retention ratio foreach frame. Subsequently, according to the retention ratio, we calculatethe local score of each token to evaluate its importance within the currentframe, thereby selecting critical visual tokens. Extensive evaluations onmainstream video benchmarks and various Video-LLMs demonstrate theeffectiveness of our approach. With only 15% of visual tokens retained,our method maintains an average of 95.8% of the original performanceacross four benchmarks.

ECCV 2026

Global Pose Control for Generative View Synthesis in Normalized Object Coordinate Space

Zhibing Li, Amogh Gupta, Behnoosh Parsa, Dan Casas

Novel View Synthesis (NVS) enables the generation of un-seen views of a scene from a single or multiple images, allowing users tofreely explore an object from any viewpoint. Despite the recent impres-sive qualitative improvements of generative models for this task, existingmethods struggle to provide global and intuitive control of target view-points because they either use input-relative camera poses or are limitedto generating sparse global views. This lack of global pose control severelylimits the number of downstream tasks potentially enabled by NVS. Toaddress this limitation, we propose a novel approach for precise cameracontrol in a customizable Normalized Object Coordinate Space (NOCS),requiring single or few unposed images. Our method operates solely onthe absolute camera pose of the target view in NOCS, eliminating theneed for a relative world frame or camera poses of the input images. Un-like previous methods that treat NVS as a standalone generation task, weformulate it as an image editing problem and build upon state-of-the-artediting models to leverage their superior generalization capability. Cam-era information is injected as dedicated camera tokens via an in-contextmulti-modal conditioning strategy. To alleviate the inherent ambiguityof NOCS, we incorporate text descriptions that explicitly define the ob-ject’s canonical coordinate frame, which also enhances generalization tounseen object categories. Furthermore, we curate a high-quality datasetwith consistently aligned orientations and corresponding NOCS text def-initions. Extensive experiments demonstrate that our method robustlygenerates novel views with accurate and consistent orientations fromarbitrary unposed images across diverse categories, achieving state-of-the-art image quality and fidelity.

ECCV 2026

GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning

Kaixun Jiang, Yuzheng Wang, Junjie Zhou, Pandeng Li, Zhihang Liu, Chen-Wei Xie, Zhaoyu Chen, Yun Zheng, Wenqiang Zhang

We introduce GenAgent, an agentic framework that unifiesvisual understanding and generation. GenAgent overcomes the limita-tions of tightly coupled unified models (which suffer from understand-ing–generation trade-offs and high training data costs) and the struc-tural bottlenecks of existing complex multi-model pipelines through aflexible, decoupled architecture: Understanding is handled by a singlemultimodal model, while generation is achieved by treating image gen-eration models as invokable tools. Given an image prompt, GenAgentadaptively engages in reasoning, tool invocation, visual judgment, andreflection to iteratively refine outputs until the criteria are met. We em-ploy a two-stage training strategy: first, we cold-start the agent with su-pervised fine-tuning on high-quality tool-invocation and reflection datato bootstrap multi-turn agent behaviors; second, we perform end-to-endagentic reinforcement learning, combining pointwise rewards (final im-age quality) and pairwise rewards (reflection accuracy), and utilizinga round-aware trajectory resampling strategy to balance improvementsacross distinct capabilities. Without modifying the underlying genera-tors, GenAgent significantly boosts the base model (FLUX.1-dev) by23.6% on GenEval++ and 14.0% on WISE. Beyond substantial per-formance gains, our framework demonstrates three key emergent prop-erties: (1) cross-tool generalization to generators with varying capabili-ties, (2) test-time scaling with consistent improvements across interactionrounds, and (3) task-adaptive reasoning. Project page: https://deep-kaixun.github.io/genagent-page/

ECCV 2026

From Phase to Phenomenon: Self-Supervised Learning of Subsurface Scattering with Minimal Phase-shift Inputs

Arjun Majumdar, Raphael Braun, Andreas Engelhardt, Hendrik Lensch

We propose a self-supervised pretraining framework for learn-ing sub-surface scattering (SSS) light transport representations fromminimal input. Our method leverages a stereo projector–camera setupthat captures only eight high-frequency phase-shift profilometry (PSP)images per view to pretrain an encoder in a multi-view, multi-object set-ting. We introduce a tailored augmentation strategy for PSP-based SSSdata, and show that it significantly outperforms standard ImageNet-styleaugmentations for SSL pretraining. The pretrained encoder learns gen-eralizable SSS representations that transfer effectively to downstreamtasks, including spatially varying relighting and representation evalua-tion using a kNN classifier. Combined with a decoder, the model recon-structs dense scattering footprint responses, trained using a dedicatedcost function that improves accuracy, particularly for anisotropic foot-prints. An overview of our method is presented in Fig. 3. Despite usingonly eight input images per view, our approach generalizes to unseenobjects with complex geometry and material properties, achieving high-fidelity reconstructions while requiring orders of magnitude fewer imagesthan prior methods. Our code is publicly available at GitHub.

ECCV 2026

DiTex4D: Direct Text-Driven 4D Generation with Structured Latent Diffusion

Xiaozhe Chen, Mengqi Rong, Jian Liu, Shuhan Shen

Despite recent progress, direct text-driven 4D object gen-eration remains challenging yet highly desirable. In this paper, we in-troduce DiTex4D, a native text-to-4D generation framework that en-ables both text-driven 4D generation from scratch and 3D animationfrom static mesh. Built upon large-scale pre-trained 3D generation mod-els, our framework DiTex4D avoids intermediate text-to-video pipelinesand costly per-object optimization. Specifically, (i) we achieve 4D spa-tiotemporal consistency via inflating 3D attention with mixed-4D RoPEand tailored correlated noise injection strategy. (ii) To enable 3D anima-tion, we introduce a mask-based diffusion model conditioned on multi-view global context to maintain strict consistency with the initial frame.We further fine-tune the framework for 4D interpolation to synthesizehigh-frame-rate sequences with smoother motion. Extensive experimentsdemonstrate that DiTex4D can achieve higher-quality, semantically align-ed, and spatiotemporally coherent 4D object generation, surpassing mostexisting state-of-the-art text-to-4D generation methods.

ECCV 2026

DiscoVL: Unveiling Disentangled Cross-Modal Representation Learning via Orthogonal Adversarial Regularization for Vision-Language Models

Mengping Dong, Jinbao Li, Fei Li

Pre-trained vision-language models excel across varied per-ception tasks, but adapting them to novel downstream settings with-out sacrificing generalization remains non-trivial. Existing parameter-efficient prompt learning method often yields inconsistent representa-tions and fails to account for semantic distribution shifts. In this work,we present DiscoVL, a disentangled cross-modal representation learningframework that couples orthogonal adversarial regularization with struc-tured cross-modal alignment for vision-language models. To address theinsufficient cross-modal interaction, our DiscoVL designs a multi-branchlow-rank residual aligner that decomposes representations into subspacesand enables bidirectional cross-modal feedback between visual and tex-tual streams at each layer. Furthermore, while conventional triplet con-straints overfit features to class centroids, we design an orthogonal reg-ularization for adversarial triplet loss, which prevents centroid collapseand substantially boosts generalization. Evaluations on 15 benchmarksdemonstrate that DiscoVL delivers consistent improvements over state-of-the-art methods for base-to-novel generalization, cross-dataset evalu-ation, and few-shot learning.

ECCV 2026

Diffusion-SDPO: Safeguarded Direct Preference Optimization for Diffusion Models

Minghao Fu, Guo-Hua Wang, Tianyu Cui, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang

Text-to-image diffusion models deliver high-quality images,yet aligning them with human preferences remains challenging. We re-visit diffusion-based Direct Preference Optimization (DPO) for thesemodels and identify a critical pathology: enlarging the preference mar-gin does not necessarily improve generation quality. In particular, thestandard Diffusion-DPO objective can increase the reconstruction er-ror of both winner and loser branches. Consequently, degradation ofthe less-preferred outputs can become sufficiently severe that the pre-ferred branch is also adversely affected even as the margin grows. Toaddress this, we introduce Diffusion-SDPO, a safeguarded update rulethat preserves the winner by adaptively scaling the loser gradient ac-cording to its alignment with the winner gradient. A first-order analysisyields a closed-form scaling coefficient that guarantees the error of thepreferred output is non-increasing at each optimization step. Our methodis simple, model-agnostic, broadly compatible with existing DPO-stylealignment frameworks and adds only marginal computational overhead.Across standard text-to-image benchmarks, including validations on thelarge-scale FLUX.1-dev model, Diffusion-SDPO delivers consistent gainsover preference-learning baselines on automated preference, aesthetic,and prompt alignment metrics. Our code is available at https://github.com/AIDC-AI/Diffusion-SDPO.

ECCV 2026

Capturing Spectral and Spatial Patterns for Federated Remote Sensing Segmentation

Yixin Xue, Wenke Huang, Haonan Guo, Bo Du

Federated remote sensing (RS) semantic segmentation en-ables privacy-preserving collaborative training of dense prediction mod-els. However, its performance is severely hindered by the structured het-erogeneity inherent in RS image: sensor diversity causes distinct spec-tral shifts, while geographic variability induces spatial shifts. Existingmethods treat client heterogeneity as an unstructured distributional mis-match, failing to accommodate these coupled, RS-specific variations.To address this, we propose FedDap, a federated prototype adapta-tion framework that disentangles domain-conditioned variations from in-variant class semantics via Domain-Adaptive Prototypes (DAPs). DAPstransform globally shared semantic anchors through two specialized mod-ulation branches: a spectral branch for radiometric alignment and a spa-tial branch for multi-scale structural adaptation. A shift-aware gatingmechanism dynamically fuses these branches, producing class-conditionalprototypes tailored to local sensing and regional contexts. By employ-ing a pixel-to-prototype contrastive objective, we align local representa-tions with both the shared semantic references and the DAPs, effectivelymitigating cross-client mismatch. Extensive experiments across diversefederated RS benchmarks demonstrate that our method consistently out-performs state-of-the-art baselines.

ECCV 2026

Beyond Attention: Convolutional Global Context for Remote Sensing Change Detection

Zhenyu Yang, Gensheng Pei, Junzhu Mao, Xinhao Cai, Tao Chen, Yazhou Yao

Remote sensing change detection is caught in a long-standingtrade-off: reliable suppression of pseudo changes requires global contex-tual reasoning, while attention-based models incur prohibitive quadraticcomplexity, hindering deployment on resource-constrained UAV and satel-lite platforms. This paper proposes ChangeGCC, a fully convolutional,linear-time framework that delivers efficient global context for change de-tection. ChangeGCC core features two key components: Globally Condi-tioned Convolution that enable convolutional global routing, and a dual-temporal fusion scheme that enhances semantic consistency across scaleswhile highlighting genuine structural changes. Extensive experiments onmultiple benchmarks show that ChangeGCC achieves state-of-the-artperformance, reaching 85.28% IoU and 92.06% F1 on the LEVIR-CD benchmark, while substantially reducing parameters, FLOPs, andinference latency compared to attention-based counterparts.

ECCV 2026

A Comprehensive Analysis about Unsupervised Outlier Detection for Images

Zhonghang LIU, Siyuan Chen, Jingwen Yu, Changshuo Wang, Kunyang Li, Jiangbo Lu

Unsupervised outlier detection that automatically identifies whether visual systems involve anomalous images is a significant research topic. However, most approaches are limited to natural images because they rely on frozen and pre-trained feature extractors. So that their performance cannot be maintained in practical scenarios, especially for industrial inspection and medical imaging. In this work, we first revisit this task and then introduce a versatile unsupervised outlier detection framework to enrich the application domains. The core idea of this framework is to improve feature discriminativess via exploiting intrinsic distribution priors. Evaluated on 3 domains and 14 benchmark datasets, our proposed solution achieves state-of-the-art performance and significantly outperforms existing methods. More importantly, we show its plug-andplay property that can be integrated into diverse visual applications to improve their robustness, such as image classification and 3D reconstruction. Our code is available at https://github.com/zhliu-uod/VUOD.

ECCV 2026

Where and What: Long-Term Object Tracking in Egocentric Videos

Jacob Chalk, Saptarshi Sinha, Dima Damen, Yannis Kalantidis, Diane Larlus

The recently established ‘Out of Sight, Not out of Mind’ (OSNOM) task for egocentric videos focuses on tracking objects that are moved by the camera wearer, online, maintaining knowledge of instance locations throughout the video even when they leave the field of view or become heavily occluded. In this paper, we propose the first learningbased solution to the OSNOM task: Whareformer, a transformer-based model with two components: an updatable memory of established tracks and a track assignment module that associates observations with existing tracks in a feed-forward manner. Whareformer jointly reasons over evolving object appearance (what) and updated 3D location (where), and employs a dedicated New Track token to reason about novel objects. Thanks to its design choices of using relative distances and evolving track representations, Whareformer is trained on a small set of 56 videos but achieves SOTA performance on 260 long test videos from three datasets: EPIC-KITCHENS-100 (unseen videos), IT3DEgo, and HD-EPIC, with significant absolute improvements over prior work.

ECCV 2026

WeEdit: A Dataset, Benchmark and Glyph-Guided Framework for Text-centric Image Editing

Hui Zhang, Juntao Liu, Zongkai Liu, Liqiang Niu, Fandong Meng, Zuxuan Wu, Yu-Gang Jiang

Instruction-based image editing aims to modify specific con-tent within existing images according to user-provided instructions whilepreserving non-target regions. Beyond traditional object- and style-centricmanipulation, text-centric image editing focuses on modifying, translating,or rearranging textual elements embedded within images. However, exist-ing leading models often struggle to execute complex text editing precisely,frequently producing blurry or hallucinated characters. We attribute thesefailures primarily to the lack of specialized training paradigms tailoredfor text-centric editing, as well as the absence of large-scale datasetsand standardized benchmarks necessary for a closed-loop training andevaluation system. To address these limitations, we present WeEdit, a sys-tematic solution encompassing a scalable data construction pipeline, twobenchmarks, and a tailored two-stage training strategy. Specifically, wepropose a novel HTML-based automatic editing pipeline, which generates330K training pairs covering diverse editing operations and 15 languages,accompanied by standardized bilingual and multilingual benchmarks forcomprehensive evaluation. On the algorithmic side, we employ glyph-guided supervised fine-tuning to inject explicit spatial and content priors,followed by a multi-objective reinforcement learning stage to align genera-tion with instruction adherence, text clarity, and background preservation.Extensive experiments demonstrate that WeEdit outperforms previousopen-source models by a clear margin across diverse editing operations.

ECCV 2026

Linguistically-Aligned and Visually-Grounded Preference Optimization for Clinically-Augmented Medical Report Generation

Qiang Hu, Yuxuan Luo, Yingjie Guo, Hao Wang, Qimei Wang, Qiang Li, Zhiwei Wang

Despite significant advances in Medical Report Generation(MRG), the reliability remains constrained by the prevalence of factualerrors. While Direct Preference Optimization (DPO) has emerged as apromising post-training paradigm to enhance the performance of Super-vised Fine-Tuned (SFT) MRG models, existing DPO-based MRG meth-ods typically adopt a naive preference construction that directly pairsmodel-generated reports with ground truth reports. This strategy inad-vertently entangles critical clinical findings with clinically irrelevant lin-guistic characteristics, and fundamentally lacks explicit vision-languagealignment. To address these challenges, we propose DPO-Clin, a novelpost-training framework that focuses preference optimization on clinicalfindings and cross-modal alignment. First, we introduce the Entity-levelClinical Diagnostic (ECD) module to perform a precise entity-level fac-tual diagnosis. ECD guides the generation of linguistically-aligned re-port preference pairs, isolating clinical discrepancies from linguistic vari-ations. Second, to achieve fine-grained cross-modal alignment, we developM2 DPO, a retrieval-augmented multi-modal DPO variant that enforcestextual preference inversion triggered by visual context switches. Third,we locate correct yet highly uncertain predicted entities and apply coun-terfactual modifications to construct targeted preference data for latentrisk mitigation, thereby further enhancing the model reliability. Exten-sive experiments on two public chest X-ray datasets (MIMIC-CXR andIU X-Ray) and an in-house endoscopy dataset demonstrate that DPO-Clin significantly improves the SFT baselines on clinical-aware metrics.Furthermore, it achieves superior performance over existing DPO-basedMRG methods, exhibiting robust generalizability across distinct baselinearchitectures and diverse medical imaging modalities.

ECCV 2026

WebEyeTrack: Scalable Eye-Tracking for the Browser via On-Device Few-Shot Personalization

Eduardo Davalos, Yike Zhang, Namrata Srivastava, Yashvitha Thatigotla, Ashwin Tudur Sadashiva, Jorge Salas, Sun-Joo Cho, Amanda Goodwin, Gautam Biswas

With advancements in AI, appearance-based gaze estimationmethods have improved benchmark performance, but practical deploy-ment still lags behind commercial eye-trackers. Factors like model size,inference time, and privacy often go unaddressed. Meanwhile, webcam-based eye-tracking methods remain sensitive to head movement. To tacklethese issues, we introduce WebEyeTrack, a browser-native frameworkthat integrates a novel lightweight gaze model, metric head pose estima-tion, and on-device few-shot learning with as few as nine per-user calibra-tion samples (k ≤ 9). WebEyeTrack adapts to new users, achieving com-petitive performance with an error margin of 2.32 cm on GazeCaptureand real-time inference latency of 2.4 milliseconds on an iPhone 14. Ouropen-source code is available at github.com/RedForestAI/WebEyeTrack.

ECCV 2026

Walk through Paintings : Ego-centric World models from Internet Priors

Anurag Bagchi, Zhipeng Bao, Homanga Bharadhwaj, Yu-Xiong Wang, Pavel Tokmakov, Martial Hebert

What if a video generation model could not only imaginea plausible future, but the correct one – accurately reflecting how theworld changes with each action? We answer this by presenting the Ego-centric World Model (EgoWM), a simple, architecture-agnostic methodthat transforms any pre-trained video diffusion model into an action-conditioned world model, enabling precisely controllable future predic-tion. Rather than training from scratch, we repurpose the rich world pri-ors of Internet-scale video models by injecting appropriately compressedmotor commands through lightweight conditioning layers. This allowsour model to follow actions faithfully while preserving generalizationand realism. Our approach scales naturally across embodiments and ac-tion spaces – from 3-DoF mobile robots to 25-DoF humanoids, wherepredicting egocentric joint-angle driven dynamics is substantially morechallenging. The model produces coherent rollouts for both navigationand manipulation, requiring only modest fine-tuning. To evaluate phys-ical correctness independent of appearance, we introduce the StructuralConsistency Score (SCS), which measures whether stable scene elementsevolve consistently with the provided actions. Our method improves SCSby up to 65% over the prior state of the art, Navigation World Models;applies seamlessly to three different video diffusion model architectures;and effectively utilizes Internet priors to generalize to unseen environ-ments, including navigation and manipulation inside paintings. Finally,we demonstrate the applicability of EgoWM to robotic planning.

ECCV 2026

Wake up for Touch! Mask-isolated Tactile Alignment Learning in MLLMs

Yoonhyung Park, Minji Kim, Sungwon Moon, Jiyoung Lee

Touch supplies the physical grounding needed to perceive intrinsic material properties, such as friction and compliance, that vision alone often cannot resolve. Recent efforts for equipping multimodal LLMs with this tactile sense, however, expose a zero-sum trade-off: the limited parameter budget of compact models forces a choice between acquiring the new sensory modality and preserving the established visionlanguage reasoning. We present Splash, a mask-isolated tactile alignment learning framework for MLLMs. Splash quantifies the significance of each pretrained parameter, and partitions the parameter space into a dormant and critical subspace. While the frozen critical subspace acts as a stable anchor to safeguard general visual knowledge, Splash updates the isolated dormant subspace to internalize tactile alignment towards LLMs. This selective, non-destructive expansion effectively prevents catastrophic forgetting and ensures non-destructive modality expansion. Extensive experiments show that Splash effectively achieves tactile reasoning without additional inference overhead in the LLM part, demonstrating state-of-the-art performance on visuo-tactile benchmarks, including SSVTP, TVL, and TacQuad, while preserving its original generalpurpose capabilities.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.