paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

ECCV 2026

Versatile Editing of Video Content, Actions, and Dynamics without Training

Vladimir Kulikov, Roni Paiss, Andrey Voynov, Inbar Mosseri, Tali Dekel, Tomer Michaeli

Controlled video generation has seen drastic improvementsin recent years. However, editing actions and dynamic events, or insert-ing contents that should affect the behaviors of other objects in real-world videos, remains a major challenge. Existing trained models strug-gle with complex edits, likely due to the difficulty of collecting relevanttraining data. Similarly, existing training-free methods are inherently re-stricted to structure- and motion-preserving edits and do not supportmodification of motion or interactions. Here, we introduce DynaEdit,a training-free editing method that unlocks versatile video editing ca-pabilities with pretrained text-to-video flow models. Our method relieson the recently introduced inversion-free approach, which does not in-tervene in the model internals, and is thus model-agnostic. We showthat naively attempting to adapt this approach to general unconstrainedediting results in severe low-frequency misalignment and high-frequencyjitter. We explain the sources of these phenomena and introduce novelmechanisms for overcoming them. Through extensive experiments, weshow that DynaEdit achieves state-of-the-art results on complex text-based video editing tasks, including modifying actions, inserting objectsthat interact with the scene, and introducing global effects (see website).

ECCV 2026

Towards Interactive Global Geolocation Assistant

Zhiyang Dou, Zipeng Wang, Xumeng Han, Guorong Li, Zhenjun Han, Zhipei Huang

Global geolocation, the task of predicting precise coordi-nates from street-view imagery, is inherently plagued by visual ambi-guity. Resolving such ambiguity necessitates a transition from static,one-shot predictions to an interactive geolocation paradigm drivenby multi-turn deductive reasoning. However, most of existing geoloca-tion models and general-purpose MLLMs fail to support this dynamicprocess, mainly due to the lack of interaction capabilities and the ge-ographic knowledge gap. Driven by the imperative to actualize thisinteractive paradigm, we introduce MG-Geo, the first large-scale mul-timodal geolocation dataset explicitly structured for spatial reasoning.Comprising 4.87M geo-tagged Meta entries, 70K image-grounded Cluesamples, and 73K multi-turn Dialog samples across 210 countries andterritories, MG-Geo separates large-scale geographic alignment fromreasoning-oriented supervision. Experiments demonstrate that GaGAachieves SOTA performance across several benchmarks. Notably, on theGWS15k dataset, it surpasses the strong Hybrid Model by 4.57% and2.92% at the country and city levels, respectively, while securing thehighest city-level accuracy (7.46%) on OSV-5M-test. More importantly,we formalize the “Similarity Trap”—a phenomenon where distributivevisual features mislead static models—and demonstrate that GaGA ef-fectively navigates this challenge through a Tiered Interaction Protocol.By dynamically integrating user-provided geographic anchors, GaGAachieves significant localization improvements. Our dataset is accessiblevia: https://huggingface.co/datasets/kendouvg/MG-Geo.

ECCV 2026

Think While You Map: Asynchronous Vision-Language Agents for Incremental 3D Scene Graphs

Deniz Bickici, Michael Pabst, Shohei Mori, Dieter Schmalstieg

Open-vocabulary 3D scene graph methods typically operatein two stages: first reconstruct, then enrich with vision-language mod-els, leaving the graph unqueryable during exploration. We argue thatthis sequential coupling is unnecessary and propose an asynchronous ar-chitecture in which lightweight online mapping runs concurrently withheavyweight semantic refinement. A probabilistic voxel-based backbonemaintains stable object identities incrementally, while background VLMagents progressively enrich the graph. This framework resolves duplicateobject tracks through semantic loop closure, attaches fine-grained visualattributes and derives spatial relations between objects. A multi-targetframe scheduler amortizes VLM cost by selecting a small set of infor-mative frames that jointly cover multiple targets. The resulting scenegraph is queryable during exploration and grows in semantic richnessover time. Our method matches or outperforms existing open-vocabulary3D scene graph methods on semantic segmentation (ScanNet, Replica)and surpasses the prior state-of-the-art across three visual groundingbenchmarks (Sr3D+, Nr3D, ScanRefer) by 15.3 to 18.8 A@0.25. Projectpage: https://denizbickici.github.io/thinkgraphs/

ECCV 2026

STAT: Soft Tail-dropping for Adaptive Visual Tokenization

Zeyuan Chen, Kai Zhang, Zhuowen Tu, Yuanjun Xiong

We present Soft Tail-dropping Adaptive Tokenizer (STAT),a discrete tokenizer that learns adaptive visual representations. STATadjusts the number of tokens allocated to each image according to itsperceptual complexity. Specifically, it encodes an image into discrete to-kens together with token-wise keep probabilities indicating whether eachtoken is necessary for faithful reconstruction or can be safely dropped.Through this learned adaptivity, STAT achieves state-of-the-art recon-struction quality while using fewer tokens on average. When integratedwith vanilla causal autoregressive (AR) modeling, STAT enables a content-aware generative model with adaptive-length sampling. The model achievescompetitive or superior visual generation quality compared with othergenerative model families while exhibiting favorable scaling behavior thathas been elusive in prior vanilla AR visual generation attempts.

ECCV 2026

Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery

Jie-Ying Lee, Yi-Ruei Liu, Shr-Ruei Tsai, Wei-Cheng Chang, Chung-Ho Wu, Jiewen Chan, Zhenjun Zhao, Chieh Hubert Lin, Yu-Lun Liu

Synthesizing large-scale, explorable, and geometrically accurate 3D urban scenes is a challenging yet valuable task for immersive and embodied applications. The challenge lies in the lack of large-scale and high-quality real-world 3D scans for training generalizable generative models. In this paper, we take an alternative route to create largescale 3D scenes by leveraging readily available satellite imagery for realistic coarse geometry and open-domain diffusion models for high-quality close-up appearance synthesis. We propose Skyfall-GS, a novel hybrid framework that synthesizes immersive city-block scale 3D urban scenes by combining satellite reconstruction with diffusion refinement, eliminating the need for costly 3D annotations, and also featuring real-time, immersive 3D exploration. We tailor a curriculum-driven iterative refinement strategy to progressively enhance geometric completeness and photorealistic texture. Extensive experiments demonstrate that SkyfallGS provides improved cross-view consistent geometry and more realistic textures compared to state-of-the-art approaches. Project page: https://skyfall-gs.jayinnn.dev/

ECCV 2026

Sen-Cap: Sensor-Flexible and Noise-Resilient Human Motion Capture via LiDAR-Camera Integration

Aoru Xue, YUJING SUN, Yiming Ren, Kwok-Yan Lam, Mao Ye, Yuexin Ma

We propose Sen-Cap, a Sensor-Flexible and Noise-Resilient3D human motion Capture framework that integrates multi-modal datafrom LiDAR and camera. While multi-modal sensors provide richer infor-mation than single-modal sensors, existing approaches still su_x001B_er fromtwo core challenges. First, multi-modal alignment/matching across ar-bitrarily deployed sensors is typically handled by explicit calibration,which propagates errors under changing viewpoints and in turn con-strains deployment to _x001C_xed, highly overlapped layouts. Second, priormethods degrade under severe noise or partial sensor failures, which areSen-common in real-world environments. To address these challenges,Cap introduces a Uni_x001C_ed Across-Sensor Motion Estimator that recon-structs local pose and shape in a human-centric space without calibra-tions between sensors, supporting a _x001D_exible number of sensors, as well asa Noise-Resistant Trajectory Tracker that maintains robustness under se-vere point cloud noise through iterative re_x001C_nement. These sensor-_x001D_exibleand noise-resilient features make Sen-Cap more practical in real-worlddeployment. Notably, operating in real time, Sen-Cap achieves state-of-the-art performance on major metrics on Human-M3 and FreeMotion,as well as strong cross-domain performance on LiDARHuman26M andRELI11D. This combination of _x001D_exibility and robustness opens new op-portunities for motion capture in real-world scenarios, e.g. sports ana-lytics, _x001C_eld robotics, and large-scale immersive environments.

ECCV 2026

Robustness Meets Uncertainty: Evidential Adversarial Training for Robust Selective Classification

Nicolas Sournac, Ahmed Baha Ben Jmaa, Bertrand Braeckeveldt

Safety-critical applications require classifiers that are bothrobust and reliable. Adversarial training is a widely adopted defense forimproving robustness in deep neural networks; however, its effect on thereliability of predictive uncertainty remains underexplored. We investi-gate this gap through the lens of selective classification, which has rarelybeen systematically analyzed alongside adversarial robustness. We in-troduce a unified benchmark for the robustness–uncertainty trade-off.It standardizes architectures, augmentations, threat models, and evalu-ation metrics across clean, adversarial, and common-corruption settings.Across a wide range of state-of-the-art adversarial training methods,we uncover a recurring failure mode: several approaches improve robustaccuracy while degrading uncertainty ranking, leading to poorer selec-tive behavior. To address this, we propose Evidential Adversarial Train-ing (EV-AT), which models uncertainty through a Dirichlet distributionand combines (i) an evidence-based loss promoting clean accuracy andreliable uncertainty with (ii) a robust evidence-alignment loss match-ing clean and adversarial predictions in log Dirichlet-parameter space.Extensive experiments show that EV-AT shifts the Pareto frontier ofrobustness–uncertainty trade-offs beyond prior state-of-the-art adversar-ial training methods. Our source code is publicly available at https://github.com/NicolasSournac/Robustness_Meets_Uncertainty.EV-AT.

ECCV 2026

RGBT-GroundBench: Visual Grounding Beyond RGB in Complex Real-World Scenarios

Tianyi Zhao, Jiawen Xi, Linhui Xiao, Junnan Li, Xue Yang, Maoxun Yuan, Xingxing Wei

Visual grounding (VG) localizes target objects in an imagefrom natural-language expressions. In real-world perception, RGB cuesoften degrade under low illumination and adverse weather, making visualgrounding substantially more challenging. However, existing VG bench-marks are largely RGB-only and provide limited, structured coverage ofsuch conditions, hindering systematic robustness evaluation and cross-spectral comparison. We present RGBT-GroundBench, the first large-scale benchmark for RGB-Thermal (TIR) visual grounding in complexenvironments. It contains over 40K images (21,535 RGB-TIR pairs) and38,760 object instances with referring expressions, bounding boxes, andfine-grained annotations at three levels: scene types, environmental con-ditions (illumination and weather), and object properties (size and oc-clusion). As a benchmark suite, RGBT-GroundBench provides not onlycurated RGB-TIR grounding annotations but also a unified evaluationprotocol supporting RGB-only, TIR-only, and RGB+TIR inputs. Underthis protocol, we benchmark 11 representative VG models across diversescenes and environmental conditions. Our results show that groundingaccuracy is strongly correlated with scene complexity, LoRA-based mod-els are more robust in complex scenes, and low-illumination conditionscause significant performance degradation that has been rarely explored.Guided by these observations, we introduce RGBT-VGNet, a simpleand reproducible reference baseline under the unified protocol, featuringAsymmetric Modality Adaptation, Language-Aware Visual Synergy, andTri-Prior Fusion for reliability-aware RGB-TIR integration. Resources,annotations, code, checkpoints, and evaluation scripts have been publiclyreleased.

ECCV 2026

ReliefSAM: A Geometry-Augmented Multi-Prior Adapter for Bas-Relief Segmentation

Yihang Chen, Xiang Lyu, Rui Xu, Jiao PAN, Fadjar Thufail, Brahmantara Brahmantara, Jiaqing Liu, Satoshi Tanaka, Liang Li

Bas-relief sculptures are vital historical records. However,many are inaccessible for modern 3D scanning due to damage or occlu-sion, leaving archival monocular photographs as the only available visualdata. Semantic segmentation, such as isolating human figures in theseimages, is essential for digital documentation and subsequent archaeo-logical analysis, yet remains highly challenging for foundation modelslike the Segment Anything Model (SAM). The intrinsic properties ofbas-reliefs, including extreme material homogeneity, subtle depth varia-tions, and indistinct soft edges, result in a severe lack of both chromaticand geometric contrast. Consequently, models pretrained on natural im-agery struggle to perceive the shallow 2.5D structural cues embeddedin relief surfaces, leading to imprecise boundary delineation. To addressthis limitation, we propose ReliefSAM, a prompt-free and parameter-efficient framework that explicitly incorporates image-aligned depth andedge priors derived from a single monocular RGB photograph. These ge-ometric cues are encoded via a lightweight Multi-Prior Feature Encoder(MFE) and injected into a frozen SAM backbone through adapter-basedinteraction blocks. For high-resolution archival imagery, we further adoptoverlapped sliding-window inference with Gaussian-weighted merging toensure spatial consistency. Extensive experiments demonstrate that whileadapting the frozen SAM image encoder with lightweight adapters es-tablishes a highly competitive pure-RGB baseline that outperforms stan-dard decoder-only fine-tuning, naively injecting a single geometric priorcan induce modality interference. Crucially, ReliefSAM’s joint integra-tion of depth and soft-edge priors achieves a geometric consensus inwhich the two priors counterbalance and neutralize prior-specific biases.This synergistic 2.5D guidance breaks the RGB-only performance ceil-ing, effectively bridging the gap between natural RGB appearance andheritage-specific structures without requiring large-scale retraining of thefoundation model.

ECCV 2026

Raymap-Guided Coupling for Drift-Robust Unposed Feed-Forward 3D Reconstruction

Xiangyu Sun, Liu.Liu Liu.Liu, Seungkwon Yang, Jingbing Han, Seungtae Nam, Zhizhong Su, Eunbyung Park

Pose-Free Feed-forward 3D Gaussian Splatting (3DGS) hasrecently emerged as a powerful paradigm for fast scene reconstruction.However, its performance degrades significantly in long image sequencesdue to cumulative camera pose estimation drift, which propagates er-rors into geometric modeling and severely limits rendering fidelity. Inthis work, we revisit the long-sequence bottleneck and identify posedrift as the primary factor restricting reconstruction quality. Further-more, while SfM-based pseudo ground-truth poses introduce sensor noise,purely rendering-based supervision often leads to optimization instabil-ity and local minima due to the entangled optimization of geometryand pose. To address the challenges, we propose a synergistic pose-free framework that explicitly couples geometry and appearance via aRaymap-Guided Coupling Module (RGC). Concretely, we anchor Gaus-sian centers to raymap-induced geometry and jointly optimize RGB re-construction, raymap consistency, and camera regularization under a uni-fied objective, yielding a bidirectional feedback loop: stronger geometryimproves rendering, and appearance supervision in turn refines geometryand pose. To further stabilize learning across wide temporal ranges, weintroduce a Dual-Frequency Viewpoint Scheduling strategy that com-bines easy-to-hard interval expansion with replay of short-interval pairs.Extensive experiments across in-domain and cross-domain datasets showconsistent gains in both rendering and pose estimation, with notably im-proved robustness on long sequences. Ablation studies validate our cen-tral insight: explicitly designed geometry-appearance synergy is the keyto scalable and drift-robust pose-free feed-forward 3D reconstruction.

ECCV 2026

Prosthesis-Aware 3D Human Pose Estimation: A Dataset and Benchmark for RSP Users

Yilin Wen, Kechuan Dong, Fumiya Suginaka, Ken Endo, Yusuke Sugano

Recovering 3D human body motion from video is impor-tant for applications such as rehabilitation assessment and sports per-formance evaluation. For prosthesis users, this requires capturing bothnatural body joints and the geometry of the prosthetic device, a challengethat existing methods are not designed to address. Model-based estima-tors rely on body models trained on non-amputee individuals and cannotrepresent prosthesis geometry, while model-free methods lack body kine-matic priors and are unreliable under occlusion. This challenge is partic-ularly prominent for users of running-specific prostheses (RSPs), wherethe RSP has a complex curved geometry and moves dynamically duringexercise. To fill this gap, we collect RSP3D, the first 3D dataset of RSPusers, covering essential daily-life and exercise actions from participantswith varied amputation conditions, using a multi-camera marker-basedmotion capture setup. We formally define the task of prosthesis-aware 3Dpose estimation, evaluate representative methods in a zero-shot setting,and confirm their individual limitations. We further propose a hybridbaseline combining model-based body joint estimation with model-freeRSP shape recovery, establishing a starting point for future research.Our project page is available at https://ut-vision.github.io/RSP3D/

ECCV 2026

NoPA: Non-Parametric Online 3D Scene Graph Generation

Qi Xun Yeo, Seungjun Lee, Yan Li, Gim Hee Lee

Classic 3D scene graph generation approaches fail to workin real-time due to the heavy computational cost of environment map-ping and the need to generate intermediate point-cloud representations.To alleviate this issue, a recent work eschews point clouds in favor ofa lightweight Gaussian distribution for each object. This approximationdrastically speeds up inference and enables real-time 3D scene graph gen-eration. However, the representation has two key weaknesses. 1) Each ob-ject is approximated by a single 3D Gaussian, which causes a severe lossof 3D geometric detail. 2) The discrepancy between this approximationand the true object geometry exacerbates the inaccurate merging of ob-ject candidates during online inference. To address these issues, we pro-pose NoPA, which represents each object as a separate non-parametricdistribution. This formulation retains 3D geometric information whilepreserving real-time inference of the parametric Gaussian formulation. Tobuild upon our novel object representation, we propose a tailored merg-ing strategy to recover coherent object instances. Specifically, we leveragemaximum mean discrepancy on kernel density estimates to enable robustmerging of object candidates during online exploration while minimizingadded computational complexity. The key is to maintain a fixed particleset per object. Furthermore, to rectify the relation loss caused by misclas-sified objects, NoPA propagates relationships between objects with highaffinity. Experiments show that NoPA substantially outperforms currentmethods without sacrificing real-time inference speed.

ECCV 2026

Point2Pose: Occlusion-Recovering 6D Pose Tracking and 3D Reconstruction for Multiple Unknown Objects Via 2D Point Trackers

Tzu-Yuan Lin, Ho Lee, Kevin Doherty, Yonghyeon Lee, Sangbae Kim

We present Point2Pose, a model-free method for causal 6Dpose tracking of multiple rigid objects from monocular RGB-D video.Initialized only from sparse image points on the objects, our approachtracks multiple unseen objects without requiring object CAD modelsor category priors. Point2Pose leverages a 2D point tracker to obtainlong-range correspondences, enabling instant recovery after complete oc-clusion. Simultaneously, the system incrementally reconstructs an on-line Truncated Signed Distance Function (TSDF) representation of thetracked targets. Alongside the method, we introduce a new multi-objecttracking dataset comprising both simulation and real-world sequences,with motion-capture ground truth for evaluation. Experiments show thatPoint2Pose trades some single-object pose accuracy for broader model-free tracking capabilities, including multi-object tracking and recoveryfrom complete occlusion. Project page: https://point2pose.github.io/.

ECCV 2026

PixelU: A U-Shaped Transformer for Efficient End-to-End Pixel Diffusion

Zipeng Guo, Lichen Ma, Yu He, Xiaolong Fu, Jingling Fu, Junshi Huang, Yan Li

End-to-end pixel-space diffusion models bypass the lossy com-pression of Latent Diffusion Models (LDMs) but struggle to jointly modellow-frequency semantics and high-frequency signals in high-dimensionalspace. Existing works heavily rely on complex pixel decoders to alleviatethis issue. In this paper, we challenge this trend by revealing that thesedecoders primarily compensate for the optimization difficulties inherentto velocity prediction (v-prediction). Under the clean data paradigm (x-prediction), they are redundant. Motivated by this insight, we advocatefor simplicity over complexity and introduce PixelU, a minimalist, single-stage U-shaped Diffusion Transformer tailored for pixel space. PixelUabandons auxiliary decoders in favor of zero-cost skip connections, whichprovide an “information highway” that directly routes uncorrupted high-frequency spatial details from shallow to deep layers. To further enablethe backbone to focus exclusively on modeling low-frequency semantics,we introduce a constant-channel spatial down-sampling mechanism asa natural low-pass filter, which compresses deep features into a com-pact, low-frequency semantic manifold. Extensive experiments demon-strate that this decoupling of frequencies could outperform the strongbaseline (JiT-G) with only about 1/3 of its computation cost. On Im-ageNet 256 × 256 and 512 × 512, PixelU achieves FID of 1.63 and 1.92respectively, surpassing recent pixel-space methods and establishing asimple yet powerful new paradigm for end-to-end diffusion models. Codewill be available at https://github.com/gzp6688/PixelU.

ECCV 2026

PhysEdit: Physically Consistent Image Editing via Causal Enforcement

Siqi Wan, Jingwen Chen, Yehao Li, Yingwei Pan, Ting Yao, Tao Mei

While instruction-guided image editing has seen significantstrides, most existing models fail to preserve physical laws and causal-ity. This limitation stems from the sparsity of physical supervision sig-nals, coupled with the static, non-causal architecture of prevailing frame-works, which together hinder a deep comprehension of the physical world.To bridge this gap, we present PhysEdit, a novel framework that en-forces physical consistency in image editing through causal generationand physics-aware reinforcement learning. Specifically, by harnessing theintrinsic physical priors of video generative models, we curate a high-quality dataset, dubbed PhysEdit-50K, wherein every sample encapsu-lates the complex physical causalities and dynamics involved in the tran-sition between input and edited images. Building upon this, a specializedCausal Image Editor is devised to factorize the image editing processalong the temporal evolution of visual transformations, thereby internal-izing the underlying physical laws and boosting causal coherence. Simul-taneously, an additional regularization term is incorporated to maintainvisual consistency in unedited regions beyond the simple flow matchingobjective, mitigating the impact of undesired viewpoint shifts or cam-era motions in the dataset synthesized by video models. Furthermore,we integrate physics-aware reinforcement learning with a tailored fine-grained reward to steer the editing process toward better adherence tophysical laws. Extensive experiments on PICABench and ImgEdit-Benchdemonstrate that our PhysEdit significantly outperforms state-of-the-artbaselines, yielding physically consistent and plausible edits. Code is pub-licly available at: https://github.com/HiDream-ai/PhysEdit/.

ECCV 2026

P-MTP: Efficient Document Parsing via Multi-Token Prediction with Progressive Depth Scaling

Le Xiang, Chenxi Zhai, Shu Wei, Jingjing Wu, Qunyi Xie, Xiao Tan, KunbinChen KunbinChen, Wei He

Vision-Language Models (VLMs) have revolutionized docu-ment parsing by enabling end-to-end mapping from images to structuredtext, imposing a significant latency bottleneck, particularly for token-dense documents. While Multi-Token Prediction (MTP) has emergedas a promising approach for accelerating inference, its potential is con-strained by optimization instability when scaling to deeper look-aheaddepth. In this paper, we propose P-MTP, a framework that leveragesProgressive Multi-Token Prediction with a lightweight MTP mod-ule to scale the look-ahead depth for high-throughput document parsing.Specifically, we introduce Progressive Curriculum Loss that adaptivelyre-weights different look-ahead depths using cumulative path reliabilityand retrospective target consistency. By effectively suppressing gradientnoise in long-range predictions, P-MTP facilitates an automated easy-to-hard optimization transition, enabling the model to master increasinglydistant look-ahead depths. Furthermore, we propose Confidence-GatedDynamic Drafting to maximize the effective look-ahead depth and ac-ceptance rate by adaptively calibrating speculative length during infer-ence, thereby minimizing computational waste and further pushing theboundaries of inference speedup. Experimental results across multiplebenchmarks and architectures demonstrate that P-MTP achieves up toa 5× speedup with negligible loss in accuracy, providing the first suc-cessful validation of extensive look-ahead MTP in the document parsingdomain.

ECCV 2026

RAU: Reference-based Anatomical Understanding with Vision-Language Models

Yiwei Li, Yikang Liu, Jiaqi Guo, Lin Zhao, Zheyuan Zhang, Xiao Chen, Boris Mailhe, Ankush Mukherjee, Terrence Chen, Shanhui Sun

Anatomical understanding, which is the ability to identify,localize, or segment anatomical structures, is critical in medical imageanalysis; however, its progress is constrained by the scarcity of expert-labeled data. A promising remedy is to leverage an annotated referenceimage to guide the interpretation of an unlabeled target. Although re-cent vision–language models (VLMs) exhibit non-trivial visual reasoning,their reference-based understanding and fine-grained localization remainlimited. We introduce RAU, a framework for reference-based anatomicalunderstanding with VLMs. We first show that a VLM learns to identifyanatomical regions through relative spatial reasoning between referenceand target images, trained on a moderately sized dataset. We validatethis capability through visual question answering (VQA) and boundingbox prediction. Next, we demonstrate that the VLM-derived spatial cuescan be seamlessly integrated with the fine-grained segmentation capabil-ity of SAM2, enabling localization and pixel-level segmentation of smallanatomical regions, such as vessel segments. Across two in-distributionand two out-of-distribution datasets, RAU consistently outperforms aSAM2 fine-tuning baseline using the same memory setup, yielding moreaccurate segmentations and more reliable localization. More importantly,its generalization ability to unseen modalities makes it scalable to unseendatasets, a property crucial for medical image applications. To the bestof our knowledge, RAU is the first to explore the capability of VLMs forreference-based identification, localization, and segmentation of anatom-ical structures in medical images. Its promising performance highlightsthe potential of VLM-driven approaches for anatomical understandingin automated clinical workflows.

ECCV 2026

ORION: Ordinal Neural Collapse as a Representation Prior for Visual Navigation

E-In Son, Jung-Taak Kim, Seung-Woo Seo

Learning robust navigation policies directly from visual ob-servations remains a fundamental challenge in vision-based robotic nav-igation. In end-to-end imitation learning approaches, the visual encoderand action decoder are jointly optimized using a single action loss, whichprovides only an indirect supervisory signal to the encoder. This indi-rect supervision frequently results in the encoder learning ambiguous,action-agnostic representations. The problem is further complicated bysubstantial variations in scene structure and appearance across diverseenvironments, as well as the prevalence of visual distractors inherent toreal-world navigation settings. Such action-agnostic features cause thenavigation policy to produce inconsistent actions at ambiguous deci-sion points, leading to navigation failure. To overcome these limitations,we propose ORION (Ordinal Neural Collapse for Visual Navigation), amethod that explicitly organizes the encoder’s representation space ac-cording to the ordinal structure of navigation actions. In the context ofgoal-directed navigation, ego-centric control categories from Far Left toFar Right exhibit a natural ordinal relationship in which neighboringclasses share similar visual contexts, while semantically opposing classesdiffer substantially in appearance. We encourage class representations tobe arranged sequentially along a single discriminative axis, while sup-pressing off-axis variance within each class. The pretrained encoder isthen integrated into a diffusion-based navigation framework, and the fullpipeline is fine-tuned end-to-end. Extensive experiments in both simula-tion and real-world settings show that ORION consistently outperformsend-to-end and neural collapse baselines in navigation success rate andgoal progress, with notable gains in visually challenging scenarios suchas complex multi-way intersections.

ECCV 2026

NearID: Identity Representation Learning via Near-identity Distractors

Aleksandar Cvejic, Rameen Abdal, Abdelrahman Eldesokey, Bernard Ghanem, Peter Wonka

When evaluating identity-focused tasks such as personalizedgeneration and image editing, existing vision encoders entangle objectidentity with background context, leading to unreliable representationsand metrics. We introduce a principled framework for matched-contextidentity confusion using Near-identity (NearID) distractors, where se-mantically similar but distinct instances are placed on the exact samebackground as a reference image, eliminating contextual shortcuts andisolating identity as the sole discriminative signal. Based on this prin-ciple, we present the NearID dataset (19K identities, 316K matched-context distractors) together with a strict margin-based evaluation pro-tocol. Under this setting, pre-trained encoders perform poorly, achievingSample Success Rates (SSR), a strict margin-based identity discrimina-tion metric, as low as 30.7% and often ranking distractors above truecross-view matches. We address this by learning identity-aware repre-sentations on a frozen backbone using a two-tier contrastive objectiveenforcing the hierarchy: same identity > NearID distractor > randomnegative. This improves SSR to 99.2%, enhances part-level discrimina-tion by 28.0%, and yields stronger alignment with human judgments onDreamBench++, a human-aligned benchmark for personalization.

ECCV 2026

MoScale: Autoregressive Next-Scale Prediction for Human Motion Generation and Editing

Inwoo Hwang, Hojun Jang, Bing Zhou, Jian Wang, Young Min Kim, Chuan Guo

We present ScaleMoGen, a scale-wise autoregressive frame-work for text-driven human motion generation. Unlike conventional au-toregressive approaches that rely on standard next-token prediction, Scale-MoGen frames motion generation as a coarse-to-fine process. We quantize3D motions into compositional discrete tokens across multiple skeletal-temporal scales of increasing granularity, learning to generate motionby autoregressively predicting next-scale token maps. To maintain struc-tural integrity, our motion tokenizers and quantizers are explicitly de-signed so that discrete tokens at every scale strictly preserve the skeletalhierarchy. Additionally, we employ bitwise quantization and prediction,which efficiently scale up the tokenizer vocabulary to preserve motiondetails and stabilize optimization. Extensive experiments demonstratethat ScaleMoGen achieves state-of-the-art performance, establishing anFID of 0.030 (vs. 0.045 for MoMask) on HumanML3D and a CLIP Scoreof 0.693 (vs. 0.685 for MoMask++) on the SnapMoGen dataset. Further-more, we demonstrate that our skeletal-temporal multi-scale representa-tion naturally facilitates training-free, text-guided motion editing.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.