ECCV 2026 论文列表

European Conference on Computer Vision

录用类型

2,834篇论文

按官方录用论文列表统计,可能与会议公布的数字略有出入。

  • Poster
    2,80398.9%
  • 其他 / 未标注
    311.1%

Top 10 主题(点击筛选)

暂无主题数据。

全部 207 个研究方向的趋势

浏览全部论文

Sen-Cap: Sensor-Flexible and Noise-Resilient Human Motion Capture via LiDAR-Camera Integration

Aoru Xue, YUJING SUN, Yiming Ren, Kwok-Yan Lam, Mao Ye, Yuexin Ma

展开摘要

We propose Sen-Cap, a Sensor-Flexible and Noise-Resilient3D human motion Capture framework that integrates multi-modal datafrom LiDAR and camera. While multi-modal sensors provide richer infor-mation than single-modal sensors, existing approaches still su_x001B_er fromtwo core challenges. First, multi-modal alignment/matching across ar-bitrarily deployed sensors is typically handled by explicit calibration,which propagates errors under changing viewpoints and in turn con-strains deployment to _x001C_xed, highly overlapped layouts. Second, priormethods degrade under severe noise or partial sensor failures, which areSen-common in real-world environments. To address these challenges,Cap introduces a Uni_x001C_ed Across-Sensor Motion Estimator that recon-structs local pose and shape in a human-centric space without calibra-tions between sensors, supporting a _x001D_exible number of sensors, as well asa Noise-Resistant Trajectory Tracker that maintains robustness under se-vere point cloud noise through iterative re_x001C_nement. These sensor-_x001D_exibleand noise-resilient features make Sen-Cap more practical in real-worlddeployment. Notably, operating in real time, Sen-Cap achieves state-of-the-art performance on major metrics on Human-M3 and FreeMotion,as well as strong cross-domain performance on LiDARHuman26M andRELI11D. This combination of _x001D_exibility and robustness opens new op-portunities for motion capture in real-world scenarios, e.g. sports ana-lytics, _x001C_eld robotics, and large-scale immersive environments.

Poster
Robustness Meets Uncertainty: Evidential Adversarial Training for Robust Selective Classification

Nicolas Sournac, Ahmed Baha Ben Jmaa, Bertrand Braeckeveldt

展开摘要

Safety-critical applications require classifiers that are bothrobust and reliable. Adversarial training is a widely adopted defense forimproving robustness in deep neural networks; however, its effect on thereliability of predictive uncertainty remains underexplored. We investi-gate this gap through the lens of selective classification, which has rarelybeen systematically analyzed alongside adversarial robustness. We in-troduce a unified benchmark for the robustness–uncertainty trade-off.It standardizes architectures, augmentations, threat models, and evalu-ation metrics across clean, adversarial, and common-corruption settings.Across a wide range of state-of-the-art adversarial training methods,we uncover a recurring failure mode: several approaches improve robustaccuracy while degrading uncertainty ranking, leading to poorer selec-tive behavior. To address this, we propose Evidential Adversarial Train-ing (EV-AT), which models uncertainty through a Dirichlet distributionand combines (i) an evidence-based loss promoting clean accuracy andreliable uncertainty with (ii) a robust evidence-alignment loss match-ing clean and adversarial predictions in log Dirichlet-parameter space.Extensive experiments show that EV-AT shifts the Pareto frontier ofrobustness–uncertainty trade-offs beyond prior state-of-the-art adversar-ial training methods. Our source code is publicly available at https://github.com/NicolasSournac/Robustness_Meets_Uncertainty.EV-AT.

Poster
RGBT-GroundBench: Visual Grounding Beyond RGB in Complex Real-World Scenarios

Tianyi Zhao, Jiawen Xi, Linhui Xiao, Junnan Li, Xue Yang, Maoxun Yuan, Xingxing Wei

展开摘要

Visual grounding (VG) localizes target objects in an imagefrom natural-language expressions. In real-world perception, RGB cuesoften degrade under low illumination and adverse weather, making visualgrounding substantially more challenging. However, existing VG bench-marks are largely RGB-only and provide limited, structured coverage ofsuch conditions, hindering systematic robustness evaluation and cross-spectral comparison. We present RGBT-GroundBench, the first large-scale benchmark for RGB-Thermal (TIR) visual grounding in complexenvironments. It contains over 40K images (21,535 RGB-TIR pairs) and38,760 object instances with referring expressions, bounding boxes, andfine-grained annotations at three levels: scene types, environmental con-ditions (illumination and weather), and object properties (size and oc-clusion). As a benchmark suite, RGBT-GroundBench provides not onlycurated RGB-TIR grounding annotations but also a unified evaluationprotocol supporting RGB-only, TIR-only, and RGB+TIR inputs. Underthis protocol, we benchmark 11 representative VG models across diversescenes and environmental conditions. Our results show that groundingaccuracy is strongly correlated with scene complexity, LoRA-based mod-els are more robust in complex scenes, and low-illumination conditionscause significant performance degradation that has been rarely explored.Guided by these observations, we introduce RGBT-VGNet, a simpleand reproducible reference baseline under the unified protocol, featuringAsymmetric Modality Adaptation, Language-Aware Visual Synergy, andTri-Prior Fusion for reliability-aware RGB-TIR integration. Resources,annotations, code, checkpoints, and evaluation scripts have been publiclyreleased.

Poster
ReliefSAM: A Geometry-Augmented Multi-Prior Adapter for Bas-Relief Segmentation

Yihang Chen, Xiang Lyu, Rui Xu, Jiao PAN, Fadjar Thufail, Brahmantara Brahmantara, Jiaqing Liu, Satoshi Tanaka, Liang Li

展开摘要

Bas-relief sculptures are vital historical records. However,many are inaccessible for modern 3D scanning due to damage or occlu-sion, leaving archival monocular photographs as the only available visualdata. Semantic segmentation, such as isolating human figures in theseimages, is essential for digital documentation and subsequent archaeo-logical analysis, yet remains highly challenging for foundation modelslike the Segment Anything Model (SAM). The intrinsic properties ofbas-reliefs, including extreme material homogeneity, subtle depth varia-tions, and indistinct soft edges, result in a severe lack of both chromaticand geometric contrast. Consequently, models pretrained on natural im-agery struggle to perceive the shallow 2.5D structural cues embeddedin relief surfaces, leading to imprecise boundary delineation. To addressthis limitation, we propose ReliefSAM, a prompt-free and parameter-efficient framework that explicitly incorporates image-aligned depth andedge priors derived from a single monocular RGB photograph. These ge-ometric cues are encoded via a lightweight Multi-Prior Feature Encoder(MFE) and injected into a frozen SAM backbone through adapter-basedinteraction blocks. For high-resolution archival imagery, we further adoptoverlapped sliding-window inference with Gaussian-weighted merging toensure spatial consistency. Extensive experiments demonstrate that whileadapting the frozen SAM image encoder with lightweight adapters es-tablishes a highly competitive pure-RGB baseline that outperforms stan-dard decoder-only fine-tuning, naively injecting a single geometric priorcan induce modality interference. Crucially, ReliefSAM’s joint integra-tion of depth and soft-edge priors achieves a geometric consensus inwhich the two priors counterbalance and neutralize prior-specific biases.This synergistic 2.5D guidance breaks the RGB-only performance ceil-ing, effectively bridging the gap between natural RGB appearance andheritage-specific structures without requiring large-scale retraining of thefoundation model.

Poster
Raymap-Guided Coupling for Drift-Robust Unposed Feed-Forward 3D Reconstruction

Xiangyu Sun, Liu.Liu Liu.Liu, Seungkwon Yang, Jingbing Han, Seungtae Nam, Zhizhong Su, Eunbyung Park

展开摘要

Pose-Free Feed-forward 3D Gaussian Splatting (3DGS) hasrecently emerged as a powerful paradigm for fast scene reconstruction.However, its performance degrades significantly in long image sequencesdue to cumulative camera pose estimation drift, which propagates er-rors into geometric modeling and severely limits rendering fidelity. Inthis work, we revisit the long-sequence bottleneck and identify posedrift as the primary factor restricting reconstruction quality. Further-more, while SfM-based pseudo ground-truth poses introduce sensor noise,purely rendering-based supervision often leads to optimization instabil-ity and local minima due to the entangled optimization of geometryand pose. To address the challenges, we propose a synergistic pose-free framework that explicitly couples geometry and appearance via aRaymap-Guided Coupling Module (RGC). Concretely, we anchor Gaus-sian centers to raymap-induced geometry and jointly optimize RGB re-construction, raymap consistency, and camera regularization under a uni-fied objective, yielding a bidirectional feedback loop: stronger geometryimproves rendering, and appearance supervision in turn refines geometryand pose. To further stabilize learning across wide temporal ranges, weintroduce a Dual-Frequency Viewpoint Scheduling strategy that com-bines easy-to-hard interval expansion with replay of short-interval pairs.Extensive experiments across in-domain and cross-domain datasets showconsistent gains in both rendering and pose estimation, with notably im-proved robustness on long sequences. Ablation studies validate our cen-tral insight: explicitly designed geometry-appearance synergy is the keyto scalable and drift-robust pose-free feed-forward 3D reconstruction.

Poster
Prosthesis-Aware 3D Human Pose Estimation: A Dataset and Benchmark for RSP Users

Yilin Wen, Kechuan Dong, Fumiya Suginaka, Ken Endo, Yusuke Sugano

展开摘要

Recovering 3D human body motion from video is impor-tant for applications such as rehabilitation assessment and sports per-formance evaluation. For prosthesis users, this requires capturing bothnatural body joints and the geometry of the prosthetic device, a challengethat existing methods are not designed to address. Model-based estima-tors rely on body models trained on non-amputee individuals and cannotrepresent prosthesis geometry, while model-free methods lack body kine-matic priors and are unreliable under occlusion. This challenge is partic-ularly prominent for users of running-specific prostheses (RSPs), wherethe RSP has a complex curved geometry and moves dynamically duringexercise. To fill this gap, we collect RSP3D, the first 3D dataset of RSPusers, covering essential daily-life and exercise actions from participantswith varied amputation conditions, using a multi-camera marker-basedmotion capture setup. We formally define the task of prosthesis-aware 3Dpose estimation, evaluate representative methods in a zero-shot setting,and confirm their individual limitations. We further propose a hybridbaseline combining model-based body joint estimation with model-freeRSP shape recovery, establishing a starting point for future research.Our project page is available at https://ut-vision.github.io/RSP3D/

Poster
NoPA: Non-Parametric Online 3D Scene Graph Generation

Qi Xun Yeo, Seungjun Lee, Yan Li, Gim Hee Lee

展开摘要

Classic 3D scene graph generation approaches fail to workin real-time due to the heavy computational cost of environment map-ping and the need to generate intermediate point-cloud representations.To alleviate this issue, a recent work eschews point clouds in favor ofa lightweight Gaussian distribution for each object. This approximationdrastically speeds up inference and enables real-time 3D scene graph gen-eration. However, the representation has two key weaknesses. 1) Each ob-ject is approximated by a single 3D Gaussian, which causes a severe lossof 3D geometric detail. 2) The discrepancy between this approximationand the true object geometry exacerbates the inaccurate merging of ob-ject candidates during online inference. To address these issues, we pro-pose NoPA, which represents each object as a separate non-parametricdistribution. This formulation retains 3D geometric information whilepreserving real-time inference of the parametric Gaussian formulation. Tobuild upon our novel object representation, we propose a tailored merg-ing strategy to recover coherent object instances. Specifically, we leveragemaximum mean discrepancy on kernel density estimates to enable robustmerging of object candidates during online exploration while minimizingadded computational complexity. The key is to maintain a fixed particleset per object. Furthermore, to rectify the relation loss caused by misclas-sified objects, NoPA propagates relationships between objects with highaffinity. Experiments show that NoPA substantially outperforms currentmethods without sacrificing real-time inference speed.

Poster
Point2Pose: Occlusion-Recovering 6D Pose Tracking and 3D Reconstruction for Multiple Unknown Objects Via 2D Point Trackers

Tzu-Yuan Lin, Ho Lee, Kevin Doherty, Yonghyeon Lee, Sangbae Kim

展开摘要

We present Point2Pose, a model-free method for causal 6Dpose tracking of multiple rigid objects from monocular RGB-D video.Initialized only from sparse image points on the objects, our approachtracks multiple unseen objects without requiring object CAD modelsor category priors. Point2Pose leverages a 2D point tracker to obtainlong-range correspondences, enabling instant recovery after complete oc-clusion. Simultaneously, the system incrementally reconstructs an on-line Truncated Signed Distance Function (TSDF) representation of thetracked targets. Alongside the method, we introduce a new multi-objecttracking dataset comprising both simulation and real-world sequences,with motion-capture ground truth for evaluation. Experiments show thatPoint2Pose trades some single-object pose accuracy for broader model-free tracking capabilities, including multi-object tracking and recoveryfrom complete occlusion. Project page: https://point2pose.github.io/.

Poster
PixelU: A U-Shaped Transformer for Efficient End-to-End Pixel Diffusion

Zipeng Guo, Lichen Ma, Yu He, Xiaolong Fu, Jingling Fu, Junshi Huang, Yan Li

展开摘要

End-to-end pixel-space diffusion models bypass the lossy com-pression of Latent Diffusion Models (LDMs) but struggle to jointly modellow-frequency semantics and high-frequency signals in high-dimensionalspace. Existing works heavily rely on complex pixel decoders to alleviatethis issue. In this paper, we challenge this trend by revealing that thesedecoders primarily compensate for the optimization difficulties inherentto velocity prediction (v-prediction). Under the clean data paradigm (x-prediction), they are redundant. Motivated by this insight, we advocatefor simplicity over complexity and introduce PixelU, a minimalist, single-stage U-shaped Diffusion Transformer tailored for pixel space. PixelUabandons auxiliary decoders in favor of zero-cost skip connections, whichprovide an “information highway” that directly routes uncorrupted high-frequency spatial details from shallow to deep layers. To further enablethe backbone to focus exclusively on modeling low-frequency semantics,we introduce a constant-channel spatial down-sampling mechanism asa natural low-pass filter, which compresses deep features into a com-pact, low-frequency semantic manifold. Extensive experiments demon-strate that this decoupling of frequencies could outperform the strongbaseline (JiT-G) with only about 1/3 of its computation cost. On Im-ageNet 256 × 256 and 512 × 512, PixelU achieves FID of 1.63 and 1.92respectively, surpassing recent pixel-space methods and establishing asimple yet powerful new paradigm for end-to-end diffusion models. Codewill be available at https://github.com/gzp6688/PixelU.

Poster
PhysEdit: Physically Consistent Image Editing via Causal Enforcement

Siqi Wan, Jingwen Chen, Yehao Li, Yingwei Pan, Ting Yao, Tao Mei

展开摘要

While instruction-guided image editing has seen significantstrides, most existing models fail to preserve physical laws and causal-ity. This limitation stems from the sparsity of physical supervision sig-nals, coupled with the static, non-causal architecture of prevailing frame-works, which together hinder a deep comprehension of the physical world.To bridge this gap, we present PhysEdit, a novel framework that en-forces physical consistency in image editing through causal generationand physics-aware reinforcement learning. Specifically, by harnessing theintrinsic physical priors of video generative models, we curate a high-quality dataset, dubbed PhysEdit-50K, wherein every sample encapsu-lates the complex physical causalities and dynamics involved in the tran-sition between input and edited images. Building upon this, a specializedCausal Image Editor is devised to factorize the image editing processalong the temporal evolution of visual transformations, thereby internal-izing the underlying physical laws and boosting causal coherence. Simul-taneously, an additional regularization term is incorporated to maintainvisual consistency in unedited regions beyond the simple flow matchingobjective, mitigating the impact of undesired viewpoint shifts or cam-era motions in the dataset synthesized by video models. Furthermore,we integrate physics-aware reinforcement learning with a tailored fine-grained reward to steer the editing process toward better adherence tophysical laws. Extensive experiments on PICABench and ImgEdit-Benchdemonstrate that our PhysEdit significantly outperforms state-of-the-artbaselines, yielding physically consistent and plausible edits. Code is pub-licly available at: https://github.com/HiDream-ai/PhysEdit/.

Poster
P-MTP: Efficient Document Parsing via Multi-Token Prediction with Progressive Depth Scaling

Le Xiang, Chenxi Zhai, Shu Wei, Jingjing Wu, Qunyi Xie, Xiao Tan, KunbinChen KunbinChen, Wei He

展开摘要

Vision-Language Models (VLMs) have revolutionized docu-ment parsing by enabling end-to-end mapping from images to structuredtext, imposing a significant latency bottleneck, particularly for token-dense documents. While Multi-Token Prediction (MTP) has emergedas a promising approach for accelerating inference, its potential is con-strained by optimization instability when scaling to deeper look-aheaddepth. In this paper, we propose P-MTP, a framework that leveragesProgressive Multi-Token Prediction with a lightweight MTP mod-ule to scale the look-ahead depth for high-throughput document parsing.Specifically, we introduce Progressive Curriculum Loss that adaptivelyre-weights different look-ahead depths using cumulative path reliabilityand retrospective target consistency. By effectively suppressing gradientnoise in long-range predictions, P-MTP facilitates an automated easy-to-hard optimization transition, enabling the model to master increasinglydistant look-ahead depths. Furthermore, we propose Confidence-GatedDynamic Drafting to maximize the effective look-ahead depth and ac-ceptance rate by adaptively calibrating speculative length during infer-ence, thereby minimizing computational waste and further pushing theboundaries of inference speedup. Experimental results across multiplebenchmarks and architectures demonstrate that P-MTP achieves up toa 5× speedup with negligible loss in accuracy, providing the first suc-cessful validation of extensive look-ahead MTP in the document parsingdomain.

Poster
RAU: Reference-based Anatomical Understanding with Vision-Language Models

Yiwei Li, Yikang Liu, Jiaqi Guo, Lin Zhao, Zheyuan Zhang, Xiao Chen, Boris Mailhe, Ankush Mukherjee, Terrence Chen, Shanhui Sun

展开摘要

Anatomical understanding, which is the ability to identify,localize, or segment anatomical structures, is critical in medical imageanalysis; however, its progress is constrained by the scarcity of expert-labeled data. A promising remedy is to leverage an annotated referenceimage to guide the interpretation of an unlabeled target. Although re-cent vision–language models (VLMs) exhibit non-trivial visual reasoning,their reference-based understanding and fine-grained localization remainlimited. We introduce RAU, a framework for reference-based anatomicalunderstanding with VLMs. We first show that a VLM learns to identifyanatomical regions through relative spatial reasoning between referenceand target images, trained on a moderately sized dataset. We validatethis capability through visual question answering (VQA) and boundingbox prediction. Next, we demonstrate that the VLM-derived spatial cuescan be seamlessly integrated with the fine-grained segmentation capabil-ity of SAM2, enabling localization and pixel-level segmentation of smallanatomical regions, such as vessel segments. Across two in-distributionand two out-of-distribution datasets, RAU consistently outperforms aSAM2 fine-tuning baseline using the same memory setup, yielding moreaccurate segmentations and more reliable localization. More importantly,its generalization ability to unseen modalities makes it scalable to unseendatasets, a property crucial for medical image applications. To the bestof our knowledge, RAU is the first to explore the capability of VLMs forreference-based identification, localization, and segmentation of anatom-ical structures in medical images. Its promising performance highlightsthe potential of VLM-driven approaches for anatomical understandingin automated clinical workflows.

Poster
ORION: Ordinal Neural Collapse as a Representation Prior for Visual Navigation

E-In Son, Jung-Taak Kim, Seung-Woo Seo

展开摘要

Learning robust navigation policies directly from visual ob-servations remains a fundamental challenge in vision-based robotic nav-igation. In end-to-end imitation learning approaches, the visual encoderand action decoder are jointly optimized using a single action loss, whichprovides only an indirect supervisory signal to the encoder. This indi-rect supervision frequently results in the encoder learning ambiguous,action-agnostic representations. The problem is further complicated bysubstantial variations in scene structure and appearance across diverseenvironments, as well as the prevalence of visual distractors inherent toreal-world navigation settings. Such action-agnostic features cause thenavigation policy to produce inconsistent actions at ambiguous deci-sion points, leading to navigation failure. To overcome these limitations,we propose ORION (Ordinal Neural Collapse for Visual Navigation), amethod that explicitly organizes the encoder’s representation space ac-cording to the ordinal structure of navigation actions. In the context ofgoal-directed navigation, ego-centric control categories from Far Left toFar Right exhibit a natural ordinal relationship in which neighboringclasses share similar visual contexts, while semantically opposing classesdiffer substantially in appearance. We encourage class representations tobe arranged sequentially along a single discriminative axis, while sup-pressing off-axis variance within each class. The pretrained encoder isthen integrated into a diffusion-based navigation framework, and the fullpipeline is fine-tuned end-to-end. Extensive experiments in both simula-tion and real-world settings show that ORION consistently outperformsend-to-end and neural collapse baselines in navigation success rate andgoal progress, with notable gains in visually challenging scenarios suchas complex multi-way intersections.

Poster
NearID: Identity Representation Learning via Near-identity Distractors

Aleksandar Cvejic, Rameen Abdal, Abdelrahman Eldesokey, Bernard Ghanem, Peter Wonka

展开摘要

When evaluating identity-focused tasks such as personalizedgeneration and image editing, existing vision encoders entangle objectidentity with background context, leading to unreliable representationsand metrics. We introduce a principled framework for matched-contextidentity confusion using Near-identity (NearID) distractors, where se-mantically similar but distinct instances are placed on the exact samebackground as a reference image, eliminating contextual shortcuts andisolating identity as the sole discriminative signal. Based on this prin-ciple, we present the NearID dataset (19K identities, 316K matched-context distractors) together with a strict margin-based evaluation pro-tocol. Under this setting, pre-trained encoders perform poorly, achievingSample Success Rates (SSR), a strict margin-based identity discrimina-tion metric, as low as 30.7% and often ranking distractors above truecross-view matches. We address this by learning identity-aware repre-sentations on a frozen backbone using a two-tier contrastive objectiveenforcing the hierarchy: same identity > NearID distractor > randomnegative. This improves SSR to 99.2%, enhances part-level discrimina-tion by 28.0%, and yields stronger alignment with human judgments onDreamBench++, a human-aligned benchmark for personalization.

Poster
MoScale: Autoregressive Next-Scale Prediction for Human Motion Generation and Editing

Inwoo Hwang, Hojun Jang, Bing Zhou, Jian Wang, Young Min Kim, Chuan Guo

展开摘要

We present ScaleMoGen, a scale-wise autoregressive frame-work for text-driven human motion generation. Unlike conventional au-toregressive approaches that rely on standard next-token prediction, Scale-MoGen frames motion generation as a coarse-to-fine process. We quantize3D motions into compositional discrete tokens across multiple skeletal-temporal scales of increasing granularity, learning to generate motionby autoregressively predicting next-scale token maps. To maintain struc-tural integrity, our motion tokenizers and quantizers are explicitly de-signed so that discrete tokens at every scale strictly preserve the skeletalhierarchy. Additionally, we employ bitwise quantization and prediction,which efficiently scale up the tokenizer vocabulary to preserve motiondetails and stabilize optimization. Extensive experiments demonstratethat ScaleMoGen achieves state-of-the-art performance, establishing anFID of 0.030 (vs. 0.045 for MoMask) on HumanML3D and a CLIP Scoreof 0.693 (vs. 0.685 for MoMask++) on the SnapMoGen dataset. Further-more, we demonstrate that our skeletal-temporal multi-scale representa-tion naturally facilitates training-free, text-guided motion editing.

Poster
Moonstone: A Multimodal Foundation Model and Benchmark for Lunar Remote Sensing

Ayush Prasad, Swarnalee Mazumder

展开摘要

Decades of orbital missions have produced multi-modal re-mote sensing data for the Moon, spanning optical imagery, spectroscopy,thermal emission, radar, gravity, and elemental composition. Yet thesedatasets remain fragmented across archives, and no benchmark exists forevaluating machine learning on lunar data. We introduce Moonstone, thefirst multi-modal foundation model benchmark for lunar remote sensing.Our contributions are: (1) a 28-channel, 128 pixels-per-degree (∼237 m)global lunar pretraining dataset from seven instrument families acrossfive missions, (2) MG-MAE, a modality-grouped masked autoencoderwith per-group convolutional tokenizers, a shared Vision Transformerencoder, attention masking for missing modalities, coverage-adaptivemasking for heterogeneous spatial coverage, and spectral continuity reg-ularization for physically plausible reconstructions, and (3) a benchmarkof six downstream tasks covering classification, regression, and segmen-tation. MG-MAE pretrained features outperform scratch baselines onall tasks and surpass both ImageNet-pretrained and vanilla MAE base-lines by large margins. We release the pretraining dataset, code, and thebenchmark suite.3

Poster
MomentSeg: Moment-Centric Sampling for Enhanced Referring Video Object Segmentation

Ming Dai, Sen Yang, Boqiang Duan, Wankou Yang, Jingdong Wang

展开摘要

Referring Video Object Segmentation (RefVOS) seeks to seg-ment target objects in videos guided by natural language descriptions,demanding both temporal reasoning and fine-grained visual comprehen-sion. Existing sampling strategies for LLM-based approaches typicallyrely on either handcrafted heuristics or external keyframe models. Theformer often overlooks essential temporal cues, while the latter increasessystem complexity. To address this, we propose a unified frameworkthat jointly optimizes Temporal Sentence Grounding (TSG) and Re-fVOS, naturally incorporating key moment grounding capability. Duringtraining, we introduce a novel TSG paradigm that employs a dedicated[FIND] token for key moment identification through temporal token sim-ilarity matching, thereby avoiding the need for external timestamp en-codings. For inference, we design a Moment-Centric Sampling (MCS)strategy that densely samples informative moments while sparsely sam-pling non-essential frames, preserving both motion details and globalcontext. To further enhance tracking stability, we develop BidirectionalAnchor-updated Propagation (BAP), which leverages the most relevantmoment as start point for high-quality mask initialization and dynam-ically updates at sampled points to mitigate accumulated errors. Codewill be released at https://github.com/Dmmm1997/MomentSeg.

Poster
ModTrack: Sensor-Agnostic Multi-View Tracking via Identity-Informed PHD Filtering with Covariance Propagation

Aditya Iyer, Jack Roberts, Nora Ayanian

展开摘要

Multi-View Multi-Object Tracking (MV-MOT) aims to localize and maintain consistent identities of objects observed by multiple sensors. This task is challenging, as viewpoint changes and occlusion disrupt identity consistency across views and time. Recent end-to-end approaches address this by jointly learning 2D Bird’s Eye View (BEV) representations and identity associations, achieving high tracking accuracy. However, these methods offer no principled uncertainty accounting and remain tightly coupled to their training configuration, limiting generalization across sensor layouts, modalities, or datasets without retraining. We propose ModTrack, a modular MV-MOT system that matches end-to-end performance while providing cross-modal, sensoragnostic generalization and traceable uncertainty. ModTrack confines learning methods to just the Detection and Feature Extraction stage of the MV-MOT pipeline, performing all fusion, association, and tracking with closed-form analytical methods. Our design reduces each sensor’s output to calibrated position-covariance pairs (z,R); cross-view clustering and precision-weighted fusion then yield unified estimates (zˆ,Rˆ) for identity assignment and temporal tracking. A feedback-coupled, identityinformed Gaussian Mixture Probability Hypothesis Density (GM-PHD) filter with HMM motion modes uses these fused estimates to maintain identities under missed detections and heavy occlusion. ModTrack achieves 95.5 IDF1 and 91.4 MOTA on WildTrack, surpassing all prior modular methods by over 21 points and rivaling the state-of-the-art end-to-end methods while providing deployment flexibility they cannot. Specifically, the same tracker core transfers unchanged to MultiviewX and RadarScenes, with only perception-module replacement required to extend to new domains and sensor modalities.

Poster
MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptation

Sajjad Ghiasvand, Haniyeh Oskouie, Mahnoosh Alizadeh, Ramtin Pedarsani

展开摘要

Prompt learning has become a dominant paradigm for adapt-ing vision-language models (VLMs) such as CLIP to downstream taskswithout modifying pretrained weights. While extending prompts to bothvision and text encoders across multiple transformer layers significantlyboosts performance, it dramatically increases the number of trainableparameters, with state-of-the-art methods requiring millions of parame-ters and abandoning the parameter efficiency that makes prompt tuningattractive. In this work, we propose MMLoP (Multi-Modal Low-RankPrompting), a framework that achieves deep multi-modal promptingwith only 11.5K trainable parameters, comparable to early text-onlymethods like CoOp. MMLoP parameterizes vision and text prompts ateach transformer layer through a low-rank factorization that constrainsprompts to a compact subspace, providing parameter efficiency whilemotivating the need for our complementary regularization components.To further close the accuracy gap with state-of-the-art methods, we intro-duce three complementary components: a self-regulating consistency lossthat anchors prompted representations to frozen zero-shot CLIP featuresat both the feature and logit levels, a uniform drift correction that re-moves the global embedding shift induced by prompt tuning to preserveclass-discriminative structure, and a shared up-projection that couplesvision and text prompts through a common low-rank factor to enforcecross-modal alignment. Extensive experiments across three benchmarksand 11 diverse datasets demonstrate that MMLoP achieves a highly favor-able accuracy-efficiency tradeoff, outperforming the majority of existingmethods including those with orders of magnitude more parameters, whileachieving a harmonic mean of 79.70% on base-to-novel generalization.Code is available at https://github.com/sajjad-ucsb/MMLoP.

Poster
MirrorPPR: Exemplar-Based Portrait Photo Retouching

Zhihong Liu, Zheng Li, Jiachun Jin, Siqi Kou, Yitao Jian, Fengpei Yu, Zhijie Deng

展开摘要

While text-guided image editing has made remarkable progress,it remains limited in structural portrait retouching. Textual descriptionsstruggle to convey fine-grained changes to facial features and body pro-portions. To address this gap, we introduce Exemplar-Based PortraitPhoto Retouching, where the model is given an exemplar pair and taskedwith inferring and applying the same retouching operations to a newquery image. Existing exemplar-based editing methods primarily focuson tasks with pronounced visual transformations. In contrast, structuralportrait retouching involves extremely delicate and localized modifica-tions, making accurate extraction and transfer of these edits challeng-ing. To tackle this, we propose MirrorPPR, a novel framework specifi-cally designed to capture and transfer subtle structural retouching oper-ations. Our method uses a Retouching Operation Extractor to capturethe subtle differences from the exemplar pair. The extracted represen-tations are then injected into a pre-trained Diffusion Transformer (DiT)through a connector and Low-Rank Adaptation (LoRA) modules. Fur-thermore, constructing perfectly aligned cross-identity training pairs isseverely hindered by operation misalignment. To overcome this, we pro-pose an advanced data self-augmentation paradigm that ensures strictlyaligned retouching operations. To alleviate data scarcity and supportthis novel task, we introduce MirrorPPR47M, a large-scale dataset withover 47 million retouched pairs. By structuring the dataset into simu-lated and professional subsets, we enable progressive curriculum learningto smoothly optimize the network. Extensive experiments demonstratethat MirrorPPR significantly outperforms existing baselines in both re-touching quality and identity preservation. The project page is availableat https://sjtu-deng-lab.github.io/MirrorPPR.

Poster