paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

ECCV 2026

Moonstone: A Multimodal Foundation Model and Benchmark for Lunar Remote Sensing

Ayush Prasad, Swarnalee Mazumder

Decades of orbital missions have produced multi-modal re-mote sensing data for the Moon, spanning optical imagery, spectroscopy,thermal emission, radar, gravity, and elemental composition. Yet thesedatasets remain fragmented across archives, and no benchmark exists forevaluating machine learning on lunar data. We introduce Moonstone, thefirst multi-modal foundation model benchmark for lunar remote sensing.Our contributions are: (1) a 28-channel, 128 pixels-per-degree (∼237 m)global lunar pretraining dataset from seven instrument families acrossfive missions, (2) MG-MAE, a modality-grouped masked autoencoderwith per-group convolutional tokenizers, a shared Vision Transformerencoder, attention masking for missing modalities, coverage-adaptivemasking for heterogeneous spatial coverage, and spectral continuity reg-ularization for physically plausible reconstructions, and (3) a benchmarkof six downstream tasks covering classification, regression, and segmen-tation. MG-MAE pretrained features outperform scratch baselines onall tasks and surpass both ImageNet-pretrained and vanilla MAE base-lines by large margins. We release the pretraining dataset, code, and thebenchmark suite.3

ECCV 2026

MomentSeg: Moment-Centric Sampling for Enhanced Referring Video Object Segmentation

Ming Dai, Sen Yang, Boqiang Duan, Wankou Yang, Jingdong Wang

Referring Video Object Segmentation (RefVOS) seeks to seg-ment target objects in videos guided by natural language descriptions,demanding both temporal reasoning and fine-grained visual comprehen-sion. Existing sampling strategies for LLM-based approaches typicallyrely on either handcrafted heuristics or external keyframe models. Theformer often overlooks essential temporal cues, while the latter increasessystem complexity. To address this, we propose a unified frameworkthat jointly optimizes Temporal Sentence Grounding (TSG) and Re-fVOS, naturally incorporating key moment grounding capability. Duringtraining, we introduce a novel TSG paradigm that employs a dedicated[FIND] token for key moment identification through temporal token sim-ilarity matching, thereby avoiding the need for external timestamp en-codings. For inference, we design a Moment-Centric Sampling (MCS)strategy that densely samples informative moments while sparsely sam-pling non-essential frames, preserving both motion details and globalcontext. To further enhance tracking stability, we develop BidirectionalAnchor-updated Propagation (BAP), which leverages the most relevantmoment as start point for high-quality mask initialization and dynam-ically updates at sampled points to mitigate accumulated errors. Codewill be released at https://github.com/Dmmm1997/MomentSeg.

ECCV 2026

ModTrack: Sensor-Agnostic Multi-View Tracking via Identity-Informed PHD Filtering with Covariance Propagation

Aditya Iyer, Jack Roberts, Nora Ayanian

Multi-View Multi-Object Tracking (MV-MOT) aims to localize and maintain consistent identities of objects observed by multiple sensors. This task is challenging, as viewpoint changes and occlusion disrupt identity consistency across views and time. Recent end-to-end approaches address this by jointly learning 2D Bird’s Eye View (BEV) representations and identity associations, achieving high tracking accuracy. However, these methods offer no principled uncertainty accounting and remain tightly coupled to their training configuration, limiting generalization across sensor layouts, modalities, or datasets without retraining. We propose ModTrack, a modular MV-MOT system that matches end-to-end performance while providing cross-modal, sensoragnostic generalization and traceable uncertainty. ModTrack confines learning methods to just the Detection and Feature Extraction stage of the MV-MOT pipeline, performing all fusion, association, and tracking with closed-form analytical methods. Our design reduces each sensor’s output to calibrated position-covariance pairs (z,R); cross-view clustering and precision-weighted fusion then yield unified estimates (zˆ,Rˆ) for identity assignment and temporal tracking. A feedback-coupled, identityinformed Gaussian Mixture Probability Hypothesis Density (GM-PHD) filter with HMM motion modes uses these fused estimates to maintain identities under missed detections and heavy occlusion. ModTrack achieves 95.5 IDF1 and 91.4 MOTA on WildTrack, surpassing all prior modular methods by over 21 points and rivaling the state-of-the-art end-to-end methods while providing deployment flexibility they cannot. Specifically, the same tracker core transfers unchanged to MultiviewX and RadarScenes, with only perception-module replacement required to extend to new domains and sensor modalities.

ECCV 2026

MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptation

Sajjad Ghiasvand, Haniyeh Oskouie, Mahnoosh Alizadeh, Ramtin Pedarsani

Prompt learning has become a dominant paradigm for adapt-ing vision-language models (VLMs) such as CLIP to downstream taskswithout modifying pretrained weights. While extending prompts to bothvision and text encoders across multiple transformer layers significantlyboosts performance, it dramatically increases the number of trainableparameters, with state-of-the-art methods requiring millions of parame-ters and abandoning the parameter efficiency that makes prompt tuningattractive. In this work, we propose MMLoP (Multi-Modal Low-RankPrompting), a framework that achieves deep multi-modal promptingwith only 11.5K trainable parameters, comparable to early text-onlymethods like CoOp. MMLoP parameterizes vision and text prompts ateach transformer layer through a low-rank factorization that constrainsprompts to a compact subspace, providing parameter efficiency whilemotivating the need for our complementary regularization components.To further close the accuracy gap with state-of-the-art methods, we intro-duce three complementary components: a self-regulating consistency lossthat anchors prompted representations to frozen zero-shot CLIP featuresat both the feature and logit levels, a uniform drift correction that re-moves the global embedding shift induced by prompt tuning to preserveclass-discriminative structure, and a shared up-projection that couplesvision and text prompts through a common low-rank factor to enforcecross-modal alignment. Extensive experiments across three benchmarksand 11 diverse datasets demonstrate that MMLoP achieves a highly favor-able accuracy-efficiency tradeoff, outperforming the majority of existingmethods including those with orders of magnitude more parameters, whileachieving a harmonic mean of 79.70% on base-to-novel generalization.Code is available at https://github.com/sajjad-ucsb/MMLoP.

ECCV 2026

MirrorPPR: Exemplar-Based Portrait Photo Retouching

Zhihong Liu, Zheng Li, Jiachun Jin, Siqi Kou, Yitao Jian, Fengpei Yu, Zhijie Deng

While text-guided image editing has made remarkable progress,it remains limited in structural portrait retouching. Textual descriptionsstruggle to convey fine-grained changes to facial features and body pro-portions. To address this gap, we introduce Exemplar-Based PortraitPhoto Retouching, where the model is given an exemplar pair and taskedwith inferring and applying the same retouching operations to a newquery image. Existing exemplar-based editing methods primarily focuson tasks with pronounced visual transformations. In contrast, structuralportrait retouching involves extremely delicate and localized modifica-tions, making accurate extraction and transfer of these edits challeng-ing. To tackle this, we propose MirrorPPR, a novel framework specifi-cally designed to capture and transfer subtle structural retouching oper-ations. Our method uses a Retouching Operation Extractor to capturethe subtle differences from the exemplar pair. The extracted represen-tations are then injected into a pre-trained Diffusion Transformer (DiT)through a connector and Low-Rank Adaptation (LoRA) modules. Fur-thermore, constructing perfectly aligned cross-identity training pairs isseverely hindered by operation misalignment. To overcome this, we pro-pose an advanced data self-augmentation paradigm that ensures strictlyaligned retouching operations. To alleviate data scarcity and supportthis novel task, we introduce MirrorPPR47M, a large-scale dataset withover 47 million retouched pairs. By structuring the dataset into simu-lated and professional subsets, we enable progressive curriculum learningto smoothly optimize the network. Extensive experiments demonstratethat MirrorPPR significantly outperforms existing baselines in both re-touching quality and identity preservation. The project page is availableat https://sjtu-deng-lab.github.io/MirrorPPR.

ECCV 2026

MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decoding

Hejun Dong, Junbo Niu, Bin Wang, Weijun Zeng, Wentao Zhang, Conghui He

Optical character recognition (OCR) has evolved from line-level transcription to structured document parsing, requiring models torecover long-form sequences containing layout, tables, and formulas. De-spite recent advances in vision-language models, most existing systemsrely on autoregressive decoding, which introduces sequential latency andamplifies error propagation in long documents. In this work, we revisitdocument OCR from an inverse rendering perspective, arguing that left-to-right causal generation is an artifact of serialization rather than anintrinsic property of the task. Motivated by this insight, we proposeMinerU-Diffusion, a unified diffusion-based framework that replaces au-toregressive sequential decoding with parallel diffusion denoising undervisual conditioning. MinerU-Diffusion employs a block-wise diffusion de-coder and an uncertainty-driven curriculum learning strategy to enablestable training and efficient long-sequence inference. Extensive experi-ments demonstrate that MinerU-Diffusion consistently improves robust-ness while achieving up to 3.2× faster decoding compared to autoregres-sive baselines. Evaluations on the proposed Semantic Shuffle benchmarkfurther confirm its reduced dependence on linguistic priors and strongervisual OCR capability.

ECCV 2026

Linear Fusion MultiDiffusion for Fast Training-Free Spherical Panorama Generation

Akio Hayakawa, YUSUKE Mukuta, Tatsuya Harada

We propose LF-MultiDiffusion, a training-free panorama gen-eration method that extends MultiDiffusion to support linear projectionsbetween target and reference image spaces. Our key idea is to reformulatelatent aggregation as a regularized least-squares problem and solve itefficiently with a Krylov-based iterative solver inside the denoising loop.This formulation enables denser and more natural mappings than priortraining-free methods, yielding more stable generation with far fewerperspective views. As a result, LF-MultiDiffusion reduces the number ofimage generator evaluations during denoising and significantly improvesinference efficiency. Experiments show that LF-MultiDiffusion achievesbetter visual quality, text alignment, and panoramic consistency thanthe strongest training-free baseline, while providing a 15.36× speedup.Our project page is available at: https://ahykw.github.io/lfmd.

ECCV 2026

LINA: Learning INterventions Adaptively for Physical Alignment and Counterfactual Generation in Diffusion Models

Shu Yu, Chaochao Lu

Diffusion models (DMs) have achieved remarkable successin image and video generation. However, there are still challenges in(1) physical alignment and (2) counterfactual generation. To diagnosethe root causes, we introduce the Causal Scene Graph (CSG) to modelDM’s generative process, and build the Physical Alignment Probe (PAP)dataset to quantify the failure modes. Our analysis yields three key in-sights: (1) DMs struggle with physical elements not explicitly determinedin the prompt; (2) the prompt embedding contains disentangled repre-sentations for texture and physics; (3) visual causal structure is dispro-portionately established during the initial, computationally constraineddenoising steps. Based on these findings, we introduce Lina (LearningINterventions Adaptively), a novel framework that learns to predictprompt-specific interventions. Lina employs (1) targeted guidance in theprompt and visual latent spaces, and (2) a reallocated, causality-awaredenoising schedule. Without modifying pre-trained DM weights, Linaenforces both physical alignment and counterfactual generation in imageand video DMs, achieving state-of-the-art performance on challengingcausal generation tasks and the Winoground dataset. Our project pageis at https://opencausalab.github.io/LINA

ECCV 2026

LEO-Fuse: A Modality- and Task-Agnostic Universal Framework for Multimodal Human Sensing

Emrecan Aslan

Robust human-body understanding benefits from fusing het-erogeneous sensors—WiFi CSI, depth, LiDAR, mmWave, and RGB—yeteach modality carries distinct noise characteristics and may be intermit-tently unavailable. Existing fusion methods often assume fixed sensorsets and can collapse onto dominant modalities, reducing robustness un-der missing-modality conditions. We propose LEO-Fuse, a task-general,modality-agnostic Local Expert Orchestration (LEO) framework for uni-versal multimodal human sensing. LEO-Fuse freezes unimodal encoders,aligns modality features in a shared token space, fuses available modal-ity tokens with a mask-aware transformer, and applies task-specific lo-cal routing for downstream tasks including 3D human pose estimation(HPE) and human activity recognition (HAR). For each target task, asingle trained model handles all 31 non-empty subsets of five modalitieswithout retraining. On MM-Fi, LEO-Fuse obtains 75.5 mm mean per-joint position error (MPJPE), averaged over all 31 subsets. For HAR,HPE-trained encoders yield 93.4% average top-1 accuracy, improving to95.7% with HAR-trained encoders. On XRF55, LEO-Fuse reaches up to95.5% HAR accuracy. These results demonstrate scalable and robust uni-versal multimodal human sensing under realistic missing-modality set-tings.

ECCV 2026

LaxMotion: Rethinking Supervision Granularity for 3D Human Motion Generation

Sheng Liu, Yuanzhi Liang, Sidan Du

Recent 3D human motion generation models demonstrateremarkable reconstruction accuracy yet struggle to generalize beyondtraining distributions. This limitation arises partly from the use of pre-cise 3D supervision, which encourages models to fit fixed coordinate pat-terns instead of learning the essential 3D structure and motion–semanticcues required for robust generalization. To overcome this limitation, wepropose LaxMotion, a framework that synthesizes realistic 3D motionswithout direct 3D pose supervision. Instead of regressing toward exactcoordinates, LaxMotion learns 3D motion as a consistent explanationof global trajectories and monocular 2D kinematic cues. We introducea structured motion factorization together with a reformulated trainingparadigm under relaxed observability. This design is further supported byrelaxed regularization objectives that enforce view-consistent alignment,orientation coherence, and structural stability. Under this relaxed super-vision paradigm, LaxMotion generates diverse, temporally coherent, andsemantically aligned 3D motions, achieving performance comparable toor surpassing fully 3D-supervised methods. These results indicate thatshifting supervision from exact coordinate matching to structural consis-tency promotes stronger reasoning and improved generalization, offeringa scalable and data-efficient paradigm for 3D motion generation.

ECCV 2026

Is Monitoring Enough? Strategic Agent Selection For Stealthy Attack in Multi-Agent Discussions

Qiuchi Xiang, Haoxuan Qu, Hossein Rahmani, Jun Liu

Multi-agent discussions have been widely adopted, motivat-ing growing efforts to develop attacks that expose their vulnerabilities.In this work, we study a practical yet largely unexplored attack sce-nario, the discussion-monitored scenario, where anomaly detectors con-tinuously monitor inter-agent communications and block detected ad-versarial messages. Although existing attacks are effective without dis-cussion monitoring, we show that they exhibit detectable patterns andlargely fail under such monitoring constraints. But does this imply thatmonitoring alone is sufficient to secure multi-agent discussions? To an-swer this question, we develop a novel attack method explicitly tailored tothe discussion-monitored scenario. Extensive experiments demonstratethat effective attacks remain possible even under continuous monitoring,indicating that monitoring alone does not eliminate adversarial risks.

ECCV 2026

HLRAD: High-dimensional Latent Representation for Unified Anomaly Detection

Tianrui Zhang, Ziheng Zang, Ningmu Zou

Unified unsupervised anomaly detection aims to train a sin-gle model to detect and localize diverse anomalies across multi-class sam-ples, representing one of the most challenging tasks in anomaly detec-tion. Existing methods typically rely on highly compressing inputs intoa low-dimensional latent space to subsequently reconstruct anomalousfeatures. However, low-dimensional latent spaces critically limit infor-mation capacity, leading to unavoidable feature loss during compressionand significantly degrading the quality of reconstruction and represen-tational power of the model. Therefore, we indicate that dimensionalitycompression in latent space is not a necessary requirement for anomalydetection, and the editability introduced by compression is not inher-ently essential for models reconstructing anomalies. Thus, we propose anovel anomaly detection framework without latent space compression,called HLRAD (High-dimensional Latent Representation for UnifiedAnomaly Detection). Unlike prior methods, HLRAD innovatively en-ables dimension expansion rather than compression in the latent space,and the method effectively avoids feature loss from compression to ensurehigh-quality reconstruction while constructing a semantically enrichedhigh-dimensional latent representation space. Furthermore, by embed-ding the expanded high-dimensional semantic features into the latentrepresentation, HLRAD enables the model to more fully capture the se-mantic feature distribution of normal samples, significantly enhancingcomplex anomaly detection performance. We conducted extensive ex-periments on major anomaly detection benchmark datasets, includingMVTec-AD, VisA, and Real-IAD. In unified settings, HLRAD outper-forms state-of-the-art methods.

ECCV 2026

Histopathology Multi-modal Embedding for Pathology Composed Retrieval

Qifeng Zhou, Wenliang Zhong, Thao Dang, Hehuan Ma, Saiyang Na, Yuzhi Guo, Junzhou Huang

To overcome the black-box nature of predictive AI and thehallucination risks of generative models, retrieval-based models o!er aninterpretable, evidence-based paradigm for pathology clinical workflow.However, real-world clinical queries are inherently interleaved (e.g., pathol-ogy images and text). Current dual-encoders su!er from an Archi-tectural Mismatch, lacking the mechanism to fuse such composedqueries. To address this, we formalize the task of Pathology ComposedRetrieval (PCR). While Multimodal Large Language Models (MLLMs)o!er deep-fusion capabilities, directly applying them exposes a TaskMismatch and a Domain Mismatch. To resolve these challenges,we propose HOMIE, a model-agnostic adaptation framework that trans-forms any generative MLLM into a specialized pathology retrieval expert.Evaluated on our newly introduced PCR Benchmark, a lightweight 2B-parameter HOMIE variant substantially outperforms existing paradigms,surpassing specialized 7B pathology MLLMs and dual-encoders by largemargins on composed retrieval, while maintaining strong performanceon traditional simple retrieval. The project page is available at https://qfchou.github.io/HOMIE_page/.

ECCV 2026

HHA: Hierarchical Hyperbolic Constraints for Imperceptible Point Cloud Attacks

Keke Tang, Yu Liao, Weilong Peng, Xiaofei Wang, Daizong Liu, Zhongyun Hua, Peican Zhu, Zhihong Tian

Adversarial attacks on point clouds require effective constraints to ensure imperceptibility. However, existing methods often overlook the intrinsic hierarchical organization of 3D shapes, thereby limiting their ability to preserve structural coherence. In this paper, we propose HHA, a novel framework that generates hierarchy-aware adversarial perturbations by leveraging hyperbolic geometry. HHA first decomposes the input point cloud into semantic and geometric substructures to capture its multi-scale organization. Then, each substructure is embedded into hyperbolic space, where localized constraints limit distortion and maintain hierarchical consistency. This hyperbolic regularization keeps perturbations aligned with the underlying structure and thereby enhances imperceptibility. Extensive experiments validate that HHA produces adversarial point clouds with improved structural coherence and imperceptibility, outperforming state-of-the-art methods.

ECCV 2026

Articulated Object Reconstruction from Rest-State Observation

Daeun Lee, Jaeah Lee, Woosung Kim, Haebeom Jung, Jaesik Park

Building interactive digital twins requires recovering both 3Dgeometry and the kinematic structures that govern how objects articu-late. Yet existing methods for articulated object reconstruction requireexplicitly observable motion from multiple articulation states. We intro-duce a rest-state formulation that reconstructs articulated objects froma single closed configuration, an inherently ill-posed setting where geom-etry, semantics, and motion priors compensate for the absence of motioncues. Our framework adopts an explicit mesh as an intermediate repre-sentation for cross-model verification and fusion, reconciling noisy out-puts from vision-language and segmentation models into spatially con-sistent part structures. To estimate joint parameters without observedmotion, we use a video diffusion model to synthesize articulation hy-potheses and validate them through geometric consistency. Our approachachieves accurate part decomposition and physically plausible articu-lation, performing competitively with motion-observing reconstruction-based, generation-based, and modular pretrained-model baselines.

ECCV 2026

Generalized Biomedicine Discovery

Luyao Tang, Yingkai Yang, Hanqi Chen, Jiewei Zheng, Chaoqi Chen, Cheng Chen

In real-world clinical practice, medical images face open-world shifts: (i) long-tailed rare diseases, (ii) subtle lesions dominated bynormal anatomy, and (iii) hierarchical taxonomies. Yet most open-worldparadigms assume flat, balanced label spaces, leaving these biomedi-cal demands unresolved. We introduce Generalized Biomedicine Dis-covery (GBD) and a unified benchmark spanning long-tail, anomaly,and taxonomy-aware discovery. Our key insight is that dominant knownpatterns form a visual manifold that masks subtle novelty. Inspired byexpert diagnosis, we propose SCAN (Surprise-evoked ComplementaryAccommodatioN), which follows a cognition-inspired perceptual progres-sion: it applies predictive suppression to filter expected norms, trig-gers surprise-evoked salience to highlight unexpected deviations, andperforms complementary accommodation to integrate these shifts intoglobal representations. Extensive experiments show that SCAN improvesnovel concept discovery while generally preserving established clinicalknowledge, and it plugs into existing architectures to better navigatethe known–unknown trade-off in medical imaging. Code is available atgithub.com/lytang63/generalized-biomedicine-discovery.

ECCV 2026

Fourier Splatting: Generalized Fourier encoded primitives for scalable radiance fields

Mihnea Jurca, Bert hauwermeiren, Adrian Munteanu

Novel view synthesis has recently been revolutionized by 3DGaussian Splatting (3DGS), which enables real-time rendering throughexplicit primitive rasterization. However, existing methods tie visual fi-delity strictly to the number of primitives: quality downscaling is achievedonly through pruning primitives. We propose the first inherently scal-able primitive for radiance field rendering. Fourier Splatting employsscalable primitives with arbitrary closed shapes obtained by parameter-izing planar surfels with Fourier encoded descriptors. This formulationallows a single trained model to be rendered at varying levels of de-tail simply by truncating Fourier coefficients at runtime. To facilitatestable optimization, we employ a straight-through estimator for gradi-ent extension beyond the primitive boundary, and introduce HYDRA,a densification strategy that decomposes complex primitives into sim-pler constituents within the MCMC framework. Our method achievesstate-of-the-art rendering quality among planar-primitive frameworksand comparable perceptual metrics compared to leading volumetric rep-resentations on standard benchmarks, providing a versatile solution forbandwidth-constrained high-fidelity rendering.

ECCV 2026

Natural Language Camera Movement Understanding

Yuwen Tan, Joey Huang, Jin Huang, Haoxiang Li, Boqing Gong

Understanding camera movement in natural language is crit-ical for training and evaluating video generation models, among other ap-plications. However, we demonstrate that existing vision-language mod-els (VLMs) fail this task in surprising ways, frequently confusing trans-lation with rotation, left with right, and object movement with cameramovement. To address these limitations, we establish natural languagecamera movement understanding as a standalone research task. We in-troduce a two-level cinematographic taxonomy and an extensive, atomicbenchmark featuring both real and synthetic videos. Furthermore, we cu-rate a large-scale, multi-source training set enhanced by targeted cameramovement augmentation. Our fine-tuned VLM-8B outperforms Gemini3.1 Pro by 10% and 11% on our benchmark’s real and synthetic videos,respectively. Despite these gains, a significant gap remains relative to hu-man performance, underscoring the need to promote and facilitate futureresearch on natural language camera movement understanding.

ECCV 2026

AVTok: 1D Unified Tokenization for Holistic Audio-Video Generation

Trung Kien Pham, I Chieh Chen, Qifeng Chen, Long Chen

Audio-video generation has recently gained unprecedentedresearch attention, aiming to synthesize high-quality sounding video con-tent with fine-grained synchronization and semantic alignment betweenthe auditory and visual components. The preceding methods predomi-nantly adopt a dual-branch design with separate tokenization and gen-eration modules per modality, neglecting the representation gap whilenecessitating intensive computational resources for proper training. In-spired by recent advancements in one-dimensional visual tokenization,we present AVTok, a novel unified tokenizer designated for holisticaudio-video generation. AVTok features a dual-stream transformer-basedarchitecture with shared encoder-decoder and modal-specific learnablequeries to efficiently and effectively encode an audio-video pair into acompact one-dimensional latent representation with a unified codebook.To cope with the heterogeneous information imbalance that hinders AV-Tok from exploiting aligned audio-visual information, we devise a hier-archical training strategy to progressively realize reconstruction capabil-ities for each modality. Extensive experiments demonstrate that AVTokexcels both in audio-video reconstruction and when integrated into down-stream pipelines for audio-to-video, video-to-audio, and class-conditionaljoint audio-video generation. AVTok paves the way for the challenge ofjoint audio-video tokenization and provides a potential direction to buildunified large multimodal models for audio-video generation.

ECCV 2026

FlexiAvatar: Unified 3D Gaussian Human Avatars Under Arbitrary Body Visibility

Yihalem Yimolal Tiruneh, Muhammad Salman Ali, Uyoung Jeong, Muneeb Khan, MD Sayem, Allanur Bayramgeldiyev, Binod Bhattarai, Seungryul Baek

Reconstructing animatable 3D human avatars from monocularvideo is a fundamental problem in computer vision with broad applicationsin AR/VR and digital content creation. Existing approaches typically coupleparametric body models with neural rendering or 3D Gaussian splattingand optimize all body regions jointly from short videos, which often de-grades fidelity in the visible areas. To overcome this limitation, we introduceFlexiAvatar, a unified framework that explicitly optimizes only the visiblebody regions, effectively eliminating artifacts arising from unobserved limbs.Our method integrates occlusion-robust SMPL-X tracking with part-specificresidual refinement to capture high-frequency geometric and appearancedetails. To complete entirely unseen regions (e.g., back views), we leverage adiffusion-based approach to generate texture consistent with the observedappearance. Experiments on full-body (NeuMan, ZJU-MoCap, WildAvatar),upper/half-body (talk-show clips), and head-only (INSTA) inputs show thatFlexiAvatar delivers consistently higher reconstruction quality, outperformingstate-of-the-art methods by an average PSNR improvement of approximately3% across datasets. Finally, by restricting optimization to observed regions,our method reduces the effective number of Gaussians that must be opti-mized and rendered, leading to reduced runtime and memory overhead inpartial-visibility scenarios. The project page can be found here.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.