paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

ECCV 2026

Granular Semantic Cognition for Visible-Infrared Person Re-Identification

Haifeng Yang, Jinjia Peng, Huibing Wang

Visible-infrared person re-identification aims to match pedes-trian images of the same identity captured by visible and infrared cam-eras, and is hindered by severe domain gaps due to the absence of colorin the infrared modality. Some methods employ text to supplement colorcues in infrared modality. However, they rely on global textual guidance,which hinders fine-grained alignment between textual color informationand local infrared semantics. To this end, this paper proposes a GranularSemantic Cognition (GSC) method, which leverages cross-modal sharedsemantics to facilitate the transfer of color features. Specifically, a Hierar-chical Refinement Module (HRM) progressively refines shared semanticprototypes via multi-stage interactions with pixel-wise features, yieldingfine-grained semantics. Based on these refined semantics, a Semantics-guided Adjustment Module (SAM) generates semantic-conditioned fu-sion weights to selectively aggregate textual color information in se-mantically relevant regions. In addition, a fusion-weighted identity con-trastive loss is proposed to alleviate semantic ambiguity across identi-ties by regularizing the semantics-guided fusion process. Extensive ex-perimental results conducted on the SYSU-MM01, RegDB and LLCMdatasets demonstrate the favorable performance.

ECCV 2026

Geometry-Aware Spatio-Temporal Context Modeling for 4D Occupancy Forecasting

Sitao Chen, Zhuangwei Zhuang, Hui Luo, Qingyao Wu, Mingkui Tan

4D occupancy forecasting models the spatio-temporal evo-lution of 3D scenes and is crucial for autonomous driving, especially forcorner-case simulation. Existing methods often rely on discrete tokeniza-tion followed by autoregressive prediction, yet struggle with geometricdistortion in static structures and inconsistent temporal coherence overthe forecasting horizon. In this work, we propose a Geometry-AwareSpatio-Temporal context modeling method (GAST) for 4D occupancyforecasting, built upon progressive explicit-implicit generation and dual-path spatio-temporal modeling. Specifically, the generation module pro-duces per-frame occupancy with high geometric fidelity and semanticplausibility through pose-driven warping, motion-aware feature modula-tion, and attention-based feature refinement. Subsequently, the spatio-temporal module enhances spatial consistency through global contextaggregation while capturing scene evolution through temporal dynam-ics extraction. This unified design enables joint optimization of histori-cal reconstruction and future forecasting in an end-to-end manner. Ex-tensive experiments on Occ3D-nuScenes demonstrate the superiority ofour method, outperforming the state-of-the-art by 7.67% in mIoU and6.44% in IoU with a 2.84× speedup, while maintaining strong perfor-mance in long-term forecasting. Our source code is publicly available athttps://github.com/chenst27/GAST.

ECCV 2026

GeoFlow: Efficient Driving Video Generation via Geometry-Aligned Priors

jiazheng liu, Hang Li, Jiawei Zhang, Jiahe Li, Xiaohan Yu, Shengyin Fan, Jin Zheng, Xiao Bai

Generative models like Di_x001B_usion Models and Flow Matchinghave demonstrated remarkable capabilities in synthesizing high-_x001C_delitydriving videos, but are severely constrained by high inference latencydue to the requirement of extensive sampling steps. We argue that thisine_x001E_ciency stems from the prevailing reliance on a standard Gaussiansource distribution, where consecutive frames are initialized as indepen-dent Gaussian noise. This paradigm disregards the rich spatiotemporalcorrelations inherent in driving videos, compelling the model to regener-ate deterministic scene structures existing in previous frames from noise,which is both computationally redundant and prone to geometric incon-sistency. To address this problem, we propose GeoFlow, a novel frame-work designed to achieve e_x001E_cient driving video generation by harnessingexplicit geometric priors. Instead of sampling from standard Gaussiannoise, we leverage multi-view geometry and spatially-adaptive noise in-jection to construct a Geometry-Aligned Prior (GAP) distribution asstarting point. This initialization bridges the gap between source dis-tribution and data distribution, yielding a signi_x001C_cantly straighter andshorter sampling trajectory. Extensive experiments demonstrate that Ge-oFlow can achieve remarkable e_x001E_ciency of both training and inference:merely several hours of _x001C_ne-tuning on baseline models can signi_x001C_cantlyboost few-step generation quality, while fully converged training drasti-cally reduces number of inference steps required for state-of-the-art videogeneration.

ECCV 2026

Generative Lane Topology Reasoning via Autoregressive Model with Geometry Prior

Jiahui Fu, Zehao Huang, Han Li, Naiyan Wang, Si Liu

Lane topology reasoning aims to construct a lane graph fromonboard sensor observations. Existing methods follow a detection and as-sociation paradigm that treats each lane instance independently, leadingto geometric inconsistency at connected endpoints and incomplete graphsdue to visual occlusions. To address these issues, we propose TopoGPT,a generative framework that learns the geometry prior from typical lanegraph structures through autoregressive sequence modeling. Specifically,we construct a large-scale map dataset comprising 3.3M scenes. For eachlane graph, a lane tokenizer serializes it into discrete tokens, while a scenecontext encoder converts it into a rasterized image and extracts globalfeatures as scene tokens. We pre-train an autoregressive lane sequencetransformer via scene-conditioned next-token prediction, endowing themodel with the geometry prior over lane graph structures. Building uponthis prior, a perception adapter aligns BEV features from multi-view im-ages with the pre-trained scene condition, transferring the learned geom-etry prior to sensor-based lane graph prediction. On the OpenLane-V2benchmark, TopoGPT outperforms existing methods by an average of+6.4 on lane-level and +11.6 on point-level metrics, and produces geo-metrically consistent and structurally complete lane graphs. Our projectpage is available at https://buaa-colalab.github.io/topogpt_page.

ECCV 2026

Gaze-to-text Generation: Beyond Categorical Decoding of Human Attention

Sounak Mondal, Dimitris Samaras, Gregory Zelinsky, Minh Hoai Nguyen

We introduce a novel learning problem: decoding gaze intonatural language descriptions of human goals across diverse visual tasks.Unlike prior work, which frames gaze decoding as a discriminative taskover predefined categories, we formulate it as a generative learning prob-lem: training a model to produce free-form descriptions that capture therich nuances and open-ended nature of human intentions beyond fixedlabels. To this end, we introduce Gazette, the first gaze-to-text decod-ing framework. Based on multimodal large language models (MLLMs),Gazette learns to decode gaze scanpaths into natural language for goalsthat may extend beyond categorical labels and require articulation innatural language. To help Gazette filter out individual differences in gazebehavior and learn the goal-specific spatiotemporal dynamics crucial forgenerating accurate natural language goal descriptions, we propose anovel strategy that leverages the encyclopedic knowledge and reason-ing abilities of a large language model to synthesize natural languageexplanations of goal-directed attentional behavior called think-aloud tran-scripts. Instruction tuning on these synthetic narratives allows Gazetteto achieve state-of-the-art performance in gaze decoding across mul-tiple tasks, demonstrating its generalizability and versatility, therebyenabling gaze to serve as a powerful, non-intrusive cue for inferringhuman goals and intentions in diverse scenarios. Code is available athttps://github.com/cvlab-stonybrook/Gazette.

ECCV 2026

From Reconstruction to Decision: A Post-Encoder Plug-in Adapter for Curvilinear Segmentation

Qin Lei, Jiang Zhong, Xin Xiao, Ymyang Ymyang, Hao Wu

Curvilinear object segmentation, including vessels and cracks, is challenging due to extreme spatial sparsity and topological fragility, where small local errors can cause severe structural disconnections. Meanwhile, modern segmentation pipelines increasingly rely on strong but hard-to-modify foundation encoders whose heavy downsampling limits fine structural recovery. Motivated by this, we focus on the post-encoder stage and study two recurring and actionable failure modes: a reconstruction bottleneck in high-resolution feature restoration and a decision bottleneck in binarization. We present PEPA, a lightweight Post-Encoder Plug-in Adapter for 2D curvilinear segmentation pipelines with accessible decoder/head features and target, query, or class descriptors. PEPA couples (i) Target-Conditioned Snake Upsampling (TCSU), which uses target-conditioned continuous snake-like sampling to better recover thin and tortuous structures during upsampling, and (ii) Target-Adaptive Differentiable Thresholding (TADT), which predicts target-specific thresholds and optimizes a soft-threshold surrogate with explicit safeguards against trivial bias shifting. Under this post-encoder interface, PEPA can be attached to both prompt-based decoders and conventional dense predictors. Experiments on five medical and industrial benchmarks show that adding PEPA to frozen-encoder baselines yields consistent improvements, with gains in topological connectivity (clDice) typically exceeding those in region overlap (IoU), indicating improved structural continuity. With only ∼0.26M additional parameters, PEPA offers a practical postencoder enhancement for structure-centric segmentation.

ECCV 2026

Flexible Control of 3D CT Generation via Text and Semantically-Defined Segmentation Prompts

Weicheng Dai, Chenyu Wang, Shantanu Ghosh, Kayhan Batmanghelich

Generative models for volumetric medical images have foundmany applications in medical imaging, ranging from data augmentationto serving as priors for inverse problems. For these applications, generat-ing high-resolution 3D images with strong controllability is essential butremains highly challenging. Existing approaches typically control gener-ation either through radiology reports used as text prompts or throughfull image segmentation. While text-based prompting is flexible, it pro-vides limited spatial control over the location, shape, and boundary ofabnormalities. In contrast, segmentation-based methods receive precisespatial guidance but are restrictive in requiring full-organ annotations.In this work, we propose MedSynV2, a flexible multimodal framework forcontrollable volumetric image generation that supports input from radi-ology reports and segmentation prompts (both optional). Our approachallows users to provide segmentation of a specific anatomy or abnormalitywithout requiring full-organ annotations. The semantic meaning of thesegmentation mask is specified through an accompanying text descrip-tion, resulting in a highly flexible and scalable conditioning mechanism.We develop a memory-e!cient architecture based on a modified di"usiontransformer that jointly processes image and segmentation tokens. Themodel further incorporates gated attention to e"ectively attend to longradiology reports. Experiments demonstrate that our method achievesstate-of-the-art perceptual and semantic scores (e.g., → 24% relative im-provement in mean FID), generates high-resolution anatomically consis-tent CT volumes, and improves data e!ciency when used for data aug-mentation. Radiologists’ evaluation further confirms strong alignmentbetween generated and real medical images. We also explore the abilityof the model to generalize towards concepts beyond training data. Codewill be released at https: // github. com/ batmanlab/ MedSynV2 .

ECCV 2026

Explainability-aware Frustum Attack: Exposing Structural Vulnerabilities in LiDAR-Based 3D Object Detectors

Chengzeng You, Binbin Xu, Soteris Demetriou

The structural vulnerabilities of point cloud–based 3D objectdetectors remain poorly understood. Prior work has studied adversarialrobustness primarily on isolated 3D object models, while recent LiDARspoofing attacks target richer and more realistic driving scenes but focusmainly on physical realizability rather than understanding detector be-havior or attack efficiency. In this work, we investigate how LiDAR-baseddetectors rely on spatial evidence in complex scenes and whether these re-liance patterns can be exploited to induce failures more efficiently. To thisend, we propose an explainability-guided adversarial analysis methodology.We introduce the Saliency-LiDAR (SALL) method, which aggregates In-tegrated Gradient attributions across scenes to produce universal saliencymaps for LiDAR-based 3D object detectors. Guided by these maps, wedesign the Explainability-aware Frustum Attack (EFA), which selectivelyperturbs only the most influential frustums rather than uniformly attack-ing entire object regions. Experiments on KITTI and nuScenes, acrossdetectors such as PointPillars and SECOND, show that EFA reducesdetection recall by more than 15 percentage points while requiring 25–50%fewer perturbed frustums than the SOTA non–saliency-aware baseline.These findings reveal that modern 3D detectors concentrate discrimina-tive evidence in a small subset of spatial regions, exposing a structuralrobustness vulnerability in current LiDAR perception systems. Our codeis released at https://github.com/SecMindLab/Saliency_LiDAR.

ECCV 2026

ELHINN: Unifying Dense Crowd Simulation Across Scales via Eulerian–Lagrangian Hydrodynamics

Yanshan Zhou, Pingrui Lai, Jiaqi Yu, Cunyan Li, Hua Yang, Xiaoyun Zhang

Developing effective dense crowd simulation is challengingdue to the scale gap between macroscopic collective coherence and mi-croscopic individual realism. Existing macroscopic (Eulerian) methodscapture global motion patterns but lack individual trajectories, whereasmicroscopic (Lagrangian) methods model individual behaviors yet oftenfail to preserve systemic consistency in dense scenarios. To address this,we propose the Eulerian–Lagrangian Hydrodynamics-Informed NeuralNetwork (ELHINN), a unified cross-scale framework that couples macro-scopic velocity evolution with microscopic trajectory refinement by usingevolved Eulerian velocity fields as physical priors to guide Lagrangian tra-jectories. For velocity evolution, we develop an enhanced Hydrodynamics-Informed Neural Network (HINN++) that incorporates a learnable gov-erning equation with Kolmogorov–Arnold Network (KAN) residual cor-rection and environmental boundary conditions, enabling accurate mod-eling of complex nonlinear dynamics under varying scenarios. For tra-jectory refinement, we use a Physics-Informed Neural Network (PINN)augmented with an entrance-aware resampling strategy (EARS) andcollision-avoidance constraints to ensure stability and fidelity. Experi-ments on two real-world crowd datasets demonstrate that ELHINN out-performs existing methods in simulating dense crowds across both scales.Codes are available at https://github.com/shanshan-zys/ELHINN.

ECCV 2026

Deconfounded Lifelong Learning for Autonomous Driving via Dynamic Knowledge Spaces

Jiayuan Du, Yuebing Song, Yiming Zhao, Xianghui Pan, Jiawei Lian, Yuchu Lu, Liuyi Wang, Chengju Liu, Qijun Chen

End-to-End autonomous driving (E2E-AD) systems face chal-lenges in lifelong learning, including catastrophic forgetting, difficulty inknowledge transfer across diverse scenarios, and spurious correlationsbetween unobservable confounders and true driving intents. To addressthese issues, we propose DeLL, a Deconfounded Lifelong Learning frame-work that integrates a Dirichlet process mixture model (DPMM) withthe front-door adjustment mechanism from causal inference. The DPMMis employed to construct two dynamic knowledge spaces: a trajectoryknowledge space for clustering explicit driving behaviors and an implicitfeature knowledge space for discovering latent driving abilities. Leverag-ing the non-parametric Bayesian nature of DPMM, our framework en-ables adaptive expansion and incremental updating of knowledge withoutpredefining the number of clusters, thereby mitigating catastrophic for-getting. Meanwhile, the front-door adjustment mechanism utilizes theDPMM-derived knowledge as mediators to deconfound spurious correla-tions, such as those induced by sensor noise or environmental changes,and enhances the causal expressiveness of the learned representations.Additionally, we introduce an evolutionary trajectory decoder that en-ables non-autoregressive planning. To evaluate the lifelong learning per-formance of E2E-AD, we propose new evaluation protocols and metricsbased on Bench2Drive. Extensive evaluations in the closed-loop CARLAsimulator demonstrate that our framework significantly improves adapt-ability to new driving scenarios and overall driving performance, whileeffectively retaining previously acquired knowledge.

ECCV 2026

MobileManiBench: Simplifying Model Verification for Mobile Manipulation

Wenbo Wang, Fangyun Wei, Qixiu Li, Xi Chen, Yaobo Liang, Chang Xu, Jiaolong Yang, Baining Guo

Vision-language-action models have advanced robotic ma-nipulation but remain constrained by reliance on the large, teleoperation-collected datasets dominated by the static, tabletop scenes. We propose asimulation-first framework to verify VLA architectures before real-worlddeployment and introduce MobileManiBench, a large-scale benchmarkfor mobile-based robotic manipulation. Built on NVIDIA Isaac Sim andpowered by reinforcement learning, our pipeline autonomously gener-ates diverse manipulation trajectories with rich annotations (languageinstructions, multi-view RGB–depth–segmentation images, synchronizedobject/robot states and actions). MobileManiBench features 2 mobileplatforms (parallel-gripper and dexterous-hand robots), 2 synchronizedcameras (head and right wrist), 630 objects in 20 categories, 5 skills(open, close, pull, push, pick) with over 100 tasks performed in 100 re-alistic scenes, yielding 300K trajectories. This design enables controlled,scalable studies of robot embodiments, sensing modalities, and policy ar-chitectures, accelerating research on data efficiency and generalization.We benchmark representative VLA models and report insights into per-ception, reasoning, and control in complex simulated environments, withall code, datasets, and models publicly released at our project website:https://dexhand.github.io/MobileManiBench/.

ECCV 2026

Stabilizing Real-World Visual Active Tracking with Action-Smooth Test-Time Adaptation

Haowei Sun, Shiteng Zhang, Jinwu Hu, Kaining Chen, Mingkui Tan

Visual Active Tracking (VAT) requires an embodied agent tocontrol cameras to follow a designated target, which is essential for ap-plications like robot navigation and security monitoring. However, trans-ferring VAT trackers to the real world faces two primary bottlenecks: (1)severe visual domain shifts between simulated environments and the realworld, leading to performance collapse during deployment; and (2) tar-get distribution shift caused by similar-looking distractors in real-worldscenes. To address these, we propose VATA, a novel test time adaptationmethod with two complementary strategies. First, we introduce a CriticValue Maximization strategy that leverages value signals from a criticmodel to filter reliable test samples, enabling unsupervised model up-dates to mitigate visual shift. Second, we propose an Action Smooth-ness Regularization term that exploits the continuity of real-worldtarget motion to bound action variations, correcting irrational actionsunder target shift. Experiments demonstrate VATA achieves a 24.4%SR improvement on the EVT-Benchmark and reaches 80% TSR in 10real-world scenarios (vs. 30% by the SOTA TTA method), providing arobust and plug-and-play solution for VAT deployment.

ECCV 2026

DICT: Data Injection and Contrastive Trajectory Refinement for Conditional Image Generation with Diffusion Models

Chunnan Shang, Zhizhong Wang, Xin Zhang, Hongwei Wang

Diffusion models have become a dominant paradigm for conditional image generation, yet existing approaches generally follow two directions: task-specific designs that can improve performance but limit generalization, and training-free loss guidance that compresses rich conditions into scalar objectives and applies stepwise guidance, leading to information bottlenecks and error accumulation along the sampling trajectory. Given the urgent need for an effective unified framework across diverse conditional image generation tasks, we propose Data Injection and Contrastive Trajectory Refinement (DICT), a training-free inference method that enhances conditional image generation without introducing task-dependent architectures. DICT introduces Data Injection, where noise-perturbed conditional signals are integrated into early denoising stages; by performing guided denoising on these injected signals, DICT adaptively selects and distills task-salient information from the raw condition, effectively preserving spatial richness and ensuring precise condition-to-generation alignment. Furthermore, DICT applies Contrastive Trajectory Refinement across adjacent denoising states, enabling pairwise comparisons that progressively improve sample quality. These designs keep inference simple while improving cross-task transfer under a unified diffusion formulation. Extensive experiments on conditional image generation tasks (e.g., style transfer, image super-resolution, and image deblurring) show consistent gains in fidelity and perceptual quality over representative task-specific and loss-guided baselines.

ECCV 2026

DETR is Secretly a Multispectral Detector: Zero-Parameter Adaptation via Semantic Alignment

Xiangyang Li, Zhiwei Jiang, Wushuai Jin, Pengyang Niu, Chunna Tian, Lingqiao Liu

Multispectral object detection enhances perceptual robust-ness in challenging environments by fusing complementary informationfrom RGB and infrared modalities. However, existing methods predomi-nantly follow a parameter-expanding paradigm, introducing customizedfusion modules that inevitably increase computational overhead and dis-rupt the topological consistency with unimodal pre-trained weights. Inthis paper, we challenge this convention and propose ZPA-MDETR,a zero-parameter adaptive multispectral framework without additionallearnable fusion parameters. By re-examining the DETR architecture,we reveal that its inherent query-memory interaction mechanism pro-vides a natural foundation for cross-modal fusion. Specifically, ZPA-MDETR strictly preserves the architecture of the unimodal DETR andachieves flexible and effective fusion via an asymmetric input organiza-tion strategy, where one modality serves as the memory, and the otherinitializes object queries. To bridge the cross-modal distribution dis-crepancy, we introduce parameter-free semantic alignment constraintsduring training. Extensive experiments on four public datasets acrossdifferent scenes demonstrate that ZPA-MDETR outperforms state-of-the-art methods while maintaining the same parameter scale as theunimodal framework, establishing a favorable trade-off between detec-tion accuracy and deployment efficiency. The source code is available athttps://github.com/UserXiangYang/ZPA-MDETR.

ECCV 2026

Depth-guided Multi-view Exposure Bracketing for HDR Robot Vision

Jinnyeong Kim, Juhyung Choi, Woohyeok Kim, Sunghyun Cho, Seung-Hwan Baek

Achieving reliable single-shot high dynamic range (HDR)imaging under extreme illumination conditions remains a long-standingchallenge, yet no comprehensive benchmark exist for evaluating HDRperception in multi-sensor robotic systems. To fill this gap, we intro-duce a large-scale dataset collected via a custom robotic vision platformand an iPhone 13 Pro: 121 real-world scenes spanning modest and ultra-high dynamic range conditions, alongside 20 synthetic video sequencesfrom the CARLA simulator. As a reference pipeline for this dataset,we propose Depth-guided Multi-view Exposure Bracketing (DMEB), asingle-shot HDR method that distributes drastically different exposuresacross multi-view low-bit-depth cameras and fuses them via depth-guidedconfidence-aware fusion. Evaluations on our dataset show that DMEBestablishes a strong reference point and highlight the promise of thissensor configuration for robust HDR perception in diverse multi-cameraand depth sensor system.

ECCV 2026

CulinaryCut: A Physics-aware Vision-Language-Action Benchmark for Food Cutting via Material Point Method

Hyunsuh Koh, Changyong Song, Youngjae Choi, Misa Viveiros, David Hyde, Heewon Kim

Food cutting is a representative non-rigid manipulation taskinvolving deformation, contact forces, and material separation, yet itRobot Simulation remains largely unexplored in VLA research. Existing robot manipula-tion datasets are primarily built around rigid objects and struggle tojointly capture the spatial precision, deformation, topology changes, andCut the banana force interactions required for cutting. To systematically study theseat the center. challenges, we introduce CulinaryCut, a benchmark that integrates anniskill MPM-based deformable simulator with a robot environment to generatedataset rather than provided as policy inputs, enabling evaluation of ex-isting VLAs while exposing where physical grounding is needed. Usingthree representative VLA baselines, we identify two core limitations innon-rigid cutting. First, a geometry gap: VLAs struggle to map ratio-and direction-based instructions onto object geometry, especially whensequential cuts change the object’s topology. Second, a physics gap: be-cause policies output motion without a notion of material stiffness orcontact resistance, a geometrically accurate path may still fail to severthe object. We show that training VLAs on physics-grounded trajecto-ries, without modifying the policy architecture or adding force inputs,improves cutting success and sim-to-real transfer. Together, CulinaryCutprovides a foundation for evaluating spatial reasoning and physical plau-sibility in deformable manipulation.

ECCV 2026

COSY: Compositional 3DGS Synthesis for Disentangled Human Head Editing

Florian Barthel, Shalini De Mello, Koki Nagano, Wieland Morgenstern, Anna Hilsmann, Peter Eisert

Recent 3D Gaussian Splatting (3DGS) GANs for humanheads synthesize and render photorealistic 3D models in real-time andoffer a vast variety in identity and appearance. However, controlling spe-cific semantic attributes such as hair color or glasses remains challenging,as edits in the entangled latent space often induce unintended changesin identity or appearance. Although there are several methods that aimto disentangle the latent space post training by estimating directionsthat only modify certain features, these methods cannot guarantee com-plete disentanglement and often require pre-trained classifiers. In ourapproach, we propose a new generator architecture that synthesizes com-ponents, such as hair, skin, glasses, and torso, completely independently.This allows for changing the latent vector for one region while keepingthe remaining parts fixed. Further, we achieve this separation using onlysparse information such as the hair or skin color, eliminating the re-quirement of segmentation masks or geometric priors, often seen in priorwork. To ensure matching shape and lighting conditions during editing,we allow minimal shared information via context tokens between the in-dependent generators. These tokens even allow us to control the shapeand light, without any prior annotation. Compared to existing works onGAN-based generation and editing, our method shows better disentan-glement, more precise editing control, and competitive visual quality.

ECCV 2026

CoDePose: Multi-View 3D Human Pose Estimation via Coupled 2D-3D Denoising Diffusion

Yanlu Cai, Yuxuan Liu, WEIZHONG ZHANG, Yuan Wu, Cheng Jin

Recent multi-view 3D human pose estimation methods de-compose the problem into per-image 2D pose detection and 3D lift-ing stages, which can naturally scale to multi-frame input and achievestrong results by leveraging temporal context. However, this 2D-3D lift-ing paradigm introduces two levels of geometric inconsistency: (i) projec-tion inconsistency, i.e., the detected 2D keypoints may disagree with the2D projections of the inferred 3D pose, and (ii) cross-view inconsistency,i.e., 2D poses across views are mutually inconsistent because off-the-shelfdetectors operate independently per camera, so there may exist no sin-gle 3D pose whose projections match all detections. We observe that 2Ddetection errors closely follow a Gaussian distribution, meaning that de-tector outputs can be naturally viewed as noisy samples from a diffusionforward process. Based on this insight, we propose CoDePose, a coupled2D-3D denoising diffusion framework that jointly performs multi-viewdenoising of the detected 2D poses and estimates the underlying 3Dpose. As training proceeds, the generated joint distribution progressivelyapproaches the real data distribution, leading to increased projection andcross-view consistency and more reliable 3D estimation. Unlike existingdiffusion-based methods that condition on fixed 2D detections, CoDe-Pose refines the 2D observations during the reverse process, preventing2D detection errors and cross-view inconsistencies from being reinforcedthroughout the whole 3D pose generation. Experimental results on Hu-man3.6M show that CoDePose achieves 12.1 mm MPJPE, a 19.9% im-provement over the previous state-of-the-art method. Cross-dataset eval-uation on HumanEva and MPI-INF-3DHP further demonstrates stronggeneralization.

ECCV 2026

BaFCo: A Document Understanding Benchmark for Complex Bangla Form Comprehension

Abu Tyeb Azad, Ishita Apan, Fahim Ahmed, Sumaiya Karim Katha, Ezharuddin Jubaer, Armun Alam, Pranjal Nandi, Amin Ali, Aman Chadha, Md Mofijul Islam, A K M Mahbubur Rahman

Document comprehension is a challenging yet impactful taskfor Multimodal Large Language Models, especially as these systems seegrowing adoption in real-world, human-centric applications. However,this adoption is limited for low-resource languages such as Bangla dueto the scarcity of high-quality annotated data. To address this gap, weintroduce BaFCo, a benchmark dataset for Bangla form comprehensionwith a focus on Document Layout Analysis (DLA) and Key InformationExtraction (KIE). BaFCo curates 200 multi-page complex Bangladeshigovernment forms, sourced from across diverse sectors including agricul-ture, education, banking, and land management. To accurately capturethe structural and contextual complexity of these forms, we define a fine-grained annotation schema comprising 26 types of form entities, alongwith a separate coarse form entity set consisting of 5 types. We evaluatethe latest MLLMs from the ChatGPT, Gemini, Claude, Qwen, and Kimiseries using zero-shot and chain-of-thought prompts under both low andhigh reasoning setups. Our results reveal limitations in current MLLMs’ability in comprehending Bangla forms, particularly in accurately local-izing highly granular form entities. Our dataset and code is available at:https://huggingface.co/datasets/Mausul/bafco

ECCV 2026

RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Robotics

Yuzhi Huang, Jie Wu, Weijue Bu, Ziyi Xiong, Gaoyang Jiang, Ye Li, Kangye Ji, Shuzhao Xie, Yue Huang, Chenglei Wu, Jingyan Jiang, Zhi Wang

Enabling reliable long-horizon robotic manipulation is a cru-cial step toward open-world embodied intelligence. However, VLM-basedplanners treat each step as an isolated observation-to-action mapping,forcing them to reinfer scene geometry from raw pixels at every decisionstep while remaining unaware of how prior actions have reshaped the en-vironment. Despite strong short-horizon performance, these systems lackthe spatio-temporal reasoning required for persistent geometric anchoringand memory of action-triggered state transitions. Without persistentstate tracking, perceptual errors accumulate across the execution horizon,temporarily occluded objects are catastrophically forgotten, and com-pounding failures lead to precondition violations that cascade throughsubsequent steps. In contrast, humans maintain a persistent mental modelthat continuously tracks spatial relations and action consequences acrossinteractions rather than reconstructing them at each instant. Inspiredby this human capacity for causal spatio-temporal reasoning with persis-tent memory, we propose RoboStream, a training-free framework thatachieves geometric anchoring through Spatio-Temporal Fusion Tokens(STF-Tokens), which bind visual evidence to 3D geometric attributesfor persistent object grounding, and maintains causal continuity via aCausal Spatio-Temporal Graph (CSTG) that records action-triggeredstate transitions across steps. This design enables the planner to tracecausal chains and preserve object permanence under occlusion withoutadditional training or fine-tuning. RoboStream achieves a 90.5% successrate on long-horizon RLBench tasks and a 44.4% success rate on challeng-ing real-world block-building tasks, where both SoFar and VoxPoser score11.1%, demonstrating that spatio-temporal reasoning and causal memoryare critical missing components for reliable long-horizon manipulation.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.