ICML 2026 Papers

Realistic simulation is critical for applications ranging from robotics to animation. Video generation models have emerged as a way to capture real-world physics from data, but they often face challenges in maintaining spatial consistency and object permanence, relying on memory mechanisms to compensate. As a complementary direction, we present 3DGSim, a learned 3D simulator that directly learns physical interactions from multi-view RGB videos. 3DGSim adopts MVSplat to learn a latent particle-based representation of 3D scenes, a Point Transformer for the particle dynamics, a Temporal Merging module for consistent temporal aggregation, and Gaussian Splatting to produce novel view renderings. By jointly training inverse rendering and dynamics forecasting, 3DGSim embeds physical properties into point-wise latent features. This enables the model to capture diverse behaviors, from rigid and elastic to cloth-like dynamics and boundary conditions (e.g., fixed cloth corners), while producing realistic lighting effects. We show that 3DGSim can generate physically plausible results even in out of distribution cases, e.g. ground removal or multi-object interactions, despite being trained only on single-body collisions.

3D Gaussian Splatting (3DGS) has demonstrated remarkable performance in novel view synthesis and 3D scene reconstruction, but its quality often degrades in real-world environments due to transient distractors, such as moving objects and varying shadows. Existing methods commonly introduce semantic priors from pre-trained vision models either to group pixels into coherent regions or to define perceptual error metrics. However, semantic grouping is often misaligned with the binary static/transient distinction, while perceptual features can be fragile under appearance perturbations introduced during 3DGS optimization. We propose 3DGS-HPC, a framework that addresses these issues by combining two complementary principles: a patch-wise classification strategy that leverages local spatial consistency for robust region-level decisions, and a hybrid classification metric that adaptively integrates photometric and perceptual cues for more reliable separation. Extensive experiments demonstrate the superiority and robustness of our method in mitigating distractors to improve 3DGS-based novel view synthesis. Our project page is https://cnhaox.github.io/3DGS-HPC/.

Precise property prediction of organic materials is pivotal for next-generation electronic and energy devices. In density functional theory (DFT), the electron density (ED) serves as the fundamental determinant of material properties. Yet, establishing it as an input modality for material property prediction has been impeded by two practical barriers: scarce large-scale ED data and the enormous computational complexity of ED representation. To bridge these gaps, we introduce VisionED, an efficient physics-informed model pre-trained on electron-density images. We curate a dataset of 2 million molecules and represent ED as multi-shot images that efficiently encode both geometric and electronic structure. VisionED is then pre-trained on 12 million multi-shot ED images via cross-scale, physics-informed pretext tasks. Empirical evaluations on photovoltaic and organic chromophore datasets show that VisionED outperforms state-of-the-art baselines by up to 27.0\%, exhibiting superior robustness under distribution shifts and data scarcity. Notably, the model generalizes to unseen device-scale applications, successfully recovering experimental trends and mixing-ratio effects in ternary blends with an average accuracy of 92.77\%. Moreover, relative to the previous ED point cloud, the ED image improves performance by 26.2\% with 2.6$\times$ fewer memory and 4.6$\times$ lower time. The code and data are available at https://github.com/ZhixiangCheng/VisionED.

LiDAR point cloud anomaly detection is critical for autonomous system safety, yet most existing methods rely only on visible measurements, overlooking occlusion as a structured consequence of the LiDAR sensing process. We argue that anomalies are characterized not only by what is observed, but also by the spatial voids they create, which alter occlusion patterns and volumetric visibility. We propose Counterfactual Occlusion-Visibility Anomaly Learning (COVAL), a framework that intervenes on volumetric visibility during training. Using physics-conformed synthetic anomaly construction, COVAL generates paired factual and counterfactual observations with identical scene geometry but different occlusion patterns. Then, we introduce two complementary objectives: Visibility-Variant Counterfactual Reconstruction, which models occlusion-induced missing regions, and Visibility-Invariant Counterfactual Consistency, which enforces stable representations across visibility changes. Together, these objectives isolate anomaly-induced structural missingness and in turn refine representation of normal scenes, thus improving anomaly sensitivity at test time. Experiments on standard LiDAR anomaly segmentation benchmarks show that COVAL achieves state-of-the-art performance.

We present WorldMirror, a unified feed-forward model for comprehensive 3D geometric prediction tasks. Unlike existing methods constrained to image-only inputs or customized for a specific task, our framework flexibly integrates diverse geometric priors, including camera poses, intrinsics, and depth maps, while simultaneously generating multiple 3D representations: dense point clouds, multi-view depth maps, camera parameters, surface normals, and 3D Gaussians. Remarkably, prior injection yields universal gains across all tasks, suggesting that input flexibility and multi-task prediction are mutually reinforcing. WorldMirror achieves state-of-the-art performance across diverse benchmarks from camera, point map, depth, and surface normal estimation to novel view synthesis, while maintaining the efficiency of feed-forward inference. Code and model weights are publicly available at https://github.com/Tencent-Hunyuan/HunyuanWorld-Mirror.

Current evaluations of spatial intelligence can be systematically invalid under modern vision-language model (VLM) settings. First, many benchmarks derive question-answer (QA) pairs from point-cloud-based 3D annotations originally curated for traditional 3D perception. When such annotations are treated as ground truth for video-based evaluation, reconstruction and annotation artifacts can miss objects that are clearly visible in the video, mislabel object identities, or corrupt geometry-dependent answers (e.g., size), yielding incorrect or ambiguous QA pairs. Second, evaluations often assume full-scene access, while many VLMs operate on sparsely sampled frames (e.g., 16-64), making many questions effectively unanswerable under the actual model inputs. We improve evaluation validity by introducing ReVSI, a benchmark and protocol that ensures each QA pair is answerable and correct under the model's actual inputs. To this end, we re-annotate object labels and geometry across 413 scenes from 5 datasets to improve data quality, and regenerate all QA pairs with rigorous bias mitigation and human verification using professional 3D visualization and annotation tools. We further enhance evaluation controllability by providing variants across multiple frame budgets (16/32/64/all) and fine-grained object visibility metadata, enabling controlled diagnostic analyses. Evaluations of general and domain-specific VLMs on ReVSI reveal systematic failure modes that are obscured by prior benchmarks, yielding a more reliable and diagnostic assessment of spatial intelligence.

Point cloud completion and generation are important across many 3D tasks, where both fidelity and sampling efficiency matter. Prevailing high-fidelity approaches rely on long sampling schedules, which incur substantial inference latency. Few-step alternatives typically use rectification or distillation, leading to multi-stage training pipelines and potential quality trade-offs. We present 3D MeanFlow (3DMF), a distillation-free model that performs one-step average-velocity transport for point cloud completion and generation. We optimize an instantaneous-average consistency objective and impose a shape-level constraint to stabilize training. Additionally, we introduce PointPlug, integrating completion into 3D object detectors and evaluating its impact. PointPlug uses adaptive selection that balances benefit and latency. Across standard benchmarks, 3DMF achieves one-step sampling with an order-of-magnitude speedup while maintaining competitive fidelity. On nuScenes and KITTI, inserting PointPlug improves all evaluated detectors under comparable settings.

Few-shot point cloud semantic segmentation (FS-PCSS) aims to achieve precise segmentation of novel categories using only limited labeled samples. Existing prototype-based methods rely on shallow feature fusion, failing to adequately model the feature distribution shift between support and query sets, resulting in insufficient prototype adaptation. To address this, we propose the Deep Prototype Refinement Network (DPR-Net), which systematically achieves progressive adaptation by constructing a coarse-to-fine prototype evolution trajectory. Our core Dynamic Prototype Refinement (DPR) module decomposes features into common and distinctive subspaces based on channel activation, enabling targeted adjustment of domain-sensitive features while preserving class-shared semantics. By cascading multiple refinement modules, we construct a prototype trajectory transitioning from support-biased to query-adapted representations, mitigating both under- and over-adaptation. Furthermore, our Mixture of Prototype Experts (MoPE) mechanism treats prototypes at different stages as experts and ensembles their predictions through confidence-driven weighting. Extensive experiments demonstrate state-of-the-art performance with high efficiency. Our code will be available at https://github.com/changshuowang/DPR-Net.

Reconstructing complete yet compact 3D Gaussian Splatting (3DGS) representations from sparse-view monocular videos remains a significant challenge. While hierarchical training with Video Frame Interpolation (VFI) improves coverage, its correlated pseudo-views and repeated merging accumulate structured, non-i.i.d. redundancy, violating the implicit independence assumptions of standard pruning methods and rendering global thresholding ineffectual. We propose CoverPruneGS, a coverage-preserving structured pruning framework specifically designed for hierarchical 3DGS. Our approach implements a coarse-to-fine pruning pipeline using voxel-based local diversity selection and ground-truth-guided lazy refinement via randomized dropout rendering. To ensure reliable refinement, we introduce a footprint-aware CUDA attribution mechanism. By aggregating ground-truth-aligned error degradation across Gaussian-influenced pixels, we generate faithful importance scores that enable precise, quantile-based "rescue" of essential primitives. Experimental results across multiple datasets demonstrate that CoverPruneGS substantially reduces Gaussian counts by 56.8\% and significantly accelerates inference speeds, all while enhancing or maintaining the quality of novel view synthesis.

Transparent objects pose a fundamental challenge for depth estimation and 3D reconstruction due to their violation of Lambertian assumptions, leading to severe geometry degradation in downstream tasks. To address this, we propose a novel geometry-guided preprocessing framework GHOST that leverages visual foundation models to transform transparent regions into opaque, structurally consistent representations without requiring downstream model retraining. Specifically, our pipeline utilizes (1) TransDINO and (2) TransDecomp to disentangle masks and transparency physical properties, while (3) DAF-Net recovers surface normal priors to encode geometric curvature. Subsequently, (4) GeoSemTransNet integrates these multi-modal cues to synthesize a texture-rich opaque RGB image that preserves the transparent object's 3D structure. Extensive experiments demonstrate that our method significantly enhances the accuracy of state-of-the-art depth estimation and reconstruction models on transparent objects by restoring essential photometric cues.

Vision–language models achieve incredible performance across a wide range of tasks, but their large size makes inference costly. Recent work has shown that multimodal processing contains significant redundancies, making it possible to skip certain layers with minimal performance loss. Yet current pruning techniques remain ad-hoc, relying on heuristics or hyperparameter sweeps rather than principled criteria for determining when layer skipping is beneficial. In this paper, we propose a unified framework that characterizes the redundancy conditions under which pruning can enhance efficiency without sacrificing performance. Central to our approach are experimentally verifiable and interpretable notions of redundancy that can be evaluated without requiring downstream task performance as a metric. Applying this framework, we corroborate prior findings that both early and late vision tokens are redundant across models, and we validate our conditions by showing they align with actual performance degradation. Beyond these empirical results, our framework provides a theoretically grounded understanding of redundancy in VLMs and unifies many of the ideas behind modern layer-skipping techniques.

Image-to-point cloud registration is often challenged by viewpoint changes, cross-modal discrepancies, and repetitive textures, which induce scale ambiguity and consequently lead to erroneous correspondences. Recent detection-free methods alleviate this issue by leveraging multi-scale features and transformer-based interactions. However, they still suffer from attention drift across layers and intra-scale inconsistencies, hindering precise registration. Inspired by complex scene observation, we propose a ``Focus--Sweep'' paradigm and develop a Hierarchical Mamba Interaction Module within an SSM-based framework to enhance multi-level cross-modal feature association. In addition, we introduce a Dynamic Layer Allocation Strategy that adaptively determines the iteration depth to better exploit geometric constraints and improve matching robustness. Extensive experiments and ablations on two benchmarks, RGB-D Scenes V2 and 7-Scenes, demonstrate that our approach achieves state-of-the-art performance.

Existing offline feed-forward methods for joint scene understanding and reconstruction on long image streams often perform global computation over an ever-growing set of past observations, causing runtime and GPU memory to increase rapidly with sequence length. We propose Streaming Semantic Gaussian Splatting (S2GS), a strictly causal and incremental framework that builds a 3D Gaussian semantic field from image streams without accessing future frames or reprocessing historical observations. S2GS continuously updates scene geometry, appearance, and instance-level semantics through a geometry--semantic decoupled dual-backbone design. The geometry branch performs causal modeling for incremental Gaussian updates, while the semantic branch leverages a 2D foundation vision model and a query-driven decoder to predict masks and identity embeddings. Query-level contrastive alignment and lightweight online association with an instance memory further stabilize temporal identities. Experiments show that S2GS matches or outperforms strong offline baselines, while substantially improving long-horizon scalability: it processes over 1,000 frames with much slower runtime and GPU memory growth, whereas offline global-processing baselines typically run out of memory at around 80 frames under the same setting. Project Page: https://stdcoutzrh.github.io/projects/s2gs/.

We address 4D reconstruction from partial point cloud sequences, where depth-sensor observations are incomplete, unordered, and lack explicit temporal correspondences. This geometry-only setting is challenging due to missing observations and ambiguous dynamics. While recent progress has largely relied on image-based methods, existing point-based approaches typically focus on single objects, assume relatively complete inputs, or require explicit correspondences. To address these limitations, we propose DynaTok, a point-based framework for correspondence-free 4D reconstruction from partial point cloud sequences without images. DynaTok encodes frames into compact latent tokens, aggregates incomplete observations over time with a Transformer-based spatiotemporal encoder, and decouples geometry and motion through residual tokens in a unified model. A flow-matching decoder then reconstructs complete, temporally consistent 4D point-cloud sequences conditioned on the latent tokens. Experiments on object- and scene-level benchmarks demonstrate improved reconstruction quality and temporal coherence from partial point cloud observations.

We present 4RC, a unified feed-forward framework for 4D reconstruction from monocular videos. Unlike existing methods that typically decouple motion from geometry or produce limited 4D attributes, such as sparse trajectories or two-view scene flow, 4RC learns a holistic 4D representation that jointly captures dense scene geometry and motion dynamics. At its core, 4RC introduces a novel encode-once, query-anywhere and anytime paradigm: a transformer backbone encodes the entire video into a compact spatio-temporal latent space, from which a conditional decoder can efficiently query 3D geometry and motion for any query frame at any target timestamp. To facilitate learning, we represent per-view 4D attributes in a minimally factorized form, decomposing them into base geometry and time-dependent relative motion. Extensive experiments demonstrate that 4RC outperforms prior methods across a wide range of 4D reconstruction tasks.

Compositional generalization, the ability to recognize familiar parts in novel contexts, is a defining property of intelligent systems, yet modern models, despite massive training sets, see only a tiny fraction of the combinatorial input space. We ask what structure representations {must} have to support generalization to unseen combinations. We formalize three desiderata (divisibility, transferability, stability) and show they impose necessary geometric constraints under standard training: representations must decompose linearly into per-concept components, orthogonal across concepts. This grounds the Linear Representation Hypothesis as a necessary consequence of compositional generalization, and yields dimension bounds linking the number of composable concepts to embedding geometry. Empirically, across CLIP, SigLIP, and DINO, we find partial linear factorization with low-rank near-orthogonal per-concept factors, and the degree of this structure correlates with compositional generalization on unseen combinations. As models continue to scale, these conditions predict the geometry they may converge to. Code: https://github.com/oshapio/necessary-compositionality

While existing methods for reconstructing hand–object interactions have made impressive progress, they either focus on rigid or part-wise rigid objects—limiting their ability to model real-world objects (e.g., cloth, stuffed animals) that exhibit highly non-rigid deformations—or model deformable objects without full 3D hand reconstruction. To bridge this gap, we present PhysHanDI (Physics-based Reconstruction of Hand and Deformable Object Interactions), a framework that enables full 3D reconstruction of both interacting hands and non-rigid objects. Our key idea is to physically simulate object deformations driven by forces induced from densely reconstructed 3D hand motions, ensuring that the reconstructed object dynamics are both physically plausible and coherent with the interacting hand movements. Furthermore, we demonstrate that such simulation of object deformations can, in turn, refine and improve hand reconstruction via inverse physics. In experiments, PhysHanDI outperforms the state-of-the-art baseline across reconstruction and future prediction.

Monocular 3D human pose estimation is fundamentally challenged by severe occlusion and inherent depth ambiguity. To address this, we propose Glimpse, a framework that learns robust 3D poses by explicitly modeling anatomical geometry from a single image. We recast the problem as geometry learning of multi-scale structural priors, realized through two synergistic components. First, structured sampling captures the body's geometric continuity through dual-level feature extraction, acquiring both local joint appearance and continuous features along skeletal limbs via deformable sampling. By propagating limb-level geometric cues to their connected joints, this design bridges information gaps caused by occlusion. Second, geometric correction ensures global 3D consistency by lifting coherent 2D features into a canonical 3D reference space, where a shared 3D anchor guides a distance-aware fusion mechanism. Extensive experiments conducted on Human3.6M and MPI-INF-3DHP demonstrate that Glimpse achieves state-of-the-art performance, with superior robustness under severe occlusion and complex articulation.

Linear attention mitigates the quadratic complexity of softmax attention but suffers from a critical loss of expressiveness. We identify two primary causes: (1) The normalization operation cancels the query norm, which breaks the correlation between a query's norm and the spikiness (entropy) of the attention distribution as in softmax attention. (2) Standard techniques for enforcing non-negativity cause destructive information loss by nullifying valid inner-product interactions. To address these challenges, we introduce **NaLaFormer**, a novel linear attention mechanism built upon a norm$\times$direction (ND) decomposition of the query and key vectors. We leverage each component to solve a distinct problem: The *query norm* is injected into our kernel to create a query-norm-aware map that restores the attention distribution's spikiness. The *direction vectors* are processed by a geometric, cosine-based similarity metric that guarantees non-negativity while preserving the rich, fine-grained information of the inner product. We validate NaLaFormer through a comprehensive multi-modal evaluation, where it sets new state-of-the-art benchmarks for linear attention. Our model achieves up to a 7.5\% accuracy gain on ImageNet-1K and a 4.7\% mIoU improvement on ADE20K over comparable baselines. It demonstrates profound efficiency, reducing peak memory by a transformative 92.3\% in token-intensive super-resolution tasks (70K+ tokens). NaLaFormer's versatility is further confirmed as it surpasses strong baselines like Mamba on common-sense reasoning and sets a new state-of-the-art on the Long Range Arena (LRA) benchmark. Source code can be found in the supplementary materials.