NeurIPS 2026 论文列表

Referring Remote Sensing Image Segmentation (RRSIS) aims to segment target objects from remote sensing images based on natural language descriptions, requiring precise vision–language alignment under significant ambiguity. Despite the strong generalization ability of Segment Anything Model 3 (SAM3), the large spatial extent and varying scales of targets in remote sensing images intensify the ambiguity of referring expressions. This poses a major challenge for fine-grained segmentation. To address the challenge, we propose **EviSAM3**, an evidence-driven parameter-efficient adaptation framework that mimics human decision-making by dynamically accumulating partial evidence. Specifically, EviSAM3 progressively integrates cross-modal evidence through an evidence memory bank, where each incoming piece is aggregated via momentum-based updates. As evidence accumulates, the model performs iterative evidence rectification to resolve ambiguity under incomplete observations. Meanwhile, the contributions of different evidence are adaptively modulated based on their diagnostic relevance, allowing more informative evidence to dominate the reasoning process and guide the final prediction. Extensive experiments demonstrate that EviSAM3 consistently outperforms state-of-the-art methods, particularly in challenging scenarios with high ambiguity and significant scale variation. The code is available in the supplementary material.

Patch-based transformers have emerged as efficient and improved long-horizon modeling architectures for time series modeling. Yet, existing approaches rely on temporally-agnostic patch construction, where arbitrary starting positions and fixed lengths fracture temporal coherence by splitting natural transitions across boundaries. This naive segmentation often disrupts short-term dependencies and weakens representation learning. We propose a novel Entropy-Guided Dynamic Patch Encoder (EntroPE), as a temporally informed framework that dynamically detects transition points via conditional entropy and dynamically places patch boundaries. This preserves temporal structure while retaining the computational benefits of patching. EntroPE consists of two key modules, namely an Entropy-based Dynamic Patcher (EDP) that applies information-theoretic criteria to locate natural temporal shifts and determine patch boundaries, and an Adaptive Patch Encoder (APE) that employs pooling and cross-attention to capture intra-patch dependencies and produce fixed-size latent representations. Extensive experiments on long-term forecasting, classification, and anomaly detection demonstrate that the proposed method improves both accuracy and efficiency, establishing entropy-guided dynamic patching as a promising new paradigm for time series modeling.

The Segment Anything Model 3 (SAM 3) has achieved significant progress in *Promptable Concept Segmentation (PCS)* by processing short noun phrases to generate segmentation masks with unique instance identifiers. While SAM3 excels at single-concept segmentation, it falls short in complex, real-world environments where multiple concepts must be segmented simultaneously. In particular, SAM3 typically fails to simultaneously process concepts with semantic overlap, leading to semantic misclassifications and redundant mask predictions. Moreover, as the number of concepts increases, the independent inference paradigm introduces prohibitive inference latency. To address these limitations, we propose the **D**ense **C**ompetitive **R**epresentations **SAM** (**DCR-SAM**) framework. By decoupling cross-modal interactions, the proposed method alleviates the inference latency. To resolve semantic misclassifications, DCR-SAM dynamically aligns textual concepts with visual representations and incorporates a dense competition head to enforce explicit inter-class competition. Furthermore, a learnable background token is applied to absorb non-target objects dynamically. These mechanisms effectively suppress overlapping predictions and generate high-fidelity masks. Extensive experimental evaluations across multiple benchmarks demonstrate that the proposed framework: I) achieves a 2.3\% $\sim$ 10.7\% performance improvement over state-of-the-art methods, II) delivers a 33$\times$ inference speedup compared to the native SAM 3, and III) exhibits strong robustness across diverse scenarios. All code will be released.

Instruction-based image editing (IIE) models have recently demonstrated strong capability in modifying specific image regions according to natural language instructions, which implicitly requires identifying where an edit should be applied. This indicates that such models inherently perform language-conditioned visual semantic grounding. In this work, we investigate whether this implicit grounding can be leveraged for zero-shot referring image segmentation (RIS), a task that requires pixel-level localization of objects described by natural language expressions. Through systematic analysis, we reveal that strong foreground-background separability emerges in the internal representations of these models at the earliest denoising timestep, well before any visible image transformation occurs. Building on this insight, we propose a training-free framework that repurposes pretrained image editing models for RIS by exploiting their intermediate representations. Our approach decomposes localization into two complementary components: attention-based spatial priors that estimate where to focus, and feature-based semantic discrimination that determines what to segment. By leveraging feature-space separability, the framework produces accurate segmentation masks using only a single denoising step, without requiring full image synthesis. Extensive experiments on RefCOCO, RefCOCO+, and RefCOCOg demonstrate that our method achieves superior performance over existing zero-shot baselines.

Open-vocabulary 3D Gaussian segmentation typically binds semantic features to per-Gaussian state via per-scene gradient training (1–4 h) or the more recent closed-form distillation. Both commit each primitive to a single feature; therefore, boundary Gaussians spanning multiple 2D instances are forced into a hard decision that degrades retrieval accuracy and yields unstable mask boundaries at render time. We present DIGS, a training-free framework with two stages, class-agnostic map- ping and multi-modal encoding, that decouples instance discovery from linguistic naming. At the Gaussian level, our distribution-informed representation maintains an explicit top-M label distribution per primitive, updated by a tempered Dirichlet rule; preservation of multi-peaked posteriors at boundary primitives allows a late-argmax renderer to recover clean instance boundaries. A bidirectional state machine aggregates these distributions into a persistent class-agnostic instance ontology. The instance-level encoding stage then fuses the CLIP visual embedding of each instance’s isolated 3D footprint with the CLIP-text embedding of an LLM-generated attribute description, disambiguating visually similar instances. DIGS maps a scene in 1–3 minutes on a single A100 and serves 3D-native retrieval in 5.4 ms, reaching 71.95% mIoU on LERF-OVS, surpassing the strongest training-free baseline SFS and the strongest training-based baseline LaGa, and setting new state-of-the-art performance on LERF-Mask (91.71%) and 3D-OVS(96.45%). Codes are at: https://anonymous.4open.science/r/DIGS-C4EF.

We cast image segmentation refinement as a \textit{discrete} diffusion process. To solve this task we introduce Cyclic Discrete Diffusion (CDD), a novel forward–reverse formulation with a cyclic label topology that maps a discrete label set to the uniform distribution and back. Instead of injecting Gaussian noise as is done for continuous diffusion, CDD evolves segmentation masks through label jumps: at each step, every pixel either remains in its current class or jumps to the next label with a fixed rate. This jump-or-stay mechanism defines a finite-state Markov process with bounded forward and reverse steps, featuring analytically consistent and tractable reverse dynamics. CDD operates natively in a multi-class label space, enabling joint refinement of all classes in a single pass without requiring class-wise decomposition, as commonly needed in prior binary refinement approaches. The proposed formulation admits a finite and controllable number of refinement steps, for which we provide a theoretical characterization based on spectral gap analysis. We evaluate CDD across diverse segmentation tasks, including multi-class semantic, instance-level, and binary segmentation refinement. Our method consistently improves both region accuracy and boundary quality across multiple backbones and datasets.

Lesion analysis in CT imaging, the task of identifying and delineating abnormal regions across diverse anatomical sites, plays a fundamental role in clinical diagnosis. Existing CT lesion datasets typically focus on a single disease type within a specific region. This design leads to two main limitations. First, co-existing lesions outside the predefined categories are left unannotated, so models may treat real lesions as background during training. Second, lesion categories within any single region follow a long-tailed distribution, in which rare but clinically important categories are underrepresented. However, collecting enough samples for every rare category within a single region is often impractical. We observe that certain lesion categories are rare in one region but occur more frequently in another, motivating the use of cross-region complementarity for lesion detection. In this work, we curate a multi-region CT lesion dataset, namely CT-Lesion, which covers brain, chest, and abdomen CT images. Our CT-Lesion is sourced from a private cohort of 500 clinical CT scans. Seven radiologists spent approximately 1,600 working hours delineating lesions of any type, rather than restricting annotations to a predefined set of categories. As a result, CT-Lesion provides clinically confirmed voxel-level segmentation masks and 28 fine-grained lesion labels. Moreover, we further group the lesions into 9 coarse categories to alleviate data scarcity issues. To demonstrate the effectiveness and utility of CT-Lesion, we benchmark the representative methods on CT-Lesion for lesion segmentation and detection. Furthermore, pretraining on CT-Lesion improves model performance on external benchmarks, including LiTS, KiTS, and LIDC-IDRI, demonstrating its potential as a transferable resource for CT lesion analysis. Our dataset is available at Kaggle.

Connectomics requires dense segmentation of large-scale electron microscopy (EM) volumes using models trained on small labeled blocks. In practice, a connectomics lab facing a new raw volume has two options: annotate a small block and train on it, or skip annotation and train on the union of available public labeled datasets. We introduce BrainEM, a benchmark that formalizes these options as two standard tracks: the Same-Source Track (Annotate-and-Train) and the Cross-Source Track (Reuse-and-Train). BrainEM brings together 16 EM datasets, including one in-house mouse dataset that we contribute, spanning multiple species, imaging modalities, and resolutions. We further add large-scale test volumes that bring the evaluation closer to realistic practice, and four diverse test targets in the Cross-Source Track. The data, post-processing, and evaluation metrics are fixed across all methods, and the framework is fully model-data decoupled so that new methods can be added with minimal effort. We conduct a systematic evaluation of six representative baselines covering the main technical directions in EM neuron segmentation, making BrainEM the first systematic evaluation at large scale and across diverse cross-source targets. The results differ from prior small-scale evaluations: rankings on the Same-Source Track no longer match those reported on small CREMI splits, and methods diverge sharply on the Cross-Source Track. Both tracks still leave clear room for improvement. We release BrainEM as a public platform to support unified comparison and faster method development in connectomics. All datasets and code are available at https://github.com/kwinderic/BrainEM

Video Reasoning Segmentation (VRS) demands both complex semantic reasoning and robust spatiotemporal mask propagation. Existing paradigms typically compress reasoning into an isolated, opaque token (e.g., \texttt{[SEG]}) via expensive fine-tuning, or rely on training-free prompt-then-track pipelines that inevitably suffer from semantic degradation and identity switches when encountering hard distractors. To fundamentally address these vulnerabilities, we propose Counterfactual Logic and Explicit Anchoring Reasoning (\textbf{CLEAR}) framework, a training-free VRS framework inspired by human dual-system cognition. Specifically, we introduce a Causal-Counterfactual Inference mechanism that transforms error-prone coordinate regression into a tractable discrete instance selection task, establishing highly reliable initial anchors via rigorous bidirectional reasoning. During mask propagation, we design an adaptive mode-switching mechanism bridged by a Heterogeneous Event Monitor. The fast-thinking mode utilizes a purely visual engine for efficient temporal propagation, while the Monitor continuously evaluates its reliability across multiple dimensions. Upon detecting propagation anomalies, the slow-thinking mode intervenes and explicitly replaces \texttt{[SEG]} tokens with a structured Contrastive Concept Dictionary for closed-loop identity recovery..Extensive experiments demonstrate that CLEAR significantly outperforms existing training-free methods and achieves performance comparable to state-of-the-art training-based approaches on comprehensive referring and reasoning VOS benchmarks. Code will be available after acceptance.

Understanding long-form video remains a fundamental challenge for multimodal large language models (MLLMs). Sparse frame sampling fails to capture fine-grained visual details, while dense sampling quickly exceeds context length limits. Retrieval-augmented generation (RAG) offers a promising middle ground by selectively retrieving relevant video segments for grounded generation, yet its effectiveness critically depends on the quality of the video segments used as retrieval units. In this paper, we investigate RAG for movie understanding, which demands story-level reasoning over characters, events, and narrative arcs spanning hours of content. Scene segmentation, a long-studied problem that partitions movies into semantically coherent units, is a natural candidate for defining such retrieval units. We reexamine whether existing methods actually serve this role through comprehensive evaluation on downstream movie understanding tasks, and find that they consistently fail to outperform naive uniform temporal chunking. Our audit of the most standard scene segmentation benchmarks reveals why: current annotations prioritize visually salient transitions over narrative event structure. Motivated by this mismatch, we introduce NarraScene, a narrative-centric scene segmentation dataset annotated with a three-level cognitive taxonomy spanning physical, character, and narrative change, where every valid boundary requires a narrative-level shift. When used as retrieval units, these narrative-grounded segments outperform uniform chunking on downstream movie understanding tasks, suggesting that the central challenge for scene segmentation in movie RAG is not detecting boundaries, but identifying the narrative event units that matter for movie understanding.

Semi-supervised medical image segmentation has gained increasing attention for its outstanding performance with limited annotations. Most methods follow the Mean Teacher framework, where Exponential Moving Average (EMA) is primarily used to stabilize predictions, while its potential roles beyond stabilization are largely overlooked. In this work, we interpret EMA as a first-order infinite impulse response (IIR) low-pass filter with inherent frequency selectivity, which enables the preservation of semantic information at different levels. Motivated by this insight, we propose a dual-EMA teachers framework, including a long-term teacher focusing on global structures and a short-term teacher emphasizing local details. Furthermore, to help the student better exploit the guidance from both teachers, a First-In-First-Out replay queue is designed to reuse historical high-confidence pseudo-labeled pairs, leveraging pseudo labels provided at different training stages and reducing the sensitivity to sampling bias. Meanwhile, we impose feature-level global semantic and local relation consistency constraints to alleviate the impact of noisy pseudo labels, facilitating robust transfer of complementary representations from dual teachers to the student model. Experiments on three public medical image datasets demonstrate that our method outperforms current SoTA methods, verifying its effectiveness.

Visual foundation models (VFMs) provide strong region-level representations for semantic segmentation, substantially improving high-level semantic understanding. However, accurate boundary recovery remains challenging, especially for fine structures and semantic transitions. A key reason is that conventional pixel-level supervision largely treats pixels independently and lacks explicit modeling of local structure and neighboring-pixel relationships. To address this issue, we propose a Local Structure (LS) regularization framework that constrains the second-order local organization of semantic transition regions. Unlike existing boundary losses that mainly supervise contour location, distance, overlap, or topology, LS models boundary neighborhoods through a structure-tensor representation of semantic probability fields. It decomposes local boundary structure into two complementary cues: Local Structure Magnitude (LSM), which measures the strength of anisotropic class transitions, and Local Structure Orientation (LSO), which describes their dominant geometric orientation. LS is model-agnostic, compatible with different VFM-backbone-decoder combinations, and introduces no extra inference cost. Extensive experiments on four diverse datasets show that LS consistently improves boundary-sensitive performance while also enhancing overall IoU, demonstrating its effectiveness for boundary-structure learning in semantic segmentation.

Anatomical mesh segmentation requires models that operate directly on irregular surface geometry while remaining robust to arbitrary patient pose and mesh resolution variation. Existing task-specific mesh and point-cloud methods are not equivariant, and can degrade sharply under test-time perturbation, for example dropping by 25-26 IoU points on intraoral scan segmentation at 40$\textdegree{}$ tilt. We present EAMS, an Equivariant Anatomical Mesh Segmentor built on Equivariant Mesh Neural Networks (EMNN), and evaluate it across four clinically distinct tasks spanning edge-, vertex-, and face-level supervision. We combine intrinsic mesh descriptors with anatomy-aware priors, including PCA-derived frames for dental arches and liver surfaces, and augment message passing to provide lightweight global context. Across intracranial aneurysm and intraoral segmentation, EAMS variants are competitive with specialized baselines on unperturbed inputs while remaining stable under geometric perturbations, and on liver surfaces they expose a favorable trade-off between canonical-pose accuracy and rotation robustness. These results show that a lightweight ($<2$M parameters) equivariant framework can deliver robust anatomical mesh segmentation across diverse supervision types without task-specific architectures.

Cross-architecture/backbone medical-segmentation pools are commonly motivated as complementary failure coverage when members make different mistakes; that assumption is rarely tested at the per-case level. We audit this assumption for a fixed pool of 11 pretrained-encoder bundles read out by a deliberately weak $1\times1$-convolution decoder (4 seeds), and for a small set of decoder-recipe perturbations on the same bundles. Across five primary task rows (Kvasir polyp, ACDC LV, BraTS foreground, RIGA Cup, RIGA Disc), this 11-bundle $1\times1$-conv readout pool yields per-case Dice correlation gaps of $\Delta=0.261\text{--}0.638$ between same-bundle and cross-bundle pairs after a Dice $\geq 0.30$ functional floor; all paired case and hierarchical bootstrap intervals exclude zero. The result is stable under floor sweeps, ICC(A,1), leave-one-out, subject aggregation, quality-controlled pair regression, and cross-fitted item-difficulty residualisation. Decoder-recipe diagnostics show that richer readouts attenuate the gap (UNet-skip; case-identical nnU-Net on RIGA Cup: $0.473\to 0.0079$, separate-split $0.034\text{--}0.047$); the reported magnitudes are recipe-specific. Additional probes (pixel error, joint-failure lift, same-architecture-family ViT, same-family cross-checkpoint, same-backbone) locate the dependence structure but do not causally separate architecture, pretraining, and recipe. The claim is scoped to 2D frozen/light-adaptation encoder pools with this readout family and to retrospective benchmark audits. We release per-case Dice traces, summary JSONs, scorer code, hashes, and metadata as the audit artifact.

Echocardiography segmentation plays a central role in quantitative cardiovascular assessment. However, automated echocardiography segmentation remains challenging due to speckle noise, poorly defined anatomical boundaries, acquisition variability, and the limited anatomical coverage of existing benchmarks. Conventional segmentation methods demonstrate limited accuracy under such low image quality and domain shifts. Inspired by the observation that expert cardiologists can maintain reliable anatomical delineation by leveraging their structural understanding of the cardiac anatomy, we propose that incorporating anatomical priors into echocardiography segmentation models can improve both precision and robustness. To this end, we propose AST-Seg, an anatomy-aware spatio-temporal framework for echocardiography segmentation. AST-Seg introduces an anatomy-aware feature encoder that regularizes transformer attention with anatomy-derived spatial-relation priors, and a Deformable Spatio-Temporal Mamba module that captures localized and periodic cardiac motion through adaptive spatial aggregation followed by temporal state-space modeling. In addition, we curate an Echocardiography Anatomy (EA) dataset with pixel-level annotations for 12 clinically relevant cardiac anatomy, enabling comprehensive multi-structure cardiac interpretation. Extensive experiments on in-distribution CAMUS and EA datasets, as well as an out-of-distribution point-of-care echocardiography dataset, demonstrate that AST-Seg provides domain-generalized segmentation with improved precision.

Temporal action segmentation (TAS) is commonly trained on large banks of pre-extracted frame features rather than raw RGB video, but storing and replaying those features becomes a practical bottleneck for long-form video, especially in incremental TAS where past-task data must remain available to mitigate catastrophic forgetting. Recent TAS replay and condensation methods reduce this burden, but they rely on dense frame-wise annotations and allocate storage through fixed segment-level structures, limiting their flexibility for long procedural videos whose frame complexity varies substantially over time. We introduce Adaptive Residual-Quantized Variational AutoEncoder (ARQ-VAE), a compact and label-free memory representation for TAS features that learns a discrete residual quantization of frame-level video features and then adaptively truncates the residual chain on a per-frame basis, retaining only the residual prefix that best reconstructs each frame feature. This yields a practical compact feature memory that does not depend on frame-wise labels and can therefore support multiple TAS settings with the same stored representation, while a lightweight fine-tuning stage further improves reconstruction quality for the truncated codes that are actually retained. Across standard TAS benchmarks, ARQ-VAE achieves a substantially improved storage--performance trade-off over prior replay and condensation baselines. On Breakfast, our final representation reduces training-set storage from 28 GB for the original I3D feature bank to 10 MB while retaining strong downstream segmentation performance; with ASFormer, it achieves 71.5 Edit and 51.2 F1@50 in the supervised setting. We further show that the same label-free memory remains effective for incremental and unsupervised TAS, and transfers to an alternative pretrained feature space based on DINOv3. We position ARQ-VAE as a practical memory representation for TAS, and a promising direction for broader dataset condensation applications.

Spherical Transformers have emerged as a promising framework for panoramic semantic segmentation (PASS) by operating directly on spherical geometry and alleviating projection-induced distortions. However, existing architectures rely on assumptions of canonical spherical structure and stable viewpoints, which are frequently violated in real-world $360^\circ$ imagery due to unconstrained camera motion, introducing significant contextual and geometric ambiguity. Consequently, they lack adaptive mechanisms to model such ambiguity, limiting robustness to unseen spherical transformations. In contrast, biological perception is inherently ambiguity-aware: rather than estimating uncertainty probabilistically, it adapts to fluctuations in cue reliability caused by geometric and contextual variations, enabling stable interpretation under complex visual transformations. Motivated by these observations, we first present a systematic analysis of existing PASS architectures under various unseen spherical transformations. Following this, we introduce AdapToPASS, a novel, bio-inspired Spherical Transformer that adaptively models contextual and geometric ambiguities for robust panoramic semantic segmentation. At the core of AdapToPASS are the Adaptive Spherical Attention (AdaSpA) blocks that dynamically modulate attention according to local contextual ambiguity, mimicking the adaptive, context-driven perception of biological vision. To address geometric ambiguities, AdapToPASS employs Bifocal Spherical Representation that reconciles the trade-off between field of view and spatial resolution; along with boundary supervision to emulate the boundary-sensitive nature of biological vision. We evaluate AdapToPASS in both indoor and outdoor semantic segmentation, where it consistently outperforms prior state-of-the-art methods. We further validate AdapToPASS under unseen spherical transformations, where it demonstrates strong robustness and surpasses the next-best method by +13.38% relative improvement in mIoU on Stanford2D3D and +18.77% on WildPASS. Additionally, we introduce a lightweight variant, AdapToPASS-Tiny, with fewer than 2M parameters, which surpasses compact baselines while retaining robustness to spherical transformations.

Neural dynamics are a central feature of biological neural systems. In the visual cortex, the functional role of neural dynamics is poorly understood, particularly for static stimuli. The statistics of still natural images have long been used by models that successfully predict firing in the early visual cortex. Those models assume that neurons represent inferences about latent image features, but they often ignore the dynamics of said inference. Here, we test if inferential dynamics for images explain visual-cortical dynamics. To do so, we present a biologically plausible model of cortical computation that assumes the inference of features is coupled with inference about how those features are organized into meaningful parts, also known as segmentation. Given an input image, our model iterates between inferring the features and the segments, through recurrence, thus inducing dynamics in the feature representation. We make theoretical and exact predictions for the induced neural dynamics. Our model predicts both gain and variability in evoked responses to natural images, and we show that it captures classical single-neuron experimental findings such as gain dynamics and variability decay during stimulus presentation. The model also predicts that those metrics are more heterogeneous than previously thought, reflecting the complexity of inference for natural scenes, and that their population-level organization reflects the inferred segments. Finally, our analytical formulation provides a clear interpretation of those results. Insights from our normative model could generalize to segmentation problems in other experimental modalities, and help address the cost of inference in artificial networks.

Dataset distillation (DD) aims to compress large-scale datasets into compact synthetic sets while preserving training efficacy. However, existing studies mainly focus on image classification, leaving dense prediction tasks such as semantic segmentation largely underexplored. In this work, we identify three key challenges for segmentation DD: (*i*) long-tailed class imbalance, (*ii*) the need for strict pixel-wise alignment between images and dense labels, and (*iii*) the high computational cost of optimizing high-resolution data with complex models. To address these challenges, we propose $\mathbf{D^{3}S^{2}}$, a **D**iffusion-guided **D**ataset **D**istillation framework for **S**emantic **S**egmentation. Our method adopts a two-stage design. In **Class-Balanced Mask Selection**, we construct a representative mask set via a greedy strategy that prioritizes underrepresented classes. In **Diffusion-Guided Image Synthesis**, we employ a pretrained layout-to-image diffusion model to generate images conditioned on the selected masks, naturally ensuring spatial alignment. To further enhance the training utility of synthesized data, we introduce guided diffusion sampling with two complementary objectives: a segmentation-consistency loss for pixel-level alignment, and a class-wise feature matching loss for aligning per-class feature statistics across layers. Extensive experiments demonstrate the superiority of $D^{3}S^{2}$. Notably, at an extremely compression rate of 1\%, our method achieves 24.99\% and 35.49\% mIoU on ADE20K and COCO-Stuff with Mask2Former (Swin-S), outperforming random selection by 9.34\% and 5.70\%, respectively.