NeurIPS 2026 论文列表

Conference on Neural Information Processing Systems

录用类型

6,437篇论文

按官方录用论文列表统计,可能与会议公布的数字略有出入。

  • Oral
    991.5%
  • Spotlight
    2403.7%
  • Poster
    6,09894.7%

浏览全部论文

主题: 多模态大模型 · 1,366 篇论文 · 重置筛选

SD-LoRA: Training-Time Structural Distillation into LoRA for Few-Shot Vision-Language Adaptation

Mohammed Rahman Sherif Khan Mohammad, Ardhendu Behera, Sandip Pradhan, Swagat Kumar, Yonghuai Liu

展开摘要

Few-shot adaptation of vision--language models must balance two competing goals: learning fine-grained visual distinctions from very limited supervision, and preserving the efficiency that makes frozen CLIP-like models attractive for deployment. Existing lightweight adaptation methods, including prompt tuning, LoRA, and cache-based adapters, typically supervise global image embeddings and therefore underuse token-level structure. Conversely, structure-aware methods can exploit local evidence but often introduce additional inference-time computation. We propose SD-LoRA, a structural distillation framework that uses token-level graph reasoning only during training and absorbs its effect into a lightweight LoRA-adapted CLIP model. During training, a heterogeneous graph teacher models patch-patch, global-local, and visual-text relations over high-resolution ViT tokens. Rather than keeping this graph at test time, SD-LoRA transfers its structural bias into the shared CLIP-LoRA backbone through implicit distillation and Prototype Predictive Alignment, which aligns graph-conditioned representations with support-derived class prototypes. At inference, the entire graph teacher is discarded; prediction uses only a single CLIP-LoRA forward pass and a cache-based classifier. Across 11 few-shot benchmarks, SD-LoRA improves over a strong CLIP-LoRA baseline by 1.3/1.2/2.4/1.9/2.0% in the 1/2/4/8/16-shot settings, while adding no graph computation at test time. Further results on base-to-novel generalization, cross-dataset transfer, OOD robustness, and calibration show that training-time structural supervision improves not only accuracy but also transfer reliability.

Poster
SCOUT: Planning under Occlusion via Object-Centric World Model Rollouts

Amir Hossain Raj, Xuesu Xiao

展开摘要

Robots manipulating in cluttered spaces must often plan from partial views where foreground objects occlude the target, surrounding geometry, and usable free space. Reactive policies often miss these hidden constraints, while exhaustive search becomes computationally prohibitive in dense clutter. We present SCOUT, an occlusion-aware planning framework that bridges high-level deliberation with grounded physical rollouts. SCOUT leverages an object-centric scene representation to reconstruct hidden geometry and derive a relational planning state capturing blocker hierarchies and accessibility corridors. To navigate the vast action space in clutter, an LLM proposes structured action candidates which are then rigorously evaluated via Monte Carlo Tree Search (MCTS) using a learned action-conditioned world model. Evaluations on ‘Reveal’ and ‘Placement’ tasks show that SCOUT outperforms Vision-Language-Action (VLA) policies, embodied agents, and 3D world models, with significant gains in long-horizon tasks requiring reasoning over occluded geometry. We further validate the framework’s robustness through real-world robotic manipulation in a cluttered shelf environment.

Poster
SCG-HF: Semantic Consistency Grouping for Hierarchical Fusion in Video Emotion Recognition

Muci Li, Yujing Rao, He Li, Mang Ye

展开摘要

Video Emotion Recognition (VER) aims to identify and understand the emotional states of characters by analyzing visual and auditory information in videos. However, conventional rigid frame sampling strategies for long videos tend to fragment continuous emotional expressions. Moreover, existing fusion methods often fail to properly model temporal dependencies and may suffer from future information leakage. To address these limitations, we propose SCG-HF, a novel framework that preserves semantic consistency through adaptive grouping and hierarchical fusion. Specifically, we introduce Semantic Consistency Grouping (SCG), which dynamically segments videos into semantically coherent units based on feature similarity, thereby maintaining the integrity of emotional patterns. Furthermore, we design a three-level Hierarchical Fusion (HF) architecture to capture emotional dynamics at different temporal granularities: (i) frame-level refinement enhances subtle local emotional cues; (ii) segment-level history-guided fusion models temporal evolution across segments while strictly preventing future information leakage via attention; and (iii) sample-level contrastive alignment synchronizes audio and visual representations in a shared latent space. Extensive experiments on the VideoEmotion-8 and Ekman-6 benchmarks demonstrate that SCG-HF achieves state-of-the-art performance.

Poster
SCDBench: A Benchmark for LLM-Based Smart Contract Decompilers

Kaihua Qin, Dawn Song, Arthur Gervais

展开摘要

Smart contract decompilation aims to recover high-level source code from bytecode, but evaluating decompilers remains difficult because existing studies use narrow datasets, inconsistent metrics, and limited semantic consistency checks. This gap is increasingly important as large language models (LLMs) begin to generate source-like Solidity that may compile and appear plausible, even when its semantics diverge from the original contract. We introduce SCDBench, a dataset and benchmark methodology for LLM-based smart contract decompilation. The dataset contains $600$ real-world Solidity contracts with paired bytecode inputs, ground-truth source code, and replayable semantic checkpoints. SCDBench evaluates decompiler outputs through four cumulative stages: format completeness, compilability, Application Binary Interface (ABI) recovery, and semantic consistency via differential replay. We evaluate Claude Opus 4.7, GPT-5.3-Codex, and GLM-5 in a zero-shot decompilation setting, including GLM-5 variants with and without extended reasoning and a zero-shot compilation-repair setting. The results show that frontier LLMs can often produce structured and compilable Solidity, but achieving semantic consistency remains far from solved: the best-performing frontier model perfectly decompiles only $42/600$ contracts. We further show that introducing same-model compilation repair substantially improves performance at modest additional cost. SCDBench establishes a common ground for rigorous, reproducible evaluation and aims to accelerate the development of reliable smart contract decompilers for blockchain security and transparency.

Poster
SAVE: Sparsity-Aware Influence Estimation for Vocabulary-Expanded LLMs

Seungyoo Lee, Giung Nam, Seanie Lee, Hyungi Lee, Juho Lee

展开摘要

Large language models adapted to specialized targets such as Lean theorem proving or tool calling often benefit from extending the tokenizer with target-specific tokens. However, target-only vocabulary-expanded fine-tuning can leave the newly added embeddings transfer-incomplete, since the new rows receive gradients only from examples containing the corresponding tokens, so target-local gains do not necessarily imply that the embeddings are integrated with related capabilities. We propose SAVE, a sparsity-aware influence estimator for selecting auxiliary healing data from generic corpora. SAVE computes token-conditional curvature over the effective update set of each new vocabulary row and combines this vocabulary-aware signal with dense LoRA-level influence. Across Lean autoformalization and tool calling, SAVE improves target performance and related transfer while remaining competitive on broad retention benchmarks.

Poster
SAMoR: Motion Modelling for Articulated Objects of Any Skeleton and Topology

Yuhao Zhang, Gerard Pons-Moll, Tolga Birdal

展开摘要

Modeling motion for articulated objects of arbitrary skeleton topology remains difficult: existing motion generators target a fixed human skeleton, and prior adaptations either fail to share a vocabulary across rigs or discard motion detail through global pooling. Our key observation is that while joint-level motion does not correspond cleanly across species, motion of functional joint groups does---a human arm, a wolf foreleg, and a bird wing share semantic motion structure despite differences in joint count and connectivity, a correspondence that joint names (e.g., ``forearm'', ``wing\_L1'') partially expose even when topology does not. We introduce SAMoR (Skeleton-Aware Motion Representation for Articulated Objects), a cross-topology motion representation that encodes each motion segment as a small fixed number ($K{=}8$) of part tokens shared across arbitrary skeletons. SAMoR takes three input signals---per-joint motion features, kinematic graph structure, and joint-name embeddings---and processes them with a graph-transformer encoder, then compresses the resulting heterogeneous per-joint features into part-level tokens via cross-attention pooling and residual vector quantization, yielding a discrete motion codebook shared across rigs. To prevent the part queries from collapsing into redundant global representations, we introduce a topology-agnostic attention supervision loss, combined with random joint-name dropout to prevent over-reliance on text labels; together these encourage the part tokens to cluster joints into functional groups from names, structure, and motion jointly. We curate a unified heterogeneous motion corpus from HumanML3D, Truebones Zoo, and animated Objaverse-XL assets, and evaluate SAMoR on held-out characters with unseen skeletons. The resulting representation supports accurate reconstruction and cross-topology motion transfer, and further enables text-conditioned generation and localized part-wise editing through a MaskGIT token generator. SAMoR reaches $2.75\!\times\!10^{-2}$ normalized MPJPE on cross-topology reconstruction---$5.8\!\times$ below the strongest adapted variable-$J$ tokenizer baseline---and remains competitive with fixed-skeleton specialists on HumanML3D for both VQ-VAE reconstruction and text-to-motion generation.

Poster
SALART-VQA: Diagnosing Whether VLMs Understand Salient Artifacts in Generated Images

Xiaoxiao Sun, Ruotian Zhang, JUNZHE HUANG, James Burgess, Serena Yeung-Levy

展开摘要

Vision-language models (VLMs) are increasingly used to detect whether AI-generated images contain visible artifacts, yet their ability to analyze such artifacts remains poorly understood. A correct image-level decision can still hide important failures: a model may correctly flag an artifact while relying on the wrong visual cue, selecting the wrong region, or describing a defect that the image does not support. To evaluate these behaviors directly, we introduce SalArt-VQA, a diagnostic benchmark for fine-grained salient artifact understanding in AI-generated images. SalArt-VQA contains 950 images and 3,681 human-authored multiple-choice questions spanning artifact images, matched real reference images, and paired generated reference images. Four aligned question types evaluate presence detection, semantic localization, spatial grounding, and evidence-grounded defect identification, while the reference splits test calibration and abstention when the annotated defect is absent. Across 20 VLMs, SalArt-VQA reveals failures that image-level detection accuracy hides: the strongest model reaches 99.37% detection recall on artifact images but answers all four artifact-side questions correctly on only 53.26% of images. Comparing artifact images with defect-absent references reveals a sensitivity–calibration tradeoff: sensitive models often make unsupported artifact claims, while conservative models avoid false alarms largely by missing real artifacts. These results show that high artifact detection accuracy alone does not imply grounded artifact understanding. SalArt-VQA exposes these hidden failure modes and provides a fine-grained evaluation of whether VLM artifact claims are supported by local visual evidence.

Oral
SAGE: Semantic-Agnostic Image Embedding for Generalized AI-Generated Image Detection

Seongho Kim, Jaehyun Choi, Dahye Kim, Sungwon Yi, Jang-Ho Choi

展开摘要

AI-generated image detection has become an important problem in media forensics as modern generative models produce increasingly realistic images. Recent CLIP-based detectors show strong generalization ability, but CLIP image features entangle semantic content with forensic cues. Since semantic content is not an intrinsic generation trace, relying on it can introduce semantic shortcuts and unstable performance under unseen generator shifts. In this paper, we propose SAGE, a Semantic-Agnostic Image Embedding framework for generalized AI-generated image detection. SAGE constructs paired triplets of COCO images, SDXL-reconstructed images, and COCO captions so that real and fake images share the same caption-level semantics. It then suppresses semantic information by suppressing the semantic direction provided by CLIP text embeddings from CLIP image embeddings. To enable caption-free inference, SAGE learns a semantic anchor that approximates the role of paired text embeddings at test time. In addition, SAGE adopts a SoftTriple-style sub-center classifier to model intra-class variation in real and fake images, yielding a more stable authentication geometry across diverse generators. Experiments on CommunityForensics demonstrate that SAGE achieves competitive detection performance while substantially reducing generator-wise performance variation. Further analyses show that SAGE effectively reduces semantic dependence and forms a clearer real/fake feature geometry than strong baselines.

Poster
SAGE: Mitigating Long-Horizon Reasoning Biases via Topological Guidance

Xinyue Zeng, Jiawei Zhang, Yujun Yan, Dawei Zhou

展开摘要

Long-horizon reasoning remains a central challenge for large language models (LLMs) under sparse-reward regimes. We argue that this brittleness arises from two biases induced by complex reasoning spaces: an exploration bias, where models are drawn toward locally plausible but structurally unstable branches, and a compounding bias, where small local deviations accumulate across depth and suppress rare rewards. We introduce Symbolic Closure Analysis (SCA) as a theoretical lens characterizing how branching structures and sparse rewards induce these biases in long-horizon reasoning with local admissibility, and as a design principle for structural priors in less formal reasoning tasks. Motivated by this analysis, we propose \model{} (\textbf{S}tructural \textbf{A}dmissibility-\textbf{G}uided \textbf{E}xploration), a unified framework that injects structural guidance to alleviate exploration bias and compounding bias in long-horizon reasoning. \model{} combines two complementary structural dimensions: algebraic sparsification, which projects locally admissible candidates onto operator-indexed algebraic subspaces to suppress spurious branching and mitigate exploration bias, and hyperbolic structural guidance, which embeds reasoning states into a negatively curved space to provide dense depth-wise signals and mitigate compounding bias. Across 12 benchmarks and 7 model families, \model{} outperforms competitive baselines. In particular, \model{} achieves up to an 8-fold improvement on the Andrews-Curtis problem, an open real-world long-horizon task. Code is available at: \url{https://anonymous.4open.science/r/SAGE-Long-Horizon-Reasoning-AD70}.

Poster
SAFTAC: Simulation-Augmented Fine-Tuning of Open-Source LLMs for Analog Circuit Design

Junsheng Huang, Yifan Sun, Zhuoer Zhang, Ning Wei, Yening Liu, Michael Molter, Pavan Kumar Hanumolu, Elyse Rosenbaum, Bin Hu, Huan Zhang

展开摘要

Circuit design for analog integrated circuits (ICs) is challenging because circuit topology, device sizing, and circuit performance are tightly coupled. Existing large language model (LLM)-based methods either rely on inference-time correction around frozen models or train on limited circuit types without directly using simulation outcomes as training signals. A key bottleneck is that existing datasets do not provide sufficient task information for training LLMs on end-to-end specification-conditioned design. To fill this gap, we first construct a large-scale simulation-grounded dataset with 8,626 design tasks across 10 different types of analog circuits, where each task includes a textual description, labeled I/O ports, loading condition, target specifications, a testbench, and a reference netlist. Using this dataset, we present SAFTAC, a simulation-augmented fine-tuning framework that trains LLMs to generate and revise analog circuit design. SAFTAC combines three-stage supervised fine-tuning, which progressively builds the capabilities required for analog circuit design and feedback-guided correction, with simulation feedback-augmented reinforcement fine-tuning, which further optimizes models using simulation-grounded rewards. We evaluate SAFTAC under single-pass generation and two-pass generation with simulation feedback, showing improvements over the strongest closed-source LLM by 15.9\% and 7.8\%, respectively.

Poster
SAFE-Hair: Scalp-Anchored Fields for Exportable Single-View Hair Reconstruction

Yucheng Wang, Zedong Wang, Yuetong Wu, Yue Ma, Dan Xu

展开摘要

Single-view 3D hair reconstruction has progressed rapidly, but many methods optimize for visual plausibility rather than producing strand assets that can be directly used in grooming, editing, or simulation pipelines. We propose SAFE-Hair, a framework that reconstructs explicit strand grooms from a single portrait by decoding hair from persistent follicles in a canonical scalp-UV domain. Given a fitted template head, SAFE-Hair projects self-supervised image features onto the scalp, uses a conditional Rectified Flow model to generate latent scalp fields, and decodes occupancy, segment directions, length allocation, and total length at a fixed set of scalp follicles. This representation enforces scalp-rooted strand identities by construction and enables direct export with follicle IDs and guide-family metadata. We further introduce attachment, collision, and bending priors to improve geometric validity without per-subject test-time optimization. To evaluate both reconstruction fidelity and asset usability, we introduce HairBench-3D, a benchmark built from public 3D hair assets with standardized alignment, rendering, resampling, and metrics for geometry, direction consistency, invisible-region completion, silhouette agreement, and collision. SAFE-Hair improves Chamfer distance from 3.74 mm to 3.28 mm, F-score from 77.9 to 81.7, invisible-region F-score from 66.1 to 69.8, and penetration ratio from 4.8% to 3.9% over the strongest baseline.

Poster
S2D: Sparse-To-Dense Keymask Distillation for Unsupervised Video Instance Segmentation

Leon Sick, Lukas Hoyer, Dominik Engel, Pedro Hermosilla, Timo Ropinski

展开摘要

In recent years, the state-of-the-art in unsupervised video instance segmentation has heavily relied on synthetic video data, generated from object-centric image datasets such as ImageNet. However, video synthesis by artificially shifting and scaling image instance masks fails to accurately model realistic instance motion in videos, such as perspective changes, movement by parts of one or multiple instances, or camera motion. To tackle this issue, we propose an unsupervised video instance segmentation model trained exclusively on real video data. We start from unsupervised instance segmentation masks on individual video frames. However, single-frame segmentations exhibit temporal noise and their quality varies through the video. Therefore, we establish temporal coherence by identifying high-quality keymasks in the video by leveraging deep motion priors. The sparse keymask pseudo-annotations are then used to train a segmentation model for implicit mask propagation, for which we propose a Sparse-To-Dense Distillation approach aided by a Temporal DropLoss. After training the final model on the resulting dense labelset, our approach outperforms the current state-of-the-art across various benchmarks.

Poster
S-EDL: Eliciting Self-Evidence from Sequence Likelihoods for Semantic Calibration of LLMs

Yawei Li, Jiazheng Li, David Rügamer, Bernd Bischl, Yulan He, Mina Rezaei

展开摘要

Calibrating large language models (LLMs) in open-ended generation is uniquely challenging because many distinct token sequences can express the same underlying meaning. Prior calibration techniques target either fixed-set classification confidence or the binary correctness of individual strings, making them inherently ill-suited for the dynamic, multi-string nature of open-ended semantic outcomes. To bridge this gap, we introduce S-EDL, a novel evidential method that natively optimizes LLMs for semantic calibration. By eliciting evidence from the sequence likelihoods of the model itself, S-EDL constructs a differentiable Dirichlet prior over prompt-specific semantic classes. Optimizing the evidential loss under this prior yields a calibration-aware training signal over variable semantic classes, completely bypassing the need for fixed multiple-choice options. Evaluations across four open-ended benchmarks and three models demonstrate that S-EDL substantially reduces semantic calibration error while improving accuracy. By aligning generative likelihoods with semantic correctness, this work establishes a framework for deploying trustworthy LLMs in complex, open-ended applications.

Poster
RxnOptBench: Benchmarking LLMs for Reaction-Condition Optimization in Organic Methodology

Lingli Ge, Junyuan Gao, Jiahe Song, Jiaxing Sun, Yubin Wang, Boyu Zhu, Haote Yang, Jingchao Wang, Lixin Ma, Jiang Wu, Yuqiang Li, Conghui He

展开摘要

Chemical reaction-condition optimization --- choosing the catalyst, ligand, solvent, reagent, temperature, time, and atmosphere that jointly maximize yield and stereoselectivity --- is a central, judgement-laden subtask of organic methodology research that large language models are increasingly expected to support. Yet existing chemistry benchmarks evaluate reaction-class labelling, retrosynthesis, or SMILES manipulation, and do not ask models to read a real condition-screening table and pick the best set. We introduce RxnOptBench, a benchmark whose every option and precedent is a real wet-lab entry mined from the optimization tables of organic-methodology papers published in 2025, graded by a continuous relative score that fuses yield with stereoselectivity (ee, dr, rr) so near-correct answers are not collapsed to zero, and equipped with a paired precedents-vs-no-precedents design that isolates in-context use of literature evidence from parametric memorization. Across nine frontier LLMs and three Chemistry LLMs, even the best models leave substantial headroom: chemistry-specialized models fall to the random-baseline floor on multi-axis selection, while open-weight models have closed most of the gap to proprietary frontier models. We release the benchmark, the human-verified extraction pipeline, and the inference and evaluation code.

Poster
Routeability Before Routing: Routeability Audit Protocol (RAP) and RouteabilityBench for Audited LLM Model Selection

Yihang Lu, Denica Kjorvezir, Ana Gjorgjevikj, Carola Doerr, Tome Eftimov

展开摘要

LLM routing aims to choose, for each prompt, which model in a portfolio is most likely to answer correctly. This only makes sense when the benchmark contains enough prompt-specific variation for different models to be best on different inputs. On coarse-scored LLM benchmarks, however, many models often tie for the best score, and the single best model may already leave little room for improvement. We therefore propose a routeability-first audit, instantiated as RAP and RouteabilityBench, a reviewer-runnable evaluation package. Across four benchmarks and 66 LLMs, generic routing enhancements such as larger embedding sets, dimensionality reduction, selective abstention, clustering, and stronger regressors do not consistently outperform the single best model. The clearest positive evidence is concentrated in portfolio-dependent Omni-MATH settings, while GPQA, MMLU-Pro, and IFEval show different ways in which routing can remain hard or predictor-sensitive. Our practical recommendation is that routing gains should be reported together with routeability diagnostics, tied-best analysis, portfolio construction details, and family-corrected statistical tests.

Poster
RotVLA: Rotational Latent Action for Vision-Language-Action Model

Qiwei Li, Xicheng Gong, Xinghang Li, Peiyan Li, Quanyun Zhou, Hangjun Ye, Jiahuan Zhou, Yadong Mu

展开摘要

Latent Action Models (LAMs) have emerged as an effective paradigm for handling heterogeneous datasets during Vision-Language-Action (VLA) model pretraining, offering a unified action space across embodiments. However, existing LAMs often rely on discrete quantization encode and decode pipelines, which can lead to trivial frame reconstruction behavior, limited representational capacity, and a lack of physically meaningful structure. We introduce RotVLA, a VLA framework built on a continuous rotational latent action representation. Latent actions are modeled as elements of ${\rm SO}(n)$, providing continuity, compositionality, and structured geometry aligned with real-world action dynamics. A triplet frame learning framework further enforces meaningful temporal dynamics while avoiding degeneration. RotVLA consists of a VLM backbone and a flow-matching action head, pretrained on large-scale cross-embodiment robotic datasets and human videos with latent-action supervision. For downstream robot control, the flow-matching head is extended into a unified action expert that jointly denoises latent and robot actions. Here, latent actions serve as a latent planner, providing high-level guidance that conditions action generation. With only 1.7B parameters and 1700+ hours of pretraining data, RotVLA achieves 98.2\% on LIBERO and 89.6\% / 88.5\% on RoboTwin2.0 under clean and randomized settings, respectively. It also demonstrates strong real-world performance on manipulation tasks, consistently outperforming existing VLA models.

Poster
Robustness of Internal Signals for Hallucination Detection in Vision-Language Models

Allison Chen, William Yang, Salma Abdel Magid, Jonathan Williams, Olga Russakovsky, Esin Tureci

展开摘要

When given an image and a text prompt, vision-language models (VLMs) can produce seemingly sophisticated text responses. However, they are also prone to hallucinate objects in their output, which can have detrimental consequences in safety-critical applications. To detect these hallucinations, certain methods interestingly leverage internal signals from the VLM as it generates the text response; potentially, these methods use the same signals that led to a hallucination to detect it. We conduct an in-depth analysis to understand the robustness of these detection methods. First, we show that a naive baseline of a generated word's \emph{token position} in the VLM text response achieves competitive performance. Further, we reveal how some detection methods \emph{rely} on token position and use this heuristic as a shortcut. Second, we explore detection methods' sensitivity to language and vision modalities. Our experiments suggest that strong detection performance is often driven by detecting language-based hallucinations, and when methods rely on token position, they tend to be less visually grounded and exhibit reduced robustness when hallucinations occur earlier in a text response. Lastly, we demonstrate that different detection methods can carry complementary signals; when ensembling methods together, adding weaker methods to an ensemble of higher performing ones still offers performance improvements. Altogether, while there may be useful internal VLM signals to detect hallucinations, our investigation reveals important limitations of the current methods that try to leverage them.

Poster
Robust and Efficient Finetuning of Vision Foundation Models via Implicit Ensembling

Masih Aminbeidokhti, Heitor Medeiros, Eric Granger, Marco Pedersoli

展开摘要

Robust finetuning of foundation models seeks to improve out-of-distribution (OOD) generalization during in-distribution (ID) adaptation. We revisit Mixout, a stochastic regularizer that intermittently replaces finetuned parameters with a reference anchor, and study its effectiveness for robust finetuning of vision foundation models. We reinterpret Mixout as a single-run, weight-sharing implicit ensemble and analyze its expected OOD error through a bias–variance–covariance–locality (BVCL) lens. This perspective reveals three key factors that govern the ID–OOD trade-off: the choice of masking anchor, the mask resampling frequency, and mask sparsity. Guided by this analysis, we propose GMixout, which generalizes Mixout in three ways. First, it explicitly controls the masking period through a resampling-frequency hyperparameter. Second, it replaces the fixed pretrained anchor with an exponential moving average snapshot that adapts throughout training. Third, it uses sparse kernels to update only a small subset of parameters at each iteration, introducing no inference-time overhead and enabling large-model finetuning on consumer-grade GPUs. Experiments across five vision benchmarks show that GMixout consistently outperforms Mixout, Model Soups, and strong parameter-efficient finetuning baselines under both covariate shift and class imbalance.

Poster
Robust Task-Aware State Estimation from Pre-trained Perception

Guy Azran, Sarah Keren

展开摘要

Agents that perform state estimation using pretrained perception models, e.g., Vision-Language Models (VLMs), face the fundamental challenge that predictions are uncertain. Still, most approaches determinize these predictions into a single best-guess state and make decisions as if it were ground truth. This is brittle because when the guess is wrong, the agent's decisions may be unsafe or infeasible. At the other extreme, maintaining a full distribution over symbolic states is often intractable. We propose a middle path, drawing on the underlying probabilities the perception model already produces (e.g., a VLM's output-layer token distributions) to construct a θ-confidence Minimal Belief Set (MBSθ), a minimal subset of states whose cumulative probability meets a user-defined threshold. The minimality of an MBSθ ensures that the agent considers only the most likely states, enhancing robustness in downstream decision-making while maintaining computational tractability. Using a novel three-layered belief system, we efficiently approximate individual state probabilities from per-feature predictions and task-level constraints, on which we base our MBSθ construction, while maintaining only a lean factored representation of the belief. Our algorithm, PRaVaSS, takes advantage of the unique structure of this belief system to construct an MBSθ with correctness guarantees and polynomial complexity. Using a VLM as the backbone of our state estimator, we validate our method's theoretical properties, scalability with problem size, and improvement of downstream decision-making when used in a task planning pipeline.

Poster
RoboExo: Structure-Guided Wrist-to-Exocentric Video Generation for Scalable Robot Learning

Rui Li, Zixuan Hu, Chenxi Li, Zhangrui Zhao, Fucheng Cai, Li Kang, Minting Pan, LINGYU DUAN, Dongzhan Zhou, Wangmeng Zuo

展开摘要

Scaling Vision-Language-Action (VLA) policy training requires diverse robot demonstration data, yet collecting such data remains expensive and labor-intensive. The community has explored ways to accelerate robot data collection: visual augmentation methods synthesize novel samples by editing visual elements in existing demonstrations, while systems such as UMI leverage wrist-mounted cameras to flexibly collect in-the-wild manipulation videos. However, the mismatched observation setups of these two directions leave them largely disconnected, limiting their synergistic potential for scalable robot data generation. To bridge this gap, we propose **RoboExo**, a generative framework for controllable wrist-to-exocentric conversion of robotic demonstrations. We introduce a geometry-motion factorization strategy that reconstructs canonical object-robot geometry and propagates motion states from noisy wrist-view videos, yielding reliable structural priors for cross-view synthesis. These priors guide a conditional video diffusion model to generate target-view exocentric videos that are spatially aligned, temporally coherent, and faithful to the underlying manipulation semantics. We evaluate RoboExo on a wrist-to-exo generation benchmark under both seen and unseen settings, where it consistently outperforms all baselines, reducing LPIPS by **76%** and FVD by **64\%**. Beyond generation quality, RoboExo improves VLA policy learning across six tasks, increasing the average success rate by **+30\%** with generated exocentric views. These results demonstrate RoboExo as an effective and scalable data engine for enriching low-cost UMI demonstrations. The code will be publicly available.

Poster