AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

arXiv

Predicted Cortex Is Not a Domain-General Prior: A Matched-Control Audit of Brain-Encoding Features for Video Memorability

Carson Rodrigues

Brain-encoding foundation models predict fMRI responses to video, audio and text well enough to win the Algonauts 2025 challenge. We ask whether their predicted responses, obtained with no scanner, are a useful feature lens for a human-behavior task: forecasting short-video memorability. Each clip is projected into TRIBE v2's predicted cortical space and scored by ridge regression against a matched control, the model's own V-JEPA2 visual backbone taken before the brain projection. The answer is dataset-dependent. Within Memento10k (499 clips) the backbone wins (Spearman 0.594 vs 0.544); within VideoMem (820 clips) the brain projection wins (0.415 vs 0.368). Because the claim is that the ordering reverses, we test the reversal itself: the dataset-by-representation interaction is +0.097, 95% CI [+0.032, +0.160], two-sided bootstrap p=0.001, and over 10 cross-validation seeds the datasets separate completely (0/10 seeds favor the brain projection on Memento10k, 10/10 on VideoMem). Cross-dataset transfer inherits the split: Memento10k->VideoMem the brain projection wins (+0.076); the reverse loses heavily (-0.311). The VideoMem advantage is not a sample-size artifact (it survives matched training size and PCA-then-ridge) and not mere compression (a compressed, heavily regularized or transfer-tuned backbone stays below it). Predicted-brain features thus carry a small but real memorability signal the backbone misses on one dataset and not the other: a dataset-specific representation, not a domain-general prior. A vision-orthogonal component (partial Spearman 0.19, permutation p=2.5e-4) localizes to ventral occipito-temporal cortex, and predicted BOLD dynamics add nothing beyond the time-average because 3-4 samples per clip cannot resolve the sub-second late memorability response. Our pre-specified within-dataset hypothesis returned NO-GO; the reversal is what survived.

arXiv

3D Scene-Adaptive Trajectory-Controllable Human Image Animation with Camera Movement

Deyin Liu, Jicheng Xu, Lin Yuanbo Wu, Xiaowei Zhao, Xiatian Zhu, ZHE JIN, Anjan Dutta

Human image animation, which aims to generate a video of a reference subject following a provided action sequence, has received increasing research interest. With the development of diffusion-based/flow-based video foundation models, existing animation works have began to upgrade the guidance information from 2D skeleton/pose to 3D modeling conditions. Despite achieving reasonable results, these approaches face challenges in synthesizing trajectory-controllable human motion within natural scene under changed camera views. In this work, we present a scene-adaptive human image animation framework that controls both human motion and camera trajectories within a reconstructed 3D environment for video generation. To achieve this, we first develop a ground-adaptive 3D motion retargeting approach to enable user-friendly motion trajectory control adapting to the changes of elevations of ground and orientations automatically. Then we design a viewpoint-adaptive latent fusion mechanism to inject point-cloud geometric priors through scene-visibility masking into the generative process, providing precise guidance of viewpoint changes under camera control. Experiments on two standard human image animation benchmark datasets demonstrate remarkable improvements of our method over the state of the arts in related video generation metics. Project page: https://robinhood256100.github.io/web-disp

arXiv

SAGE: An Expert-Annotated South Asian GI Endoscopy Dataset for Multimodal Learning and Hallucination Analysis

Niyoj Oli, Sachin Acharya, Sandesh Pokhrel, Sanjay Bhandari, Ramesh Rana, Nikesh Mani Shrestha, Ram Bahadur Gurung, Yash Raj Shrestha, Prashnna K Gyawali, Binod Bhattarai

Gastrointestinal cancers represent a growing health burden in the South Asian region, driven largely by rapid changes in socio-economic conditions and lifestyle habits. However, early diagnosis remains limited by inadequate equipment, financial resources, and scarce GI expertise. AI-assisted diagnosis and report generation show great promise in alleviating this problem by providing non-specialist healthcare workers the technical expertise to perform diagnosis. Yet, almost all open-source, publicly available datasets are predominantly collected from the European region, with minimal representation from the South Asian region. The lack of open-source GI datasets from diverse geographic regions has made it difficult to assess whether population bias is present in existing models, and to develop geographically inclusive AI tools for automated GI diagnosis. To address this gap, we introduce SAGE: An Expert-Annotated South Asian GI Endoscopy dataset for Multimodal Learning and Hallucination Analysis, for image captioning, multi-label classification, and VQA tasks. It consists of 1,300 images, their captions along with hallucination tag, 18 labels and 14,276 question-answer pairs making it well-suited for diverse range of tasks including classification, benchmarking, and fine-tuning LMMs. We further conducted benchmarking of task-specific models, like multi-class classifiers on the effect of population shift which reveals that such models suffers the most with an average F1 score drop of 54 points on South Asian dataset. Further, benchmarking of contemporary LMMs reveals a substantial drop in the average GREEN score for anatomical landmark detection (0.308) and abnormality detection (0.410). We open-source our dataset under the CC BY-SA 4.0 license, and we hope to encourage others to contribute toward more inclusive dataset representation and help counteract population bias in medical AI.

arXiv

V2VCrafter: Consistent Street-View Image Generation Across Vehicles

Yihang Tao, Yu Guo, Senkang Hu, Yanan Ma, Zihan Fang, Sam Kwong, Yuguang Fang

Connected and autonomous driving (CAD) systems leverage vehicle-to-vehicle (V2V) communication for multi-agent collaborative perception, yet remain constrained by scarce annotated real-world V2V datasets and limited generalization across diverse driving conditions. While image generation offers a feasible solution for data augmentation, existing single-vehicle multi-view generation frameworks face two key challenges in multi-agent settings: (1) the expanded learning objective degrades generation quality, and (2) dynamic inter-agent variation hinders consistency modeling for physical attributes (e.g., color, category) of jointly observed objects. To bridge this gap, we propose V2VCrafter, the first framework for generating controllable and realistic multi-view driving images across vehicles. For effective learning, we develop a progressive multi-agent diffusion model based on a single-agent backbone, using neighboring agents' latent states to progressively guide single-to-multi-agent generation. To address cross-vehicle inconsistency, we further propose a cross-agent attention module that leverages a collaboration view graph and learnable jointly observed object representations to model dynamic cross-vehicle camera view relationships. Experiments on real-world V2X-Real dataset show that V2VCrafter generates high-fidelity, controllable, and consistent street views across vehicles, thereby effectively enhancing downstream collaborative 3D object detection tasks.

arXiv

Semi-MedRef: Semi-Supervised Medical Referring Image Segmentation with Cross-Modal Alignment

Yuchen Li, Ziru Wei, Zhen Zhao, Yi Liu, Luping Zhou

Medical referring image segmentation (MRIS) predicts lesion masks from medical images and natural-language referring expressions, but acquiring paired pixel-level annotations and referring texts is costly. Semi-supervised learning (SSL) can alleviate this burden by exploiting unlabeled data, yet its effectiveness depends on preserving image--text alignment under strong perturbations. Existing SSL methods for referring segmentation rely on independent or simple multimodal perturbations (e.g., left--right flips), while stronger augmentations such as CutMix remain largely unexplored because they can disrupt cross-modal correspondence. We propose Semi-MedRef, a teacher--student SSL framework that explicitly preserves alignment between medical images and positional language through three complementary components: T-PatchMix, an alignment-preserving cross-modal augmentation that synchronizes patch mixing with positional-language and pseudo-mask updates; PosAug, a position-aware text augmentation that regularizes reliance on positional expressions; and Positional Affinity Contrastive Learning (PACL), which exploits coarse positional cues to construct region-aware supervision through anatomically weighted soft positives, encouraging anatomically grounded cross-modal representation learning. Experiments on QaTa-COV19 and MosMedData+ demonstrate that Semi-MedRef consistently outperforms state-of-the-art fully supervised and semi-supervised MRIS methods across all label regimes.

arXiv

AdaDINO: Context-Adaptive DINO-Distilled Vision Foundation Models for Efficient Open-Vocabulary Edge Inference

Yiwei Zhao, Yi Zheng, Huapeng Su, Jieyu Lin, Stefano Ambrogio, Cijo Jose, Michael Ramamonjisoa, Patrick Labatut, Barbara De Salvo, Chiao Liu, Phillip B Gibbons, Ziyun Li

Always-on contextual AI runs language-aligned vision foundation models (VFMs) on edge devices, where the on-device model is the dominant continuous compute cost under strict latency and power limits. Due to an observed low-frequency shift in scene context and its relevant vocabulary, we present AdaDINO, an adaptive framework that makes on-device VFM inference efficient by matching execution to the current scene and task. We build on a known phenomenon, that the accuracy drop of shrinking model sizes depends on the task, and turn it into task-level adaptive execution. AdaDINO integrates neural architecture search (NAS) into a language-aligned VFM backbone distilled from DINOv2, training a single family of subnets for efficient execution during runtime. A multimodal large language model (LLM) on the cloud, invoked at low frequency, refines the candidate class set from scene context, while a learned selector activates the least-cost subnet predicted to retain a target fraction of accuracy. With the backbone and semantic pipeline held fixed, learned selection alone reduces average compute by $37\%$ over the best fixed subnet at equal segmentation accuracy. Across zero-shot classification and open-vocabulary segmentation, AdaDINO establishes a strong accuracy-efficiency frontier, improving over evaluated models of comparable sizes by up to $7.9\%$ in acc@1 on IN1K and $5.2\%$ mIoU on ADE20K, and reducing average FLOPs by up to $74.9\%$ at similar accuracy.

arXiv

QShield: Securing Neural Networks Against Adversarial Attacks using Quantum Circuits

Navid Azimi, Aditya Prakash, Yao Wang, Li Xiong

Deep neural networks remain highly vulnerable to adversarial perturbations, limiting their reliability in security- and safety-critical applications. To address this challenge, we introduce QShield, a modular hybrid quantum-classical neural network (HQCNN) architecture designed to enhance the adversarial robustness of classical deep learning models. QShield integrates a conventional convolutional neural network (CNN) backbone for feature extraction with a quantum processing module that encodes the extracted features into quantum states, applies structured entanglement operations under realistic noise models, and outputs a hybrid prediction through a dynamically weighted fusion mechanism implemented via a lightweight multilayer perceptron (MLP). We systematically evaluate both classical and hybrid quantum-classical models on the MNIST, OrganAMNIST, and CIFAR-10 datasets, using a comprehensive set of robustness, efficiency, and computational performance metrics. Our results demonstrate that classical models are highly vulnerable to adversarial attacks, whereas the proposed hybrid models with entanglement patterns maintain high predictive accuracy while substantially reducing attack success rates across a wide range of adversarial attacks. Furthermore, the proposed hybrid architecture significantly increased the computational cost required to generate adversarial examples, thereby introducing an additional layer of defense. These findings indicate that the proposed modular hybrid architecture achieves a practical balance between predictive accuracy and adversarial robustness, positioning it as a promising approach for secure and reliable machine learning in sensitive and safety-critical applications.

arXiv

Rethinking IRSTD: Single-Point Supervision Guided Encoder-only Framework is Enough for Infrared Small Target Detection

Rixiang Ni, Boyang Li, Jun Chen, Yonghao Li, Feiyu Ren, Yuji Wang, Haoyang Yuan, Wujiao He, Wei An

Infrared small target detection (IRSTD) aims to separate small targets from clutter backgrounds. Extensive research is dedicated to the pixel-level supervision-guided "encoder-decoder" segmentation paradigm. Although having achieved promising performance, they neglect the fact that small targets only occupy a few pixels and are usually accompanied with blurred boundary caused by clutter backgrounds. Based on this observation, we argue that the first principle of IRSTD should be target localization instead of separating all target region accompanied with indistinguishable background noise. In this paper, we reformulate IRSTD as a centroid regression task and propose a novel Single-Point Supervision guided Infrared Probabilistic Response Encoding method (namely, SPIRE), which is indeed challenging due to the mismatch between reduced supervision network and equivalent output. Specifically, we first design a Point-Response Prior Supervision (PRPS), which transforms single-point annotations into probabilistic response map consistent with infrared point-target response characteristics, with a High-Resolution Probabilistic Encoder (HRPE) that enables encoder-only, end-to-end regression without decoder reconstruction. By preserving high-resolution features and increasing effective supervision density, SPIRE alleviates optimization instability under sparse target distributions. Finally, extensive experiments on various IRSTD benchmarks, including SIRST-UAVB and SIRST4 demonstrate that SPIRE achieves competitive target-level detection performance with consistently low false alarm rate (Fa) and significantly reduced computational cost. Code is publicly available at: https://github.com/NIRIXIANG/SPIRE-IRSTD.

arXiv

MMPhysVideo: Physically Plausible Video Generation Through Joint RGB-Perception Modeling

Shubo Lin, Xuanyang Zhang, Wei Cheng, Weiming Hu, Gang Yu, Jin Gao

Despite advancements in generating visually stunning content, video diffusion models (VDMs) often yield physically inconsistent results due to pixel-only reconstruction. To address this, we propose MMPhysVideo, the first study to enhance physical plausibility in video generation through joint multimodal modeling. We recast perceptual cues, specifically semantics, geometry, and spatio-temporal trajectories, into a unified pseudo-RGB format, enabling VDMs to directly capture complex physical dynamics. To mitigate cross-modal interference, we propose a Bidirectionally Controlled Teacher architecture, which utilizes parallel branches to fully decouple RGB and perception processing and adopts two zero-initialized control links to gradually establish pixel-wise consistency. For inference efficiency, the teacher's physical prior is distilled into a single-stream student model via representation alignment. Furthermore, we present MMPhysPipe, an end-to-end data curation and annotation pipeline tailored for constructing physics-rich multimodal datasets. MMPhysPipe employs a vision-language model (VLM) guided by a chain-of-visual-evidence rule to pinpoint physical subjects, enabling expert models to extract multi-granular perceptual information. Without additional inference costs, MMPhysVideo consistently improves physical plausibility of advanced models on the Videophy and PhyGenbench benchmarks and achieves superior performance among existing methods.

arXiv

Structured Observation Language for Efficient and Generalizable Vision-Language Navigation

Daojie Peng, Fulong Ma, Jun Ma

Vision-Language Navigation (VLN) requires an embodied agent to navigate complex environments by following natural language instructions, which typically demands tight fusion of visual and language modalities. Existing VLN methods often convert raw images into visual tokens or implicit features, requiring large-scale visual pre-training and suffering from poor generalization under environmental variations (e.g., lighting, texture). To address these issues, we propose SOL-Nav (Structured Observation Language for Navigation), a novel framework that translates egocentric visual observations into compact structured language descriptions for efficient and generalizable navigation. Specifically, we divide RGB-D images into a NxN grid, extract representative semantic, color, and depth information for each grid cell to form structured text, and concatenate this with the language instruction as pure language input to a pre-trained language model (PLM). Experimental results on standard VLN benchmarks (R2R, RxR) and real-world deployments demonstrate that SOL-Nav significantly reduces the model size and training data dependency, fully leverages the reasoning and representation capabilities of PLMs, and achieves strong generalization to unseen environments.

arXiv

SARe: Structure-Aware Generative 3D Fragment Reassembly

Hanze Jia, Chunshi Wang, yuxiao Yang, Zhonghua Jiang, Yawei Luo, Shuainan Ye, Tan Tang

3D fragment reassembly estimates the rigid pose of each fragment to recover a complete object from unordered point clouds or meshes. The task becomes increasingly challenging as the fragment count grows, since irregular fragments provide weak semantic cues and admit rapidly increasing numbers of plausible contact relations and global configurations. We propose Structure-Aware Reassembly (SARe), a generative framework that integrates query-aligned local geometry and task-native structural supervision into point-flow assembly. SARe-Gen conditions each transported surface query on a local latent and jointly supervises intermediate flow tokens with query-level fracture-region and fragment-level contact targets. Because these heads are optimized together with flow matching, structural supervision directly shapes the representations that drive coordinate transport, without requiring additional reassembly-specific pretraining or a separate teacher-alignment stage. At inference time, SARe-Refine geometrically verifies predicted relations and uses reliable local subassemblies to guide a second sampling pass, reinforcing consistent regions while resampling uncertain fragments. We evaluate SARe across three settings, including synthetic fractures, simulated fractures from scanned real objects, and scans of physically fractured objects. The results demonstrate state-of-the-art performance, with higher part accuracy and more graceful degradation in challenging many-fragment settings.

arXiv

MVHOI: Bridge Multi-view Condition to Complex Human-Object Interaction Video Reenactment via 3D Foundation Model

Jinguang Tong, Jinbo Wu, Kaisiyuan Wang, Zhelun Shen, Xuan Huang, Mochu Xiang, Xuesong Li, Yingying Li, Haocheng Feng, Chen Zhao, Hang Zhou, Wei He, Chuong Nguyen, Jingdong Wang, Hongdong Li

Human-Object Interaction (HOI) video reenactment aims to transfer the interaction dynamics of a source video to a novel target object while preserving realistic hand-object coordination. Existing methods typically rely on sparse 2D motion controls and monocular references, which are insufficient for complex out-of-plane motion and large viewpoint changes. We present MVHOI, a two-stage framework combining implicit motion extraction, 3D-aware multi-view reasoning, and video generation. In the first stage, a motion extractor encodes object dynamics into implicit motion descriptors. Conditioned on these descriptors, our Motion-Driven Object Prior (MDOP) module queries a 3D foundation model over multi-view references of the target object and autoregressively predicts coarse object anchors, a sequence of images that track the object's evolving orientation and appearance under the source motion without any explicit pose estimation. In the second stage, a DiT-based video generation model uses these anchors as structural guidance and the multi-view references as appearance guidance. We further reuse cross-view attention from MDOP as a soft attention bias to reduce reference-view confusion. For long videos, a cross-iterative inference strategy refreshes subsequent object priors using refined video outputs. Experiments demonstrate consistent improvements over state-of-the-art methods in object fidelity, motion consistency, visual quality, and interaction realism.

arXiv

DriveCode: Domain Specific Numerical Encoding for LLM-Based Autonomous Driving

Zhiye Wang, Yanbo Jiang, Rui Zhou, Bo Zhang, Fang Zhang, Zhenhua Xu, Yaqin Zhang, Jianqiang Wang

Large language models (LLMs) have shown great promise for autonomous driving. However, discretizing numbers into tokens limits precise numerical reasoning, fails to reflect the positional significance of digits in the training objective, and makes it difficult to achieve both decoding efficiency and numerical precision. These limitations affect both the processing of sensor measurements and the generation of precise control commands, creating a fundamental barrier for deploying LLM-based autonomous driving systems. In this paper, we introduce DriveCode, a novel numerical encoding method that represents numbers as dedicated embeddings rather than discrete text tokens. DriveCode employs a number projector to map numbers into the language model's hidden space, enabling seamless integration with visual and textual features in a unified multimodal sequence. Evaluated on OmniDrive, DriveGPT4, and DriveGPT4-V2 datasets, DriveCode demonstrates superior performance in trajectory prediction and control signal generation, confirming its effectiveness for LLM-based autonomous driving systems.

arXiv

From Representational Complementarity to Dual Systems: Synergizing VLM and Vision-Only Backbones for End-to-End Driving

Sining Ang, Yuguang Yang, Chenxu Dang, Canyu Chen, Cheng Chi, Haiyan Liu, Xuanyao Mao, Jason Bao, Xuliang, Bingchuan Sun, Yan Wang

Vision-Language-Action (VLA) driving augments end-to-end (E2E) planning with language-enabled visual backbones, yet it remains unclear how vision-language models (VLMs) differ from standard vision-only encoders and whether these differences survive policy learning. We study this question under a unified VLM-hidden + diffusion-policy paradigm, comparing multiple VLM families/scales (e.g., InternVL3 and Qwen3VL) with vision-only encoders (e.g., ResNet, ViT, and EVA-CLIP). We ask three questions: how similar are their representations, do residual differences induce meaningful behavioral differences, and how can they improve accuracy-cost trade-offs? We find that VLM and vision-only policies share a substantial common subspace after policy learning, yet both retain non-transferable residual subspaces. Using a Shared-Unique SAE, we show that these residual factors are behaviorally relevant: vision-only encoders are stronger in simple, geometry-dominant scenarios, whereas VLMs are stronger in long-tail, semantically complex, and interaction-heavy cases. The two policy families also exhibit distinct driving styles, with vision-only models being more conservative on average and VLMs more assertive. Exploiting the complementarity between a VLM branch and a ViT branch yields an oracle upper bound of 93.58 PDMS on NAVSIM. We introduce HybridDriveVLA, which runs both branches and uses a learned trajectory scorer for selection, improving PDMS to 92.10 (+1.30 over the VLM baseline), and DualDriveVLA, a fast-slow variant that invokes the VLM in only 15% of scenarios, achieving 91.00 PDMS (+0.20) with about a 1.9x latency speedup over the VLM baseline. Code will be released.

arXiv

Human-like working memory signatures emerge from intrinsically plastic artificial neurons for robust dynamic vision

Jingli Liu, Huannan Zheng, Bohao Zou, Kezhou Yang

While the unsustainable energy cost of artificial intelligence necessitates physics-driven computing, its performance superiority over full-precision GPUs remains a challenge. We bridge this gap by repurposing the Joule-heating relaxation dynamics of magnetic tunnel junctions, conventionally suppressed as noise, into neuronal intrinsic plasticity, realizing working memory with human-like features. Traditional AI utilizes energy-intensive digital memory that accumulates historical noise in dynamic environments. Conversely, our Intrinsic Plasticity Network (IPNet) leverages thermodynamic dissipation as a temporal filter. We provide direct system-level evidence that this physics-driven memory yields an 18x error reduction compared to spatiotemporal convolutional models in dynamic vision tasks, reducing memory-energy overhead by >90,000x. In autonomous driving, IPNet reduces prediction errors by 12.4% versus recurrent networks. This establishes a neuromorphic paradigm that shatters efficiency limits and surpasses conventional algorithmic performance.

arXiv

Seeing the Unseen: Towards Training-Free Inspection for Wind Turbine Blades Using Knowledge-Augmented Vision Language Models

Yang Zhang, Qianyu Zhou, Farhad Imani, Jiong Tang

Wind turbine blades operate in harsh environments, making timely damage detection essential for preventing failures and optimizing maintenance. Drone-based inspection and deep learning are promising, but typically depend on large, labeled datasets, which limit their ability to detect rare or evolving damage types. To address this, we propose a zero-shot-oriented inspection framework that integrates Retrieval-Augmented Generation (RAG) with Vision-Language Models (VLM). A multimodal knowledge base is constructed, comprising technical documentation, representative reference images, and domain-specific guidelines. A hybrid text-image retriever with keyword-aware reranking assembles the most relevant context to condition the VLM at inference, injecting domain knowledge without task-specific training. We evaluate the framework on 30 labeled blade images covering diverse damage categories. Although the dataset is small due to the difficulty of acquiring verified blade imagery, it covers multiple representative defect types. On this test set, the RAG-grounded VLM correctly classified all samples, whereas the same VLM without retrieval performed worse in both accuracy and precision. We further compare against open-vocabulary baselines and incorporate uncertainty Clopper-Pearson confidence intervals to account for the small-sample setting. Ablation studies indicate that the key advantage of the framework lies in explainability and generalizability: retrieved references ground the reasoning process and enable the detection of previously unseen defects by leveraging domain knowledge rather than relying solely on visual cues. This research contributes a data-efficient solution for industrial inspection that reduces dependence on extensive labeled datasets.

arXiv

DiffPhysCam: Differentiable Physics-Based Camera Simulation for Inverse Rendering and Embodied AI

Bo-Hsun Chen, Nevindu M. Batagoda, Dan Negrut

Generating synthetic images that closely mimic those from real cameras is instrumental in training visual models and enabling end-to-end visuomotor learning. We introduce DiffPhysCam, a differentiable camera simulator designed to support robotics and embodied AI applications by enabling gradient-based optimization in visual perception pipelines. Differentiable rendering also allows inverse reconstruction of real-world scenes as digital twins, facilitating simulation-based robotics training. Existing virtual cameras offer limited control over intrinsic settings, have difficulty capturing optical artifacts, and lack tunable calibration parameters. DiffPhysCam addresses these limitations through a multi-stage pipeline that provides fine-grained control over camera settings, models key optical effects such as defocus blur, and supports calibration with real-world data. It enables both forward rendering for image synthesis and inverse rendering for 3D scene reconstruction, including mesh and material texture optimization. We show that DiffPhysCam enhances robotic perception performance in synthetic image tasks. As an illustrative example, we create a digital twin of a real-world scene using inverse rendering and use it to set up a virtual experiment in a multi-physics simulation, in which we demonstrate navigation of an autonomous ground vehicle using images generated by DiffPhysCam. The code, data, and output results associated with this paper are available online for reproducibility at Chen, Bo-Hsun (2026a,b,c).

arXiv

Onboard Satellite Image Classification for Earth Observation: A Comparative Study of ViT Models

Thanh-Dung Le, Vu Nguyen Ha, Ti Ti Nguyen, Duc-Dung Tran, Hung Nguyen-Kha, Luis M. Garces-Socarras, Juan Carlos Merlano-Duncan, Symeon Chatzinotas

Remote sensing (RS) image classification is central to Earth observation, but onboard deployment requires models that are accurate, efficient, and robust to sensor and transmission degradation. Following a train-on-ground, infer-onboard workflow, we evaluate 14 backbones, including CNNs, ResNets, compact Transformers trained from scratch, and pre-trained Vision Transformers, on EuroSAT and PatternNet. We assess clean-data performance, computational cost, power consumption, and robustness to Gaussian noise, motion blur, and an end-to-end DVB-S2(X) transmission chain with channel impairments and JPEG compression. Pre-trained Vision Transformers generally outperform models trained from scratch while providing better efficiency and corruption resilience. MobileViTV2 achieves the highest clean EuroSAT accuracy at 99.09%, whereas EfficientViT-M2 provides the strongest overall trade-off. It attains 98.76% accuracy, precision, and recall on EuroSAT and 99.52% accuracy on PatternNet, with 203.53 MFLOPs, a 38.19 MB footprint, and the best overall robustness score of 0.79. It also degrades most gracefully under transmission loss and consumes 63.35% less power than MobileViTV2 and 73.33% less than Swin Transformer. These results identify EfficientViT-M2 as a strong backbone for reliable, energy-efficient onboard RS image classification. Code for data augmentation, corruption generation, training, and inference is publicly available.

arXiv

On the Viability of Semi-Supervised Segmentation Methods for Statistical Shape Modeling

Asma Khan, Tushar Kataria, Janmesh Ukey, Shireen Y. Elhabian

Statistical Shape Models (SSMs) excel at identifying population level anatomical variations, which is at the core of various clinical and biomedical applications, including morphology-based diagnostics and surgical planning. However, the effectiveness of SSMs is often constrained by the necessity for expert-driven manual segmentation, a time-intensive and expensive process that restricts their broader utility. While deep learning approaches offer a potential workaround by directly estimating SSMs from unsegmented images, they merely shift the burden. Although these models do not require segmentation during deployment, they still fail to address the challenge of acquiring the manual annotations needed for training, particularly in resource-limited settings. Semi-supervised models for anatomy segmentation present a logical solution to the annotation burden. However, the lack of established guidelines leaves end-users uncertain about the actual effectiveness of these approaches for the downstream task of constructing SSMs. In this study, we bridge this gap by systematically evaluating semi-supervised methods as viable alternatives to manual segmentation. By applying these methods under low-annotation settings and utilizing the predicted segmentations for SSM generation, we establish a comprehensive new performance benchmark. Our findings reveal a clear divide in performance: while certain methods yield noisy segmentations that degrade SSM quality, others accurately capture the population's modes of variation comparable to those obtained from manual-segmentation SSMs despite a 60-80% reduction in manual annotation requirements.

arXiv

Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity

Yongxi Zhou, Junwei Yao, Yuanzhe Liu, Zihan Dong, Wenbo Ye, Jiaxi Wen, Lai Yun Choi

A benchmark score is a measurement instrument, yet most benchmarks read each item at a single canonical surface form. We ask whether that reading is faithful: when an item's intent is held fixed and only its meaning-preserving surface form varies, does the canonical-form score estimate model behavior well, and how much of any variation is decoding/judge noise rather than signal? We instantiate this in safety, a high-stakes setting with no gold label to average toward. To avoid prior confounds, we pre-author the reformulations (refusal-free, mostly non-LLM: machine back-translation and a Matrix-Language-Frame code-switch generator) so an identical surface form reaches every model, score all responses with one human-anchored, vendor-neutral judge (Claude, kappa = 0.86 vs. human on unsafe compliance, stable across languages, cross-checked by GPT-4o), and verify intent preservation. On 370 seeds x 5 surface forms x 5 models, no single transformation is uniformly most dangerous (6 of 20 per-transformation McNemar tests survive correction, most protective). Yet evaluating only the canonical prompt underestimates unsafe compliance: the union of unsafe outcomes across forms exceeds even the worst single form by 3.3-12.9 pp, with bootstrap 95% CIs excluding zero for all five models, and 5-13% of seeds safe on canonical are unsafe under some reformulation -- above a zero stochasticity floor (canonical resampled five times at temperature 0 gives 0/370 new exposures). The size of this gap is model-dependent (largest on Gemini 2.5 Pro). One form recovers only ~53% of a model's observed unsafe surface and about three reach 85% -- a redundancy characterization of this form set, not of a defined population. A benign control (XSTest) suggests the instability is bidirectional, though the benign and harmful pools are not item-matched. We release the dataset, code, and per-response labels.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.