paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

arXiv

Reliable Inference in Edge-Cloud Model Cascades via Conformal Alignment

Jiayi Huang, Sangwoo Park, Nicola Paoletti, Osvaldo Simeone

Edge intelligence enables low-latency inference via compact on-device models, but assuring reliability remains challenging. We study edge-cloud cascades that must preserve conditional coverage: whenever the edge returns a prediction set, it should contain the true label with a user-specified probability, as if produced by the cloud model. We formalize conditional coverage with respect to the cloud predictive distribution, and introduce a conformal alignment-based (CAb) cascading mechanism that certifies this property with user control over the risk level. Our method casts escalation from edge to cloud models as a multiple-hypothesis testing (MHT) problem, tailoring conformal alignment (CA) to select which inputs can be safely handled at the edge. The proposed CAb model cascading method yields statistical guarantees on the average fraction of edge decisions that satisfy cloud-level conditional coverage. The procedure applies to arbitrary edge prediction sets, including variants of conformal prediction (CP), and exposes a tunable trade-off among coverage, deferral rate, and set size. Experiments on CIFAR-100 image classification and the TeleQnA question-answering (QA) benchmark show that the proposed CAb cascade maintains the target conditional coverage for edge predictions while substantially reducing offloading to the cloud and incurring modest increases in prediction-set size.

arXiv

Adaptive Online Learning with LSTM Networks for Energy Price Prediction

Salih Salihoglu, Ibrahim Ahmed, Afshin Asadi

Accurate prediction of electricity prices is crucial for stakeholders in the energy market, particularly for grid operators, energy producers, and consumers. This study focuses on developing a predictive model leveraging Long Short-Term Memory (LSTM) networks to forecast day-ahead electricity prices in the California energy market. The model incorporates a variety of features, including historical price data, weather conditions, and the energy generation mix. A novel custom loss function that integrates Mean Absolute Error (MAE), Jensen-Shannon Divergence (JSD), and a smoothness penalty is introduced to enhance the prediction accuracy and interpretability. Additionally, an adaptive online learning framework is implemented to allow the model to adapt to new data incrementally, ensuring continuous relevance and accuracy. The results demonstrate that the custom loss function can improve the model's performance, aligning predicted prices more closely with actual values, particularly during peak intervals. The adaptive online learning framework achieves the best overall performance, reducing MSE, MAE, and RMSE by approximately 23.0%, 3.4%, and 12.2%, respectively, compared with the next-best model, namely the static model, which is trained once and kept fixed during testing. The inclusion of the energy generation mix further enhances the model's predictive capabilities, highlighting the importance of comprehensive feature integration. This research provides a robust framework for electricity price forecasting, which offers valuable insights and tools for better decision-making in dynamic electricity markets.

arXiv

Variational Mixture of Graph Neural Experts for Alzheimer's Disease Recognition across Frequency Bands in EEG Brain Networks

Jun-En Ding, Anna Zilverstand, Shihao Yang, Albert Chih-Chieh Yang, Feng Liu

Dementia disorders such as Alzheimer's disease (AD) and frontotemporal dementia (FTD) exhibit overlapping electrophysiological signatures in electroencephalography (EEG) that challenge accurate diagnosis. Existing EEG-based methods are limited by full-band frequency analysis, which hinders precise differentiation of dementia subtypes and severity stages. To address this limitation, we propose a Variational Mixture of Graph Neural Experts (VMoGE) framework that integrates multi-band EEG analysis with variational graph neural networks and a mixture-of-experts architecture. Each expert specializes in a specific EEG frequency band and models brain connectivity using a Gaussian Markov Random Field prior, while a variational gating mechanism adaptively integrates expert outputs. This design enables the model to learn frequency-specific brain network representations while modeling latent uncertainty through variational inference. Experimental results on two EEG dementia datasets show that VMoGE achieves strong performance, with an area under the curve (AUC) of 0.89 for healthy controls (HC) vs. AD classification in the main comparison and competitive results across dementia subtyping and Clinical Dementia Rating (CDR) staging tasks. Clinically, VMoGE offers three key translational values: the expert gating weights correlate with Mini-Mental State Examination (MMSE) scores and CDR severity, slow-wave $δ/ θ$-band contributions are associated with AD-related EEG slowing and disease progression, and spatially localized activation maps reveal posterior $θ$/$α$-band alterations and region-specific $β$-band changes, providing neurophysiologically interpretable patterns aligned with known AD neuropathology.

arXiv

Patch-based Memory Gate Model in Time Series Foundation Model

Samuel Yoon, Jongwon Kim, Juyoung Ha, Young Myoung Ko

Recently reconstruction-based deep models have been widely used for time series anomaly detection, but as their capacity and generalization capability increase, these models tend to over-generalize, often reconstructing unseen anomalies accurately. Prior works have attempted to mitigate this by incorporating a memory architecture that stores prototypes of normal patterns. Nevertheless, these approaches suffer from high training costs and have yet to be effectively integrated with time series foundation models (TSFMs). To address these challenges, we propose MOMEMTO, an improved TSFM variant for anomaly detection, enhanced with a patch-based memory module to mitigate over-generalization. The memory module is designed to capture representative normal patterns from multiple domains and enables a single model to be jointly fine-tuned across these domains through a multi-domain training strategy. MOMEMTO initializes memory items with latent representations from a pre-trained encoder, organizes them into patch-level units, and updates them via an attention mechanism. We evaluate our method using 23 univariate benchmark datasets. Experimental results demonstrate that MOMEMTO, as a single model, achieves higher scores on AUC and VUS metrics compared to baseline methods, and further enhances the performance of its backbone TSFM, particularly in few-shot learning scenarios.

arXiv

Local Cluster Cardinality Estimation for Adaptive Mean Shift

Étienne Pepin

This article presents an adaptive mean shift algorithm in which every parameter used at a point is derived from that point's own distance distribution. The distance distribution from a point to all others is used to estimate the cardinality of the local cluster by identifying a local minimum in the density of that distribution; the statistics of the identified subset then set the bandwidth and the kernel radius threshold applied at that point. The estimator built this way is scale invariant, since the $γ$ function it rests on is unchanged when the data is multiplied by a positive constant, so no length constant has to be chosen for the scale of the data. It is also local: $γ$ evaluated at rank $k$ depends only on the $k$ nearest distances, and the mean shift kernel is truncated at the estimated cluster radius, so data lying beyond that radius neither enters the estimate of the local cluster nor contributes to the weighted mean. This contrasts with kernel density estimation, which in its basic form measures density in a neighborhood of fixed size and needs a bandwidth chosen for the dataset as a whole. Our algorithm is competitive within the adaptive mean shift family: it obtains a higher Rand index than the weighted adaptive mean shift method of Ren et al. (2014) on seven of the nine datasets of that study, four of them by more than 0.03 and three by less than 0.012, and it performs competitively on a broader clustering benchmark, in both cases without being given the number of clusters.

arXiv

Program Semantic Inequivalence Game with Large Language Models

Antonio Valerio Miceli-Barone, Vaishak Belle, Ali Payani

Large Language Models (LLMs) can achieve strong performance on everyday coding tasks, but they can fail on complex tasks that require non-trivial reasoning about program semantics. Finding training examples to teach LLMs to solve these tasks can be challenging. In this work, we explore a method to synthetically generate code reasoning training data based on a semantic inequivalence game (SInQ): a generator agent creates program variants that are semantically distinct, derived from a dataset of real-world programming tasks, while an evaluator agent has to identify input examples for which they behave differently. The agents train each other semi-adversarially, improving their ability to understand the underlying logic of code. We evaluated our approach on multiple code generation and understanding benchmarks, including cross-language vulnerability detection (Lu et al., 2021),, where our method improves vulnerability detection in C/C++ code despite being trained exclusively on Python code, and the challenging Python builtin identifier swap benchmark (Miceli Barone et al., 2023),, showing that whereas modern LLMs still struggle with this benchmark, our approach yields substantial improvements. We release the code needed to replicate the experiments, as well as the generated synthetic data, which can be used to fine-tune LLMs.

arXiv

A Variational Analysis of Kernel Learning with Learnable Linear Transformations

Yang Li, Feng Ruan

The classical kernel ridge regression problem aims to find the best fit for the output $Y$ as a function of the input data $X\in \mathbb{R}^d$, with a fixed choice of regularization term imposed by a given choice of a reproducing kernel Hilbert space, such as a Sobolev space. Here we consider a generalization of the kernel ridge regression problem, by introducing an extra matrix parameter $U$, which aims to detect the scale parameters and the feature variables in the data, and thereby improve the efficiency of kernel ridge regression. This naturally leads to a nonlinear variational problem to optimize the choice of $U$. We study various foundational mathematical aspects of this variational problem, including its Euler-Lagrange equation, continuity and first variation, limiting behavior under degenerate or diverging transformations, and the structure of its local minimizers. Particular attention is given to two data-distribution settings, namely multi-scale and multi-index models, where the learned transformation $U$ encodes intrinsic scale parameters and the essential low-dimensional feature variables, respectively.

arXiv

Heterogeneous transfer learning for high-dimensional regression with feature mismatch

Jae Ho Chang, Massimiliano Russo, Subhadeep Paul

We study Heterogeneous Transfer Learning (HTL) for high-dimensional regression with differing feature sets. Such feature mismatch arises when some variables available in a data-rich source domain are unavailable in a data-poor target domain. Yet most homogeneous TL methods require the same feature space in both the source and target domains, limiting their practical applicability. Conversely, existing HTL methods lack statistical error guarantees, limiting their utility for scientific discovery. We propose an HTL method that first learns a feature map between the missing and observed features leveraging the vast source data, imputes the unavailable features in the target, and then performs a two-step TL for penalized regression. We consider both the linear and the nonparametric feature maps. We develop upper bounds on the estimation and prediction errors of HTL, assuming that the source and target parameters differ sparsely, without requiring the target model itself to be sparse. We also establish matching minimax lower bounds, showing that the proposed procedures achieve optimal rates. Our results elucidate the effects of model complexity, sample size, the quality and differences in feature maps, and differences in the models across domains. We also derive minimax rates for the misspecified homogeneous TL model that discards unavailable features and show that our HTL procedure can attain a smaller error rate than homogeneous TL. We further extend the framework to multiple source domains and develop a negative-transfer defense that provably excludes adversarial sources from transfer with high probability.

arXiv

Deep Activity Model: A Generative Approach for Human Mobility Pattern Synthesis

Xishun Liao, Qinhua Jiang, Brian Yueshuai He, Yifan Liu, Chenchen Kuai, Jiaqi Ma

Human mobility plays a crucial role in transportation, urban planning, and public health, but current approaches face important limitations. Existing deep learning models tend to overlook the semantic interdependencies among activities and households and rely on restricted GPS data, while activity-based models depend on rigid assumptions and extensive data, making them costly and difficult to adapt to new regions, especially those with limited conventional travel data. To address these limitations, we propose a generative Transformer model that uses socio-demographic and household attributes to synthesize daily activity chains, with a location module assigning spatial zones to produce complete daily trajectories. Trained on open-source and widely available household travel survey data and then fine-tuned with local data, the model captures national activity patterns and transfers effectively to California, Washington, and Mexico City. This approach offers potential for advancing synthetic human mobility modeling and provides urban planners and policymakers with improved tools for simulating transportation systems and supporting decisions in urban development and public health. Its practical utility is demonstrated through large-scale traffic simulations in Los Angeles County. Compared to the SCAG Activity-Based Model (ABM), the proposed method produces consistent spatial demand patterns, achieving an activity location cosine similarity of 0.997 and a network-level vehicle-miles-traveled Mean Absolute Percentage Error (MAPE) of 4.97%. Compared to real-world observations from Caltrans PeMS, the simulated traffic achieves MAPE of 5.85% for traffic volume and 4.36% for speed on California's I-405 corridor, presenting the practicality of learning-based synthetic mobility for regional simulation and planning.

arXiv

Redesigning and Auditing Deep Research Writing for Faithful Reports

Hiroaki Hayashi, Pranav Narayanan Venkit, Prafulla Kumar Choubey, Chien-Sheng Wu

Rubric-based evaluations of deep-research (DR) systems often obscure fine-grained factual failures in generated reports. We introduce CLAIMPROBE, a claim-level audit that decomposes DR reports into claims and measures hallucination, misattribution, citation hygiene, and necessary-fact recall against retrieved evidence. Using CLAIMPROBE, we find that strong DR pipelines can omit key evidence and misattribute claims even when their rubric scores remain stable. We then propose CLAIMWRITER, a hierarchical claim-based writer that extracts source facts, maps them to a query-derived outline, and drafts each section from a source-linked claim representation. Across three prior DR frameworks, replacing only the report writer with CLAIMWRITER reduces hallucination by 2.6 to 4.5 times and improves necessary-fact recall by 1.2 to 1.7 times, while largely preserving overall report quality. CLAIMWRITER also enables localized revision: when sources change, it propagates changed source facts into revised reports at the highest rate among update methods, while also being more cost-effective.

arXiv

Do CNNs Internally Represent Real and Fake Images Differently? A Hidden-Layer Analysis

Moumita Sen Sarma, Pascal Hitzler, Eugene Y. Vasserman

Fake/synthetic images are increasingly prevalent, but it remains unclear whether Convolutional Neural Networks (CNNs) process real and fake images in the same internal manner. This work examines the hypothesis that CNNs represent real and fake images differently, such that fake images induce different hidden-layer activation patterns even when semantic content is preserved. The hypothesis is evaluated in scene recognition settings using trained CNN models. Dense-layer activations are extracted, and neurosymbolic methods assign semantic labels to selected neurons. For each real test image, corresponding fake images are generated with similar semantic content using object-label-guided text-to-image and image-to-image generation based on Stable Diffusion variants. Paired real-fake activation patterns are then compared statistically. Additional experiments with another dataset, CNN architecture, generative model, and JPEG/blur degradation analysis assess robustness. Results suggest that fake images evoke different hidden-neuron activations, and these differences are not explained only by simple image degradation. Overall, the findings indicate that real and fake images differ in CNN hidden-layer activation behavior at least in some settings, which opens the door for follow-up work on making use of this different behavior to improve fake image detection.

arXiv

Low Cost Two-Stage Fabric Defect Detection at the Edge

Rasel Hossen, Diptajoy Mistry, Mosaddek Hossain Kamal

Fabric inspection in the garment industries of low-income economies remains largely manual, and commercial vision systems are priced beyond most small and medium mills. Because defects are sparse under controlled production, a natural response is a cascade: screen every frame with a cheap anomaly detector and invoke a full detector only on suspicious frames. We build such a cascade for four knit-fabric defect classes and deploy it end-to-end on an NVIDIA Jetson Nano with TensorRT FP16. Stage 1 is a compact convolutional autoencoder with decoder attention gates, an edge-weighted reconstruction loss, and feature-level distillation from a frozen YOLOv5n teacher; Stage 2 is YOLOv5n, invoked only on flagged frames. On a 249-image benchmark disjoint from detector training (20 defective, 229 non-defective), Stage 1 at a recall-prioritised threshold flags all 20 defective images (95% CI 0.83-1.00) at a false-positive rate of 49.3% (113/229), reducing false positives by 19.3% relative to a plain autoencoder (p=0.011). The parallel pipeline reaches 13.45 FPS against 9.86 FPS for a sequential YOLO-only loop. Our central finding comes from decomposing that 1.36x: 91% of it is attributable to overlapping JPEG decode with inference rather than to the cascade, which contributes only a 5.1% inference reduction at the measured forwarding rate p = 0.534. We further show that forwarding here is false-positive-limited rather than prevalence-limited - 85% of forwarded frames are false alarms - and quantify the 29-45% inference reduction attainable under tighter calibration. We report this as a caution for cascade speedups measured without controlling the data path, and position the system as AI-assisted triage rather than autonomous acceptance.

arXiv

Spatial Attention Noise Masking for Causally Sufficient Interpretability

Benjamin Formby, Kuang-Ching Wang, D Hudson Smith

We present a novel causal approach to interpretability for computer vision models that dynamically masks the input image prior to classification. The interpretability of deep learning predictions is critical in high-stakes fields such as medical imaging, security, and autonomous driving. Most interpretability methods are applied passively to already trained models, which typically result in correlational rather than causal explanations. Existing causal interpretability methods are limited to post hoc analysis, weakening the causal claims. Additionally, existing active methods generally lack explanations that explicitly assign responsibility to input features. This work proposes a spatial attention noise masking framework that provides causal explanations about the features sufficient for the prediction. The proposed framework consists of: 1) a UNet-style mask generator, and 2) a Resnet18 encoder and linear classifier that classifies both masked and unmasked versions of an input image. The generated masks are regularized to be sparse and spatially smooth, while masked image embeddings are constrained to remain consistent with embeddings from the corresponding unmasked images. The resulting masks can be interpreted as feature attribution maps that are competitive with related interpretability methods while additionally providing strong causal explanations of model predictions. Quantitative evaluations demonstrate mask faithfulness, near-baseline classification performance across five classification tasks despite substantial masking of image information, and robustness to distribution shifts such as background swapping and natural adversarial examples. Qualitative comparisons further demonstrate mask behavior and competitive interpretability relative to state-of-the-art feature attribution methods.

arXiv

Privacy-Preserving Dataset Curation for Kuala Lumpur Urban Traffic: Grounded Vision-Language Detection with Spatial Vehicle-Context Filtering

Mohammed Abdul Al Arafat Tanzin, Rudzidatul Akmam Dziyauddin

The rapid advancement of intelligent transportation systems and autonomous driving relies heavily on multi-modal urban traffic datasets. However, curating high-fidelity video imagery in complex tropical urban environments---specifically Kuala Lumpur, Malaysia---presents severe challenges for Personally Identifiable Information (PII) anonymization due to high motorcycle density, dark acrylic license plates, dynamic camera tilt, and extreme tropical glare. We propose an automated anonymization framework tailored for the Kuala Lumpur Road Dataset, captured via a mobile cycling platform at 2 FPS. We document how legacy Haar cascades and YOLOv8 fail under these conditions---generating false positives on background elements while missing rotated or occluded targets. Our architecture resolves this by integrating Grounding DINO---a zero-shot open-set vision-language transformer---with a novel Spatial Vehicle Region of Interest (ROI) Containment Engine. By requiring license plate centroids to reside within validated vehicle boundaries, the pipeline suppresses environmental false positives while automatically obfuscating faces, heads, and license plates. An initial evaluation on 1,266 frames demonstrates a $\sim$95\% success rate, with remaining failures restricted to small, heavily occluded, oblique, or ambiguous targets. Coupled with temporal persistence mechanisms and an automated quality-control auditor, the framework minimizes privacy-related false negatives while preserving scene context for downstream vision tasks. While formal legal compliance depends on broader governance procedures, this publicly available pipeline and demonstration notebook provide an auditable preprocessing stage for privacy-aware dataset curation.

arXiv

A Vision Transformer for ECG-Based Detection of Left Ventricular Systolic Dysfunction Across Multiple Clinical Sites

Burcu Ozek, Aruna Mohan, David Vorchheimer, Daniel Weiss, Eyal Kedar, Tamar Sobol, Or Zilbershot, Fatemeh Afghah

Reduced left ventricular ejection fraction (LVEF) is frequently asymptomatic and often detected only after advanced heart failure develops. Electrocardiograms are recorded routinely yet underused for this condition, because reduced LVEF has no single diagnostic waveform. We trained an ensemble of vision transformers from scratch to detect reduced LVEF ($\leq$40%) from 12-lead ECGs, analyzing each heartbeat individually, using 10,142 patients across seven sites in three US health systems. In a held-out external cohort of 4,092 patients from three geographically independent US clinical sites at a real-world reduced-LVEF prevalence of 8.72%, the model achieved an AUROC of 0.88 (95% CI 0.86-0.89), sensitivity 81.2%, specificity 81.0%, and negative predictive value 97.8%. Sensitivity remained high across sex, race, ethnicity, and comorbidity subgroups, while specificity was lower in older patients and those with atrial fibrillation or cardiomyopathy. Beat-level attention maps provided interpretability into the model's predictions, showing consistent focus on the QRS complex rather than the P wave. These findings support the potential of routine ECGs as a scalable first-pass triage step to identify patients who should undergo echocardiography for reduced ejection fraction across diverse patient populations.

arXiv

Braided Vision Transformer for Stroke Detection in Multi-view Retinal Fundus Imaging

Aysen Degerli, Mika Hilvo

Stroke remains a leading cause of mortality and morbidity worldwide, emphasizing the importance of its accurate and immediate assessment. Retinal fundus imaging has emerged as a promising modality for stroke assessment, as the retina reflects cerebrovascular and neurological risk factors. Contrary to conventional neuroimaging techniques, retinal fundus imaging offers a non-invasive, cost-effective, and portable alternative for rapid screening. This paper explores the feasibility of retinal fundus imaging for stroke and transient ischemic attack (TIA) detection using macula-centric and optic nerve head-centric views captured from both eyes. Our study introduces, to the best of our knowledge, the first vision transformer model for retinal fundus imaging in stroke assessment, offering a novel approach for capturing retinal patterns. Thereby, we propose the Braided Vision Transformer (BViT) model, which extracts representative features from the given multi-view images while simultaneously capturing inter-view relationships across both eyes, enabling a more informative understanding of retinal biomarkers associated with cerebrovascular events. Experiments conducted on our collected Stroke-Data dataset demonstrate that BViT achieves an AUC score of 0.75 for stroke detection, outperforming regular vision transformers.

arXiv

AeroGround: A Comprehensive Benchmark for Aerial-Ground Collaborative Reasoning

Shenghong Yi, Lin Zhang, Muzian Li, Jiakang Yuan, Haoyu Zhang, Peng Ye, Jiayuan Fan, Huafeng Qin, Tao Chen

Vision-language models (VLMs) have been widely employed in understanding and reasoning tasks for unmanned aerial vehicles (UAVs). Existing UAV benchmarks primarily focus on aerial-view scenarios. However, whether current VLMs can perform well on understanding and reasoning tasks in aerial-ground collaborative scenarios which are practical in real-world applications like rescue and infrastructure inspection remains underexplored. To address this gap, we introduce AeroGround, a comprehensive benchmark for evaluating VLMs in aerial-ground collaborative reasoning. AeroGround is built upon a simulated aerial-ground dataset containing approximately 29,000 multimodal observation groups from diverse open environments, and provides 2,250 high-quality question-answering instances covering cross-view correspondence, spatial understanding, and reasoning. Experiments on 16 pretrained VLMs, together with two domain-adapted variants, reveal a substantial gap between current models and human performance: the best model achieves an average accuracy of 54.4%, whereas humans reach 93.3%. By systematically revealing the strengths and limitations of existing models in aerial-ground collaborative reasoning, AeroGround provides a foundation for developing more capable aerial-ground collaborative embodied intelligence systems.

arXiv

DeCo-MIL: Debiased Counterfactual Reasoning for Long-Tailed Whole Slide Image Analysis

Xiaoxiao Li, Xitong Ling, Jiawen Li, Weiming Chen, Zhenyang Cai, Xidong Wang, Tian Guan, Benyou Wang, Yonghong He

Multiple instance learning (MIL) is widely used for weakly supervised whole slide image (WSI) analysis. However, under long-tailed distributions, MIL-based WSI analysis faces a nested dual long-tail: an inter-slide class long tail and an intra-slide long tail of instance-level discriminative evidence. The two long tails are coupled: tail classes have few training slides, while their limited diagnostic evidence is concentrated in a few patches and obscured by abundant within-bag redundancy. This coupling biases models toward head classes and degrades rare-class recognition. To address this, we propose DeCo-MIL for long-tailed WSI analysis, which jointly alleviates the nested dual long-tail through frequency-debiased counterfactual reasoning. For the inner long tail, DeCo-MIL clusters patches into tissue-morphology anchors, replaces each anchor with its matched normal prototype to perform a counterfactual intervention, and estimates its counterfactual contribution to the ground-truth class using class-frequency-corrected predictions. These contributions guide redundancy masking to preserve scarce discriminative instances. For the outer long tail, DeCo-MIL constructs anchor-stratified pseudo-bags from redundancy-reduced bags and combines tail-aware oversampling with consistency regularization, increasing effective supervision for tail classes while preserving tissue-morphology composition. Extensive experiments on three long-tailed WSI benchmarks demonstrate that DeCo-MIL achieves state-of-the-art performance in both tail-class recognition and overall classification.

arXiv

Local Gains and Fixed-Assignment Set Losses in Shared Set Decoders

Ze Zhang, Yang Zhang

A query-relation deletion can improve the edited slot while reducing the utility of the prediction set that contains it. We study this tension in two related ResNet-50 DETR-family checkpoints using recorded, selection-conditional evidence from 710 paired image-relation units per checkpoint. The primary comparison subtracts a matched active control, which deletes the same leader source at a different recorded recipient, from the selected target deletion. It is therefore a composite contrast rather than a same-recipient placebo. The target-minus-control contrast is locally positive and fixed-assignment negative in both checkpoints. The opposite-sign pattern occurs within 302/710 DETR units and 460/710 DINO units. After rematching, the corresponding counts are 285/710 and 433/710. Rematching and native selection absorb enough of the mean loss for DETR intervals to cross zero, whereas DINO intervals remain negative, so persistence across readouts differs by checkpoint. A fixed-map comparison between hard deletion and a mass-preserving edit also differs before rematching. That comparison is conditional on the outcome-blind map and does not establish same-dose transport. Local intervention success therefore does not determine the consequence for a jointly decoded set. The supported conclusion is selection-conditional deletion sensitivity whose persistence depends on the readout and intervention operator. We do not identify an intervention-invariant edge mechanism, detector-level degradation, population prevalence, or the value of a training-time regularizer.

arXiv

Mitigating Bias in Large Vision-Language Models via Counterfactual Ensemble Decoding

Yisong Xiao, Aishan Liu, Yongxin Huang, Zonghao Ying, Shiji Zhao, Tianlin Li, Yong Han, Jian Yang, Xianglong Liu

Large Vision-Language Models (LVLMs) have achieved remarkable performance across a wide range of tasks; however, they often inherit social biases from their training data, resulting in biased behavior when processing portraits from different social groups. Existing debiasing approaches typically compare token probabilities between the original and biased generations during decoding, but they are fundamentally limited by their reliance on a single, stereotyped viewpoint and fail to account for the diversity of social perspectives. Inspired by the social science principle that diversity fosters fairness, we propose Counterfactual Ensemble Decoding (CED), a novel framework that constructs multi-group counterfactual perspectives within the visual representation space and integrates them during decoding to promote equitable model behavior. CED first performs counterfactual steering in the visual space by identifying semantic directions associated with each social group and generating counterfactual representations along these directions, thereby offering diverse perspectives that disrupt stereotypical narratives. During decoding, CED locates the decoder layer exhibiting the greatest divergence among these perspectives and ensembles their token distributions using uncertainty-aware weights, prioritizing high-confidence tokens from different groups to yield a more balanced probability distribution that guides fairer generation. Extensive experiments on three social bias evaluation benchmarks demonstrate that \tool achieves substantial improvements over leading baselines, reducing bias by up to 47.97% across scenarios involving occupations, descriptors, and persona traits. Moreover, CED also preserves the core capabilities of the original model with minimal degradation.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.