WWW 2025 论文列表

ACM Web Conference

录用类型

409篇论文

按官方录用论文列表统计,可能与会议公布的数字略有出入。

  • Oral
    15537.9%
  • Poster
    25462.1%

Top 10 主题(点击筛选)

暂无主题数据。

全部 207 个研究方向的趋势

浏览全部论文

Training-free Graph Anomaly Detection: A Simple Approach via Singular Value Decomposition

Cheng Zhou, Gx Li, Hao Weng, Yiyu Xiang

展开摘要

Graph anomaly detection has been widely applied in real-world applications, where deep learning-based methods have demonstrated promise. However, prior methods often suffer from various limitations, such as poor detection accuracy, long training time, complicated training schemes, and lack of scalability. To combat this dilemma, we propose TFGAD, a simple yet effective training-free approach for graph anomaly detection. Particularly, TFGAD comprises two transformation matrices, each of which serves to process one type of node feature (attributes or local structure). Notably, these matrices can be optimally determined via singular value decomposition, thus requiring no prior training. Further, we tailor a lightweight anomaly scoring function, which integrates the reconstruction error of attributes with the projection length of local structures to quantify graph anomalies. Extensive experiments demonstrate that TFGAD leads to significant improvements over state-of-the-art reconstruction-/contrastive-based deep learning baselines while reaching much less runtime and memory overhead.

Poster
Cutting through the Confusion: A Measurement Study of Homograph Domains in Ethereum Name Service

Jianwei Huang, Sridatta Raghavendra Chintapalli, Mengxiao Wang, Guofei Gu

展开摘要

In recent years, the Ethereum Name Service (ENS) has garnered significant attention within the community for enabling the use of Unicode in domain names, thereby facilitating the inclusion of a wide array of character sets such as Greek, Cyrillic, Arabic, and Chinese. While this feature enhances the versatility and global accessibility of domain names, it concurrently introduces a substantial security vulnerability due to the presence of homoglyphs—characters that are visually similar to others across Unicode and ASCII sets. These similarities can be exploited in homoglyph attacks, posing a distinct threat to domain name integrity. This study investigates the prevalence and security implications of homoglyph domains within the ENS ecosystem, revealing that these domains present a more pronounced security concern compared to their counterparts in the traditional Domain Name System (DNS). Despite community efforts to counteract this issue through a normalization process prior to domain resolution, our analysis uncovers significant discrepancies in how the normalization processes are applied across various applications. This inconsistency could result in the same domain name being resolved to different addresses in different applications, underscoring a critical vulnerability. To systematically evaluate this inconsistency, we designed a tool for detecting application-level discrepancies in domain normalization process without requiring access to the application’s source code. Our evaluation on hundreds of real-world Web3 applications identifies widespread deviations from established homoglyph mitigation practices, with more than 60% digital wallets and 80% dApps (decentralized applications) not able to produce consistent ENS resolving results, potentially impacting millions of users. This analysis underscores the urgent need for a standardized implementation of normalization processes to safeguard the integrity and security of ENS domains.

Poster
A Cooperative Multi-Agent Framework for Zero-Shot Named Entity Recognition

Zihan Wang, Ziqi Zhao, Yougang Lyu, Zhumin Chen, Maarten de Rijke, Zhaochun Ren

展开摘要

Zero-shot named entity recognition (NER) aims to develop entity recognition systems from unannotated text corpora. This task presents substantial challenges due to minimal human intervention. Recent work has adapted large language models (LLMs) for zero-shot NER by crafting specialized prompt templates. And it advances the models’ self-learning ability by incorporating self-annotated demonstrations. Two important challenges persist: (i) Correlations between contexts surrounding entities are overlooked, leading to wrong type predictions or entity omissions. (ii) The indiscriminate use of task demonstrations, retrieved through shallow similarity-based strategies, severely misleads the inferences made by LLMs. In this paper, we introduce CMAS, or cooperative multi-agent system, a framework for zero-shot NER that uses the collective intelligence and tailored abilities of multiple agents to address the challenges outlined above. Cooperative multi-agent system (CMAS) has four main agents: (i) a self-annotator, (ii) a type-related feature (TRF) extractor, (iii) a demonstration discriminator, and (iv) an overall predictor. To explicitly capture correlations between contexts surrounding entities, CMAS reformulates NER into two subtasks: recognizing named entities and identifying entity type-related features within the target sentence. Moreover, pseudo-labels for TRFs are generated using mutual-information criteria without requiring human effort, facilitating the prediction of the TRF extractor. To assess the quality of demonstrations, a demonstration discriminator is established to incorporate the self-reflection mechanism, automatically evaluating helpfulness scores for the target sentence and enabling controllable utilization of demonstrations. Experimental results show that CMAS significantly improves zero-shot NER performance across six benchmarks, including both domain-specific and general-domain scenarios. Furthermore, CMAS demonstrates its effectiveness in few-shot settings and with various LLM backbones.

Poster
MDEval: Evaluating and Enhancing Markdown Awareness in Large Language Models

Zhongpu Chen, Yinfeng Liu, Long Shi, Zhi-Jie Wang, Xingyan Chen, Yu Zhao, Fuji Ren

展开摘要

Large language models (LLMs) are expected to offer structured Markdown responses for the sake of readability in web chatbots (e.g., ChatGPT). Although there are a myriad of metrics to evaluate LLMs, they fail to evaluate the readability from the view of output content structure. To this end, we focus on an overlooked yet important metric --- Markdown Awareness, which directly impacts the readability and structure of the content generated by these language models. In this paper, we introduce MDEval, a comprehensive benchmark to assess Markdown Awareness for LLMs, by constructing a dataset with 20K instances covering 10 subjects in English and Chinese. Unlike traditional model-based evaluations, MDEval provides excellent interpretability by combining model-based generation tasks and statistical methods. Our results demonstrate that MDEval achieves a Spearman correlation of 0.791 and an accuracy of 84.1% with human, outperforming existing methods by a large margin. Extensive experimental results also show that through fine-tuning over our proposed dataset, less performant open-source models are able to achieve comparable performance to GPT-4o in terms of Markdown Awareness. To ensure reproducibility and transparency, MDEval is open sourced at https://anonymous.4open.science/r/MDEval-Benchmark-1730/.

Poster
Horizontal Federated Heterogeneous Graph Learning: A Multi-Scale Adaptive Solution to Data Distribution Challenges

Jia Wang, Yawen Li, Zhe Xue, Yingxia Shao, Zeli Guan, Wenling Li

展开摘要

Federated heterogeneous graph learning, an extension of federated learning, enables effective representation of complex multidimensional relationships while preserving data privacy. In horizontal federated heterogeneous graph learning, data from different parties often differ in topology and semantic distributions, causing sensitivity to distribution imbalance and amplifying the complexity of the topological structure. This interaction makes it difficult for models to learn shared representations, leading to increased instability during training. To address these challenges, this paper proposes a novel multi-scale adaptive horizontal federated heterogeneous graph learning method MAFedHGL. A random masking mechanism forces the model to infer missing connections. The model also captures multi-hop and multi-path connections using high-order topology mining, enhancing robustness against structural heterogeneity. Dynamic semantic consistency modeling uses a masking matrix to recover and integrate diverse node attributes, ensuring both global and local semantic consistency. Using clustering coefficients as aggregation weights enables clients with richer structural information to contribute more effectively to the global model, improving adaptability and performance across varying data distributions in horizontal federated heterogeneous graph learning. Extensive experiments on multiple public heterogeneous graph datasets validate that the proposed method outperforms state-of-the-art methods in both performance and robustness across various data distribution scenarios.

Poster
Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categories

Tianlong Wang, Xianfeng Jiao, Yinghao Zhu, Zhongzhi Chen, Yifan He, Xu Chu, Junyi Gao, Yasha Wang, Liantao Ma

展开摘要

Recent studies have indicated that Large Language Models (LLMs) harbor an inherent understanding of truthfulness, yet often fail to express fully and generate false statements. This gap between "knowing" and "telling" poses a challenge for ensuring the truthfulness of generated content. Inspired by recent work on the practice of encoding human-interpretable concepts linearly within large language models, we treat truthfulness as a specially linearly encoded concept within LLMs, and introduce Adaptive Activation Steering (ACT), a tuning-free method that adaptively shifts LLM's activations in the "truthful" direction during inference. ACT addresses diverse categories of hallucinations by utilizing diverse truthfulness-related steering vectors and adjusting the steering intensity adaptively. Applied as an add-on across various models, ACT significantly improves truthfulness in LLaMA ($\uparrow$ 142\%), LLaMA2 ($\uparrow$ 24\%), Alpaca ($\uparrow$ 36\%), Vicuna ($\uparrow$ 28\%), LLaMA2-Chat ($\uparrow$ 19\%), and LLaMA3($\uparrow$ 34\%). Furthermore, we verify ACT's scalability across larger models (13B, 33B, 65B), underscoring the adaptability of ACT to large-scale language models. Code: https://anonymous.4open.science/r/ACT24.

Poster
LargePiG for Hallucination-Free Query Generation: Your Large Language Model is Secretly a Pointer Generator

ZhongXiang Sun, Zihua Si, Xiaoxue Zang, Kai Zheng, Yang Song, Xiao Zhang, Jun Xu

展开摘要

Recent research on query generation has focused on using Large Language Models (LLMs), which despite bringing state-of-the-art performance, also introduce issues with hallucinations in the generated queries. In this work, we introduce relevance hallucination and factuality hallucination as a new typology for hallucination problems brought by query generation based on LLMs. We propose an effective way to separate content from form in LLM-generated queries, which preserves the factual knowledge extracted and integrated from the inputs and compiles the syntactic structure, including function words, using the powerful linguistic capabilities of the LLM. Specifically, we introduce a model-agnostic and training-free method that turns the **Large** Language Model into a **P**o**i**nter-**G**enerator (**LargePiG**), where the pointer attention distribution leverages the LLM's inherent attention weights, and the copy probability is derived from the difference between the vocabulary distribution of the model’s high layers and the last layer. To validate the effectiveness of LargePiG, we constructed two datasets for assessing the hallucination problems in query generation, covering both document and video scenarios. Empirical studies on various LLMs demonstrated the superiority of LargePiG on both datasets. Additional experiments also verified that LargePiG could reduce hallucination in large vision language models and improve the accuracy of document-based question answering and factuality evaluation tasks.

Poster
ITMPRec: Intention-based Targeted Multi-round Proactive Recommendation

Yahong Lian, Chunyao Song, Tingjian Ge

展开摘要

Personalized user preference driven recommendations have seamlessly intertwined with our daily lives. However, item providers may expect specific items to gradually increase their appeal to users over the course of users’ long-term interactions with the system, but few studies pay attention to this problem. In this paper, we propose a novel intention-based targeted multi-round proactive recommendation method, dubbed ITMPRec. Specifically, we first choose a set of target items from the target category, by conducting a pre-match strategy. Afterward, we utilize a multi-round nudging recommendation method, in which we design a module to quantify the intention-level dynamic evolution of users so that we could choose more appropriate intermediate items during guidance. Besides, we model each user’s sensitivity to the changes in representation induced by the intermediate items they accept. Finally, we propose a design for a Large Language Model (LLM) agent as a pluggable component to simulate user feedback. This design offers an alternative to the traditional click model based on distribution, relying on the agent’s external knowledge and reasoning capabilities. Through extensive experiments on four public datasets, we demonstrate the superiority of ITMPRec compared to seven baseline models. The code repository is available at https://anonymous.4open.science/r/ITMPRec-D821.

Poster
A Theory-Driven Approach to Inner Product Matrix Estimation for Incomplete Data: An Eigenvalue Perspective

Fangchen Yu, Yicheng Zeng, Jianfeng Mao, Wenye Li

展开摘要

Addressing the critical challenge of data incompleteness in inner product matrix estimation, we introduce a novel eigenvalue correction method designed to precisely reconstruct true inner product matrices from incomplete data. Utilizing random matrix theory, our method adjusts the eigenvalue distribution of the estimated inner product matrix to align with the ground-truth. This approach significantly reduces estimation errors for both inner product matrices and the derived Euclidean distance matrices, thereby enhancing the effectiveness of similarity searches on incomplete data. Our method surpasses traditional data imputation and similarity calibration techniques in both maximum inner product search and nearest neighbor search tasks, demonstrating marked advancements in managing incomplete data. It exhibits robust performance across various missing rates and diverse scenarios.

Poster
Graph Meets LLM for Review Personalization based on User Votes

Sharon Hirsch, Lilach Zitnitski, Slava Novgorodov, Ido Guy, Bracha Shapira

展开摘要

Review personalization aims at presenting the most relevant reviews of a product according to the preferences of the individual user. Existing studies of review personalization use the reviews authored by the user as a proxy for their preferences, and henceforth as a means for learning and evaluating personalization quality. In this work, we suggest using review votes rather than authorship for personalization. We suggest MAGLLM, an approach that leverages heterogeneous graphs for modeling the relationships among reviews, products, and users, with large language model (LLM) to enrich user representation on the graph. Our evaluation over a unique public dataset that includes user voting information indicates that the vote signal yields substantially higher personalization performance across a variety of recommendation methods and e-commerce domains. It also indicates that our graph-LLM approach outperforms comparative baselines and algorithmic alternatives. We conclude with concrete recommendations for e-commerce platforms seeking to enhance their review personalization experience.

Poster
LLMCloudHunter: Harnessing LLMs for Automated Extraction of Detection Rules from Cloud-Based CTI

Yuval Schwartz, Lavi Ben-Shimol, Dudu Mimran, Yuval Elovici, Asaf Shabtai

展开摘要

As the number and sophistication of cyber attacks have increased, threat hunting has become a critical aspect of active security, enabling proactive detection and mitigation of threats before they cause significant harm. Open-source cyber threat intelligence (OSCTI) is a valuable resource for threat hunters, however, it often comes in unstructured formats that require further manual analysis. Previous studies aimed at automating OSCTI analysis are limited since (1) they failed to provide actionable outputs, (2) they did not take advantage of images present in OSCTI sources, and (3) they focused on on-premises environments, overlooking the growing importance of cloud environments. To address these gaps, we propose LLMCloudHunter, a novel framework that leverages large language models (LLMs) to automatically generate generic-signature detection rule candidates from textual and visual OSCTI data. We evaluated the quality of the rules generated by the proposed framework using 20 annotated real-world cloud threat reports. The results show that our framework achieved a precision of 83\% and recall of 99\% for the task of accurately extracting API calls made by the threat actor and a precision of 99\% with a recall of 97\% for IoCs. Additionally, 99.18\% of the generated detection rule candidates were successfully compiled and converted into Splunk queries.

Poster
TriG-NER: Triplet-Grid Framework for Discontinuous Named Entity Recognition

Rina Carines Cabral, Caren Han, Areej Alhassan, Riza Batista-Navarro, Goran Nenadic, Josiah Poon

展开摘要

Discontinuous Named Entity Recognition (DNER) presents a challenging problem where entities may be scattered across multiple non-adjacent tokens, making traditional sequence labelling approaches inadequate. Existing methods predominantly rely on custom tagging schemes to handle these discontinuous entities, resulting in models tightly coupled to specific tagging strategies and lacking generalisability across diverse datasets. To address these challenges, we propose TriG-NER, a novel Triplet-Grid Framework that introduces a generalisable approach to learning robust token-level representations for discontinuous entity extraction. Our framework applies triplet loss at the token level, where similarity is defined by word pairs existing within the same entity, effectively pulling together similar and pushing apart dissimilar ones. This approach enhances entity boundary detection and reduces the dependency on specific tagging schemes by focusing on word-pair relationships within a flexible grid structure. We evaluate TriG-NER on three benchmark DNER datasets and demonstrate significant improvements over existing grid-based architectures. These results underscore our framework’s effectiveness in capturing complex entity structures and its adaptability to various tagging schemes, setting a new benchmark for discontinuous entity extraction.

Poster
TAPE: Tailored Posterior Difference for Auditing of Machine Unlearning

Weiqi Wang, Zhiyi Tian, An Liu, Shui Yu

展开摘要

Increasing studies focus on machine unlearning as it upholds users' right to be forgotten, under which individuals can request the removal of their specified samples from trained models. However, the auditing of machine unlearning processes remains significantly underexplored. Although some existing methods offer unlearning auditing by leveraging backdoors, these backdoor-based approaches are inefficient and impractical, as they necessitate involvement in the initial model training process to embed the backdoors. In this paper, we propose a TAilored Posterior diffErence (TAPE) method to provide unlearning auditing independently of original model training. We observe that the process of machine unlearning inherently introduces changes in the model, which contains information related to the erased data. TAPE leverages unlearning model differences to assess how much information has been removed through the unlearning operation. Firstly, TAPE mimics the unlearned posterior differences by quickly building unlearned shadow models based on first-order influence estimation. Secondly, we train a Reconstructor model to extract and evaluate the private information of the unlearned posterior differences to audit unlearning. Existing privacy reconstructing methods based on posterior differences are only feasible for model updates of a single sample. To enable the reconstruction effective for multi-sample unlearning requests, we propose two strategies, unlearned data perturbation and unlearned influence-based division, to augment the posterior difference. Extensive experimental results indicate the significant superiority of TAPE over the state-of-the-art unlearning verification methods, at least 4.5$\times$ efficiency speedup and supporting the auditing for broader unlearning scenarios.

Poster
Seed: Bridging Sequence and Diffusion Models for Road Trajectory Generation

Xuan Rao, Shuo Shang, Renhe Jiang, Peng Han, Lisi Chen

展开摘要

Road trajectory generation creates synthetic yet realistic trajectories to tackle data collection costs and privacy concerns. Existing methods generate a trajectory either segment-by-segment using sequence models or holistically in one step using diffusion models. Sequence-based models have good regularity and consistency (i.e., resemble the input trajectories) but lack diversity, while diffusion-based models enhance diversity but sacrifice regularity and consistency. To combine the merits of existing methods, we propose $\textit{Seed}$, by bridging $\underline{\textit{se}}$quenc$\underline{\textit{e}}$ and $\underline{\textit{d}}$iffusion models for trajectory generation. In particular, Seed adopts a \textit{conditional diffusion structure}, where a Transformer models the movement of each trajectory along the road segments, and conditioned on the Transformer's output, a diffusion model recovers the next road segment from random noise. The rationale is that the Transformer captures sequential movement patterns for regularity and consistency, while the diffusion model introduces diversity by recovering from noise. To train Seed, we adopt Node2vec to learn embeddings for the road segments to prepare model inputs, supervise learning using the task of trajectory reconstruction, and design a curriculum learning strategy to accelerate convergence. We compare Seed with 8 state-of-the-art trajectory generation methods on 3 datasets, and the results show that Seed improves the best-performing baseline by over 50\%.

Poster
TimeChain: A Secure and Decentralized Off-chain Storage System for IoT Time Series Data

Yixiao Teng, jiamei lv, Ziping Wang, Yi Gao, Wei Dong

展开摘要

Blockchain-based distributed storage systems offer enhanced security, transparency, and lower costs compared to traditional centralized storage, making them ideal for peer-to-peer collaboration. However, with the trend towards the Web of Things (WoT), lower transaction speeds and higher computational requirements limit their access to high-density data such as IoT. To address this, we propose TimeChain, an efficient off-chain blockchain storage system for IoT time series data. TimeChain batches discrete time series data, storing only the hash value of each batch on-chain while keeping the complete data off-chain. This significantly reduces storage overhead on the blockchain and storage latency by 37.4 times. In order to reduce the additional transmission latency in range queries, TimeChain employs an adaptive packaging mechanism. We convert the batching problem to a graph partitioning problem by representing data and historical co-query as graph vertices and edge weights respectively. To reduce the size of the transmission size in data integrity verification, a Locality-Sensitive Hashing (LSH)-based data integrity verification mechanism, which minimizes the data required for integrity checks by transmitting only non-redundant parts. TimeChain also integrates a node selection mechanism based on consensus protocol, which reduces the overhead by combining node selection and consensus processes. Our evaluation shows a reduction in query latency by 64.6% and storage latency by 35.3% compared to existing systems.

Poster
Path-LLM: A Multi-Modal Path Representation Learning by Aligning and Fusing with Large Language Models

Yongfu Wei, Yan Lin, Hongfan Gao, Ronghui Xu, Sean Bin Yang, Jilin Hu

展开摘要

The advancement of intelligent transportation systems has led to a growing demand for accurate path representations, which are essential for tasks such as travel time estimation, path ranking, and trajectory analysis. However, traditional path representation learning (PRL) methods often focus solely on single-modal road network data, overlooking important physical and regional factors that influence real-world traffic dynamics. To overcome this limitation, we introduce Path-LLM, a multi-modal path representation learning model that integrates large language models (LLMs) into PRL. Our approach leverages LLMs to interpret both topological and textual data, enabling robust multi-modal path representations. To effectively align and merge these modalities, we propose TPalign, a contrastive learning-based pretraining strategy that ensures alignment within the embedding space. We then present TPfusion, a multimodal fusion module that dynamically adjusts the weight of each modality before integration. To further optimize LLM training, we introduce a \textit{Two-stage Overlapping Curriculum Learning (TOCL)} approach, which progressively increases the complexity of the training data. Finally, we evaluate Path-LLM on two real-world datasets across traditional PRL downstream tasks, achieving up to a 61.84\% improvement in path ranking performance on the Xi'an dataset. Additionally, Path-LLM demonstrates superior performance in both few-shot and zero-shot learning scenarios. Our code is available at: https://anonymous.4open.science/r/Path-LLM-F053.

Poster
Posted Price Mechanisms for Online Allocation with Diseconomies of Scale

Hossein Nekouyan Jazi, Bo Sun, Raouf Boutaba, Xiaoqi Tan

展开摘要

This paper addresses the online $k$-selection problem with diseconomies of scale (OSDoS), where a seller seeks to maximize social welfare by optimally pricing items for sequentially arriving buyers, accounting for increasing marginal production costs. Previous studies have investigated deterministic dynamic pricing mechanisms for such settings. However, significant challenges remain, particularly in achieving optimality with small or finite inventories and developing effective randomized posted price mechanisms. To bridge this gap, we propose a novel randomized dynamic pricing mechanism for OSDoS, providing a tighter lower bound on the competitive ratio compared to prior work. Our approach ensures optimal performance in small inventory settings (i.e., when $k$ is small) and surpasses existing online mechanisms in large inventory settings (i.e., when $k$ is large), leading to the best-known posted price mechanism for optimizing online selection and allocation with diseconomies of scale across varying inventory sizes.

Poster
Semi-Supervised Anomaly Detection through Denoising-Aware Contrastive Distance Learning

Jianling Gao, Chongyang Tao, Zhenchao Sun, Xiya Jiang, Shuai Ma

展开摘要

Semi-supervised anomaly detection (AD) has garnered growing attention due to its ability to effectively combine limited labeled data with abundant unlabeled data. However, current methods of-ten impose artificial constraints on the proportion of unlabeled anomalies in the training set or overlook potential noise from these anomalies, thereby impeding the effective training of models for anomaly detection in real-world scenarios where several anomalies may be present in the unlabeled dataset. Additionally, existing methods often struggle to effectively exploit and model the complex relationships between data instances, which is critical for learning more discriminative features and accurate distance measures. Distance-based methods, in particular, typically rely on Euclidean distance metric, which lacks the flexibility to capture complex correlations across different data dimensions. To address above challenges, we propose CAD, a denoising-aware Contrastive distance learning framework for semi-supervised AD. It introduces a contrastive training objective to facilitate the learning of distinctive representations by contrasting the average distance between anomalies and unlabeled samples. To fully exploit the information from the unlabeled data meanwhile mitigate the effects of noise, we incorporate a two-stage anomaly denoising and expansion strategy to refine the dataset by identifying high-confidence samples from the unlabeled set. Furthermore, we employ a parameterized bilinear tensor distance layer to learn a customized distance metric, enabling the model to capture intricate relationships among data points. Extensive experiments on 10 real-world datasets demonstrate that CAD significantly outperforms existing semi-supervised AD models. Code available at https://github.com/CADrepo/CAD.

Poster
Preserving Label Correlation for Multi-label Text Classification by Prototypical Regularizations

Fanshuang Kong, Richong Zhang, Xiaohui Guo, Junfan Chen, Ziqiao Wang

展开摘要

Multi-label text classification (MLTC) aims to assign multiple relevant labels to a given sentence. An inherent challenge of MLTC is capturing label correlations compared with multi-class text classification. Existing MLTC models primarily focus on leveraging correlation information but often overlook the common issue of overfitting. Meanwhile, plug-and-play regularization methods struggle to preserve correlations effectively. In this paper, we distinguish two types of label correlations: explicit co-occurring correlation and implicit semantic correlations, and propose two regularization methods based on prototypical label embeddings for two correlation preservation, respectively. Specifically, we first generate the prototypical label embedding of multiple co-occurred labels as an intermediate. We then apply a prototypical label regularization on the distance between the sentence embedding and corresponding prototypical label embedding to alleviate the over-alignment issue caused by binary cross entropy loss and facilitate explicit correlation preservation. We finally extend the vanilla Mixup, which solely mixes multi-hot labels, on prototypical label embedding mixing to promote implicit correlation preservation. Empirical studies show the effectiveness of our regularization methods.

Poster
Paths-over-Graph: Knowledge Graph Enpowered Large Language Model Reasoning

Xingyu Tan, Xiaoyang Wang, Qing Liu, Xiwei Xu, Xin Yuan, Wenjie Zhang

展开摘要

Large Language Models (LLMs) have achieved impressive results in various tasks but struggle with hallucination problems and lack of relevant knowledge, especially in deep complex reasoning and knowledge-intensive tasks. Knowledge Graphs (KGs), which capture vast amounts of facts in a structured format, offer a reliable source of knowledge for reasoning. However, existing KG-based LLM reasoning methods face challenges like handling multi-hop reasoning, multi-entity questions, and effectively utilizing graph structures. To address these issues, we propose Paths-over-Graph (PoG), a novel method that enhances LLM reasoning by integrating knowledge reasoning paths from KGs, improving the interpretability and faithfulness of LLM outputs. PoG tackles multi-hop and multi-entity questions through a three-phase dynamic multi-hop path exploration, which combines the inherent knowledge of LLMs with factual knowledge from KGs. In order to improve the efficiency, PoG prunes irrelevant information from the graph exploration first and introduces efficient three-step pruning techniques that incorporate graph structures, LLM prompting, and a pre-trained language model (e.g., SBERT) to effectively narrow down the explored candidate paths. This ensures all reasoning paths contain highly relevant information captured from KGs, making the reasoning faithful and interpretable in problem-solving. PoG innovatively utilizes graph structure to prune the irrelevant noise and represents the first method to implement multi-entity deep path detection on KGs for LLM reasoning tasks. Comprehensive experiments on five benchmark KGQA datasets demonstrate PoG outperforms the state-of-the-art method ToG across GPT-3.5-Turbo and GPT-4, achieving an average accuracy improvement of 18.9\%. Notably, PoG with GPT-3.5-Turbo surpasses ToG with GPT-4 by up to 23.9\%.

Poster