Incorporating Relational Knowledge into Word Representations using Subspace Regularization

Abstract

Incorporating lexical knowledge from semantic resources (e.g., WordNet ) has been shown to improve the quality of distributed word representations. This knowledge often comes in the form of relational triplets ( x, r, y ) where words x and y are connected by a relation type r . Existing methods either ignore the relation types, essentially treating the word pairs as generic related words, or employ rather restrictive assumptions to model the relational knowledge. We propose a novel approach to model relational knowledge based on low-rank subspace regularization, and conduct experiments on standard tasks to evaluate its effectiveness.