20 October 2021 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Knowledge graph embedding models have gained significant attention in AI research. The aim of knowledge graph embedding is to embed the graphs into a vector space in which the structure of the graph is preserved. Recent works have shown that the inclusion of background knowledge, such as logical rules, can improve the performance of embeddings in downstream machine learning tasks. However, so far, most existing models do not allow the inclusion of rules. We address the challenge of including rules and present a new neural based embedding model (LogicENN). We prove that LogicENN can learn every ground truth of encoded rules in a knowledge graph. To the best of our knowledge, this has not been proved so far for the neural based family of embedding models. Moreover, we derive formulae for the inclusion of various rules, including (anti-)symmetric, inverse, irreflexive and transitive, implication, composition, equivalence and negation. Our formulation allows to avoid grounding for implication and equivalence relations. Our experiments show that LogicENN outperforms the existing models in link prediction.
18 July 2022 in IEEE Access
Knowledge graph embedding models have become a popular approach for knowledge graph completion through predicting the plausibility of (potential) triples. This is performed by transforming the entities and relations of the knowledge graph into an embedding space. However, knowledge graphs often include further textual information stored in literal, which is ignored by such embedding models. As a consequence, the learning process stays limited to the structure and the connections between the entities, which has the potential to negatively influence the performance. We bridge this gap by leveraging the capabilities of pre-trained language models to include textual knowledge in the learning process of embedding models. This is achieved by introducing a new loss function that guides embedding models in measuring the likelihood of triples by taking such complementary knowledge into consideration. The proposed solution is a model-independent loss function that can be plugged into any knowledge graph embedding model. In this paper, Sentence-BERT and fastText are used as pre-trained language models from which the embeddings of the textual knowledge are obtained and injected into the loss function. The loss function contains a trainable slack variable that determines the degree to which the language models influence the plausibility of triples. Our experimental evaluation on six benchmarks, namely Nations, UMLS, WordNet, and three versions of CodEx confirms the advantage of using pre-trained language models for boosting the accuracy of knowledge graph embedding models. We showcase this by performing evaluations on top of the five well-known knowledge graph embedding models such as TransE, RotatE, ComplEx, DistMult, and QuatE. The results show an improvement in accuracy up to 9% on UMLS dataset for the Distmult model and 4.2% on the Nations dataset for the ComplEx model when they are guided by pre-trained language models. We additionally studied the effect of multiple factors such as the structure of the knowledge graphs and training steps and presented them as ablation studies.
27 October 2020 in IEEE Access
Knowledge Graph Embeddings (KGE) are used for representation learning in Knowledge Graphs (KGs) by measuring the likelihood of a relation between nodes. Rotation-based approaches, specially axis-angle representations, were shown to improve the performance of many Machine Learning (ML)-based models in different tasks including link prediction. There is a perceived disconnect between the topics of KGE models and axis-angle rotation-based approaches. This is particuarly visible when considering the ability of KGEs to learn relational patterns such as symmetry, inversion, implication, equivalence, composition, and reflexivity considering axis-angle rotation-based approaches. In this article, we propose RodE , a new KGE model which employs an axis-angle representation for rotations based on Rodrigues’ formula. RodE inherits the main advantages of 3-dimensional rotation from angle, orientation and distance preservation in the embedding space. Thus, the model efficiently captures the similarity between the nodes in a graph in the vector space. Our experiments show that RodE outperforms state-of-the-art models on standard datasets.