Fangrong Wang

Fangrong Wang

Auto-generated profile, based on publicly available information

Fangrong Wang

School of Informatics, University of Edinburgh, United Kingdom

Research Focus & Metrics

Representative Publications

View All Publications
15 December 2023
Verified
Conference Paper
Open access
RZRuiqi Zhu
Ruiqi Zhu
ABAlan Bundy
Alan Bundy
Jeff Z.  PanJeff Z. Pan
Jeff Z. Pan
KNKwabena Nuamah
Kwabena Nuamah
FWFangrong Wang
Fangrong Wang
SMStefano Mauceri
Stefano Mauceri
Knowledge Graph (KG) Construction is the prerequisite for all other KG research and applications. Researchers and engineers have proposed various approaches to build KGs for their use cases. However, how can we know whether our constructed KG is good or bad? Is it correct and complete? Is it consistent and robust? In this paper, we propose a method called LP-Measure to assess the quality of a KG via a link prediction tasks, without using a gold standard or other human labour. Though theoretically, the LP-Measure can only assess consistency and redundancy, instead of the more desirable correctness and completeness, empirical evidence shows that this measurement method can quantitatively distinguish the good KGs from the bad ones, even in terms of incorrectness and incompleteness. Compared with the most commonly used manual assessment, our LP-Measure is an automated evaluation, which saves time and human labour.

27 October 2022
Verified
Conference Paper
Open access
RZRuiqi Zhu
Ruiqi Zhu
FWFangrong Wang
Fangrong Wang
ABAlan Bundy
Alan Bundy
XLXue Li
Xue Li
KNKwabena Nuamah
Kwabena Nuamah
Jeff Z.  PanJeff Z. Pan
Jeff Z. Pan
When the estimated probabilities do not match the relative frequencies, we say these estimated probabilities are uncalibrated [39], which may cause incorrect decision making, and is particularly undesired in high-stakes tasks [45]. Knowledge Graph embedding models are reported to produce uncalibrated probabilities [36], e.g., for all the triples predicted with probability 0.9, the percentage of them being truly correct triples is not . In this article, we take a closer look at this problem. First, we confirmed the issue that typical KG Embedding models are uncalibrated. Then, we show how off-the-shelf calibration techniques can be used to mitigate this issue, among which binning-based calibration produces more calibrated probabilities. We also investigated the possible reasons for the uncalibrated probabilities and found that the expit transform, the way used to convert embedding scores into probabilities, is ineffective in most cases.