This repository contains codes, experiment scripts, and datasets to reproduce results of the following papers:
Realistic Re-evaluation of Knowledge Graph Completion Methods: An Experimental Study Farahnaz Akrami, Mohammed Samiul Saeef, Qingheng Zhang, Wei Hu, and Chengkai Li, SIGMOD 2020
Re-evaluating Embedding-Based Knowledge Graph Completion Methods Farahnaz Akrami, Lingbing Guo, Wei Hu, and Chengkai Li, CIKM 2018
A summary of the paper is published in a Medium blog post.
To find Cartesian Product relations in FB15k and perform link prediction on these relations:
python fb15k_cartesian_product.py
To find the reverse and duplicate relations in FB15k:
python FB15k_redundancy.py
To find redundant information in test set of FB15k:
python FB15k_test_redundancy.py
Implementations for conducting link prediction using rules generated by AMIE.
Head and tail entity link prediction for a specific test relation:
python left.py relation_id
python right.py relation_id
Results obtained for each test relation are stored in ./AMIE_LinkPrediction_results.
To have the results of link prediction using AMIE:
python combine_res.py
To obtain the link prediction results for each test relation of FB15k-237, WN18RR, and YAGO3-10:
python result_by_rel_all.py
To calculate the head and tail entity link prediction results for different relation types:
python result_by_relcategory.py
Percentage of triples on which each method outperforms others, separately for each relation:
python heatmap.py
Percentage of test triples with better performance than TransE that have reverse and duplicate triples in training set:
python reverse_percentage.py
To obtain the number of relations on which each model is the most accurate and charts:
python detailed_results.py
- Cloned October 2018.
- Implementations for ComplEx and DistMult.
- Original repository: (https://github.com/ttrouill/complex).
- evaluation.py is modified to calculate raw metrics.
- Cloned October 2019
- Implementations for ConvE.
- Original repository: (https://github.com/TimDettmers/ConvE).
- evaluation.py is modified to print results for each test triple and calculate raw metrics.
- Cloned October 2019.
- Implementations for TransE.
- Original repository: (https://github.com/thunlp/OpenKE).
- Test.h is modified to print results for each test triple.
- Cloned October 2019.
- Implementations for RotatE.
- Original repository: (https://github.com/DeepGraphLearning/KnowledgeGraphEmbedding).
- Modified to print results for each test triple.
- Cloned October 2019.
- Implementations for TuckER.
- Original repository: (https://github.com/ibalazevic/TuckER).
- Modified to print results for each test triple.