de Boer, V.

de Kleijn, M.T.M.

van Harmelen, F.A.H.

Wilcke, W.X.

English

VU Research Portal

VU Research PortalA Deep Neural Network for Link Prediction on Knowledge GraphsWilcke, W.X.; de Boer, V.; van Harmelen, F.A.H.; de Kleijn, M.T.M.2016document versionPublisher's PDF, also known as Version of recorddocument licenseCC BY-SALink to publication in VU Research Portalcitation for published version (APA)Wilcke, W. X., de Boer, V., van Harmelen, F. A. H., & de Kleijn, M. T. M. (2016). A Deep Neural Network for LinkPrediction on Knowledge Graphs. Abstract from ICT.OPEN 2016, Amersfoort, Netherlands.General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.            • Users may download and print one copy of any publication from the public portal for the purpose of private study or research.            • You may not further distribute the material or use it for any profit-making activity or commercial gain            • You may freely distribute the URL identifying the publication in the public portal ?Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.E-mail address:vuresearchportal.ub@vu.nlDownload date: 13. Dec. 2021A Deep Neural Network for Link Prediction onKnowledge GraphsWilcke W.X.1,2, de Boer V1, van Harmelen F.A.H.1, and de Kleijn M.T.M.21 Department of Computer Science2 Department of Spatial EconomicsVU University AmsterdamAmsterdam, The Netherlands{w.x.wilcke, v.de.boer, frank.van.harmelen, mtm.de.kleijn}@vu.nlRecent years have seen the emergence of graph-based Knowledge Bases buildupon Semantic Web technologies, known as Knowledge Graphs (KG). Popularexamples are DBpedia and LinkedGeoData, which offer semantically-annotatedand interconnected information extracted from Wikipedia and Open Street Map,respectively. Factual information stored in such KGs is encoded through relations(edges) between entities (vertices), both sets of which their members’ semanticsare strictly defined by shared ontological background knowledge. Due to thesecharacteristics, we are able to generate new hypotheses about this informationby predicting which links (i.e. relations and the entities they relate) are likely toexist given certain constraints.Currently-popular approaches to link prediction on KGs are Inductive LogicProgramming[1], logic and graph-based kernels [2, 3], and matrix and tensor fac-torization [4, 5]. As of late, deep Neural Networks (NN) are being considered aswell, mainly due to their recent successes on solving complex learning problems.Moreover, being a latent-feature model, they are rather robust towards noise andinconsistencies, as well as able to cope with large-scale and high-dimensional datasets. Despite these useful characteristics, only a handful have yet investigatedthe effectiveness of deep NNs on KGs [6–8], and even fewer have taken on thechallenge of exploiting the ontological background knowledge (e.g. graph fea-tures) for improved predictive performance, even though this has been provenuseful [9, 10]. We endeavour to fill this gap.Our study is investigating the effectiveness of a hybrid latent and graph-feature model capable of performing link prediction on real-world KGs. For thispurpose, we are employing a deep feedforward NN that uses autoencoders forpre-training purposes. Moreover, its output consists of a vector of probabilitiesthat correspond to all possible relations, whereas its input expects a concate-nated vector with each describing any two entities. This description vector isconstructed following one of several propositionalization strategies that we havedeveloped, and uses a sparse encoding of both entities’ local neighbourhood upto depth n.Initial results with KGs up to 15 thousand unique links indicate scalabilityissues, which we believe are caused by the large length of the currently-usedinput vector, as well as the yet sub-optimized network and training algorithms.To compensate, we kept the number training epoch low and fixed the learningrate at the rather high value of 0.1. As a consequence, we will refrain frompublishing evaluations, as we deem them an unreliably measure of our model’seffectiveness at present.For the near future, we are working on method to optimize our model’s hyper-parameters with the help of Bayesian optimization using Random Forrest as asurrogate model. To cope with the expected increase in required computationalresources, we intent to run future experiments on a parallel architecture. Finally,we are working to further improve our propositionalization strategies with theaim of achieving a better representation of an entity while additionally improvingscalability.Keywords: Knowledge Graphs·Semantic Web·Deep Learning·Neural Networks·PropositionalizationStrategiesReferences1. Rettinger, A., Lsch, U., Tresp, V., d’Amato, C., Fanizzi, N.: Mining the semanticweb. Data Mining and Knowledge Discovery 24(3) (2012) 613–6622. dAmato, C., Fanizzi, N., Esposito, F.: Classification and retrieval through semantickernels. In: Knowledge-Based Intelligent Information and Engineering Systems,Springer (2008) 252–2593. Losch, U., Bloehdorn, S., Rettinger, A.: Graph kernels for rdf data. Number 5781-5782 in Lecture notes in computer science, Lecture notes in artificial intelligence,Springer (2009)4. Nickel, M., Tresp, V., Kriegel, H.P.: Factorizing YAGO: scalable machine learningfor linked data. In: Proceedings of the 21st international conference on World WideWeb, ACM (2012) 271–2805. Franz, T., Schultz, A., Sizov, S., Staab, S.: Triplerank: Ranking semantic web databy tensor decomposition. Springer (2009)6. Socher, R., Perelygin, A., Wu, J.Y., Chuang, J., Manning, C.D., Ng, A.Y., Potts,C.: Recursive deep models for semantic compositionality over a sentiment tree-bank. In: Proceedings of the conference on empirical methods in natural languageprocessing (EMNLP). Volume 1631., Citeseer (2013) 16427. Huang, H., Heck, L., Ji, H.: Leveraging Deep Neural Networks and KnowledgeGraphs for Entity Disambiguation. arXiv preprint arXiv:1504.07678 (2015)8. Yu, D., Deng, L., Seide, F.: Large Vocabulary Speech Recognition Using DeepTensor Neural Networks. In: INTERSPEECH. (2012)9. Dong, X., Gabrilovich, E., Heitz, G., Horn, W., Lao, N., Murphy, K., Strohmann,T., Sun, S., Zhang, W.: Knowledge vault: A web-scale approach to probabilis-tic knowledge fusion. In: Proceedings of the 20th ACM SIGKDD internationalconference on Knowledge discovery and data mining, ACM (2014) 601–61010. Nickel, M., Jiang, X., Tresp, V.: Reducing the Rank in Relational FactorizationModels by Including Observable Patterns. In: Advances in Neural InformationProcessing Systems. (2014) 1179–1187

A Deep Neural Network for Link Prediction on Knowledge Graphs

http://hdl.handle.net/1871.1/ca63a27c-6121-48d2-bf29-a6b1283c8e58

A Deep Neural Network for Link Prediction on Knowledge Graphs

Abstract

Similar works

Full text

Available Versions

VU Research Portal