Hierarchical graphs are a frequent solution for capturing symbolic data due the importance of hierarchies for defining knowledge. In these graphs, relations among elements may contain large portions of the element’s semantics. However, knowledge discovery based on analyzing the patterns of hierarchical relations is rarely used. We outline four inference based algorithms exploiting semantic properties of hierarchically represented knowledge for producing new links, and test one of them on a generalization of Cyc’s KB. Finally, we argue why such algorithms can be useful for unsupervised learning and supervised analysis of a KBPeer ReviewedPostprint (author’s final draft

Cortés García, Claudio Ulises

Garcia Gasulla, Dario

English

UPCommons. Portal del coneixement obert de la UPC

Hierarchical Inference Applied to CycDario GARCIA-GASULLA a Ulises CORTE´S ae-mail: {dariog,ia}@lsi.upc.edua KEMLg, Universitat Polite`cnica de CatalunyaAbstract. Hierarchical graphs are a frequent solution for capturing symbolic datadue the importance of hierarchies for defining knowledge. In these graphs, relationsamong elements may contain large portions of the element’s semantics. However,knowledge discovery based on analyzing the patterns of hierarchical relations israrely used. We outline four inference based algorithms exploiting semantic proper-ties of hierarchically represented knowledge for producing new links, and test oneof them on a generalization of Cyc’s KB. Finally, we argue why such algorithmscan be useful for unsupervised learning and supervised analysis of a KB.1. IntroductionHierarchical graphs (HG) include generalization and specialization relations usuallyfound in taxonomies and ontologies. These graphs are frequently used for KnowledgeRepresentation (KR) (e.g., ontologies, social networks, semantic web data, etc.), but thereare still few knowledge discovery techniques for this type of structure[3]. Hierarchicalproperties, such as inheritance and abstraction, are powerful tools for KR and knowledgediscovery. These properties however are difficult to capture through the components of agraph which has motivated the use of logics for that purpose (FOL, Inductive Logic Pro-gramming, etc.). Some research fields have developed learning tools for HG[3], but noneof these approaches use hierarchical semantics as basis for inferential learning. Someconsider inheritance, but focus on spatiotemporal relations through Bayesian inference[1]. Some apply logic inference instead as its basis for learning [4]. In that context, ourhypothesis is that features provided by strong generalization and inheritance propertiescapture enough of the data semantics as to empower inference to achieve learning.In this paper we outline algorithms based on inference to be applied on a HG. Givena graph’s topology and assuming properties of hierarchical knowledge the algorithms usethe principles of inference to predict edges. These algorithms are an unsupervised learn-ing methodology for HG. The results can also be used in a supervised manner to detectflaws in KBs (e.g., inconsistencies and unbalances). We test one of these algorithms ona generalized version of Cyc KB, and overview the results.2. BackgroundConceptual Clustering obtains a classification from a set of data descriptions[6], whichmay include a hierarchy. These methods work on weakly structured data and use weakinterpretations of hierarchy (e.g., no multiple inheritance). [2] presents a methodologyfor hierarchy discovery. Given a graph, it produces a set of hierarchies ranked by like-liness, turning any graph into a HG. Assuming the importance of hierarchy for definingthe data, it predicts links according to it. Its missing links prediction is based on its pro-posed hierarchy, meaning it can only be applied to graphs hierarchized by their method.DeSTIN[1] is a Deep Learning System (DLS) which builds a hierarchy of nodes in whicheach layer represents a different spatiotemporal scale. Each node includes a set of pat-terns and of belief states. DLS are most suitable for uniformly and statically distributeddata (e.g., sensory information), but may not be appropriate for knowledge without astable hierarchical pattern (e.g. ontologies). Logic-based systems represent hierarchicalproperties through logic predicates, like Cyc[5] which includes components for reason-ing and learning. Cyc’s KB is an ontology represented hierarchically through collectionsin a graph like structure. Cyc’s predicates are strongly typed and logical deduction is itsmain source of reasoning. Opencog[4] KB is based on a hypergraph generalization whichdefines several types of vertices and edges. It uses a wide variety of methods for learn-ing, among which there is probabilistic logic inference, clustering and pattern mining.Other approaches like Link Prediction and Probabilistic Graphical Models are relevantcontributions to the field [3] but no not consider hierarchies for learning.3. Motivation and algorithmsWe argue that the topology of hierarchical edges among symbolic entities holds a largeportion of its meaning and that it can be used to infer knowledge about those entities.We implement algorithms for extending a KB’s only using its distribution of hierarchicalrelations based on an unlabeled, directed graph in which there is a single type of edgeand node. For any two nodes linked in the graph, we will refer to the node source ofthe relation as the child and to the node destination as the parent. The relevancy of thismodel is the achievement of inferential reasoning without using higher level tools suchas logics. While logics allow a more accurate and rich approach to inference, they partlydetach the KR and reasoning from the graph. Our proposal works only with knowledgefound in the graph. In the current context with large hierarchical KBs available this workcan contribute to the construction, enrichment and evaluation of these KBs.The algorithms we define apply the principles of inference to hierarchies. Deduc-tively, one may say that a child node inherits the edges from its parent node. This spe-cialization process can be implemented through a top-down transitivity closure for edges(i.e., the edges of each parent are directly inherited by each children), and is frequentlyfound in most hierarchical datasets. Inductively, one may say that the relations of a par-ent are a generalization of the relations of all its children. This generalization processcan be implemented through a bottom-up weighted transitivity for edges. To evaluate theexistence of the edge A→ B we consider how many of A’s children are also children ofB and use that data as weighted evidence. Finally, abductively one can find parenthoodrelations which explain the existence of other relations. Given B → A and C → A, anexplanation for the latter could be a newly proposed relation C → B. We define two ofthose algorithms, the first proposes the edge C → B given that most of A children arealso of B (and therefore by induction also C). The second one proposes the edge C → Bgiven C shares a pattern of edges which is shared by B but not by most children of A.More details on the algorithms can be found in [3].car land transportation device object underneath which a human can move pastwheeled vehicle single purpose device self-powered vehicle man-made pollution sourceTable 1. Sample of induced parents for automobile model4. Testing on CycWe test our algorithms on OpenCyc, the open version of Cyc’s KB, which contains 120KClass concepts and 537K relations among them. To transform it into a HG we kept thetwo most common relations, rdf:type and rdfs:subClassOf since both their meanings arehierarchical. The rest of relations were omitted but we still work with most of the KB(these two relations represent over 80% of all edges). We unlabel them and consider theresultant relation as a unique transitive (through deduction) relation. After applying thesechanges the resultant HG is composed by 116K nodes and 354K edges.Due to space limitation we will present only the results obtained by the inductivealgorithm. Further analysis on the algorithms precision can be found in [3]. For the in-ductive algorithm we considered a 99% uncertainty threshold (i.e., 99% children of nhad to share an edge for it to be induced by n). We added further restrictions regard-ing the specificity level of the candidate nodes to avoid too abstract inductions and toosimilar inferences[3]. The algorithm produced 2.181 relations from induction and 2.152from applying deduction to the newly induced edges. As an overview of the results letus focus on the inductions produced for the element automobile model. Originally, thenode had 7 transitive parents (e.g., type of object, facet collection, road vehicle model,etc.) and 631 transitive children (e.g., Volvo 140). 17 additional parents were discoveredby induction (see a small sample in Table 1) and 85 new relations were produced withdeduction for its children. One of the discovered parent was car, which was captured inOpenCyc’s KB through an ObjectProperty by a generalizing predicate. Although this re-lation was deleted as only Class-to-Class relations were kept, the algorithm was capableof re-discovering it from the patterns of automobile model children.In the original KB, 6 of the 17 induced parents were related with car and thereforewith automobile model, but those relations were deleted in the adaptation process. Thealgorithm rebuilt those relations from patterns of automobile model children. This im-plies that the children of an element contain information regarding the element’s prop-erties, information which can be used to produce new data. The remaining 10 elementswere not directly related with car in OpenCyc’s KB, meaning the algorithm producedinformation which was not directly found on the original KB. The relations added byapplying deduction to the induced edges are less relevant since they are a direct effectof the induced relations. A brief example are elements AcuraCL and ChevyS10Blazer,children of automobile model which by deduction are now children of the 17 new parentsof automobile model, including those of Table 1. Finally, in Table 2 we present a sampleof other inductions which we consider to be semantically interesting.5. Conclusions and discussionIn this paper we propose the use of algorithms inspired on inferential reasoning for learn-ing in HG. For that purpose it is necessary to assume that hierarchical relations representboth a top-down specialization and a bottom-up generalization of concepts. We appliedcoral reef - natural thing the smell of chemically acidic substances - rust causing stuffpure breed - tame animal unalloyed metal - stuff composed entirely of one elementbeheading - hostile actions mass cell division -event in which some quantity changesgroup of sports events - conflict typical artillery target type - high payoff target typemedical school graduate - skilled worker logical feature of binary predicates - truth functionnew years resolution - agreement topic product with warranty - repairable productTable 2. Sample of inductions. Left element of cell is induced to be a children of the right element of cellthese algorithms on a directed, unlabeled graph, and tested it on a generalization of arich KB to study the type of relations we can infer. The results seem to indicate that suchmethodologies can be useful to produce additional information about concepts withoutthe use of exogeneous inferential knowledge such as logics. At the same time, the uni-versality of inference allows the algorithms to produce unbounded and complex learningwhich is hard to obtain through statistical methods.To the best of our knowledge no other contribution in the current state of the art com-bines hierarchical features with inference in a similar manner. These algorithms couldbe applied to any KB which can be reduced to a HG, the relations of which satisfy theproperties of hierarchically represented knowledge. The results proved the approach tobe highly dependent on the KB’s topology and on the existence of irrelevant relationsand sparsity on the graph. The method is an unsupervised learning approach for seman-tic learning which produces new relations. Additionally, by supervising the results thealgorithms can be used to improve the contents and topology of a KB by finding incon-sistencies and unbalances in it. Both functionalities are currently appealing as there aremany large symbolic datasets available which could benefit from it.6. AcknowledgmentsThanks to the ACIA fellowship for assistance to CCIA’2013.References[1] I. Arel, D. Rose, and R. Coop. Destin: A scalable deep learning architecture with application to high-dimensional robust pattern recognition. In Proc. AAAI Workshop on Biologically Inspired CognitiveArchitectures, pages 1150–1157, 2009.[2] A. Clauset, C. Moore, and M. E. Newman. Hierarchical structure and the prediction of missing links innetworks. Nature, 453(7191):98–101, 2008.[3] D. Garcia-Gasulla and U. Corte´s. Hierarchical Link Prediction and Reasoning. PhD thesis, Proposal,2013.[4] B. Goertzel. OpenCog Prime: Design for a Thinking Machine. http://opencog. org, 2008.[5] D. Lenat, R. Guha, K. Pittman, M. Shepherd, and D. Pratt. Cyc: toward programs with common sense.Communications of the ACM, 33(8):30–49, 1990.[6] R. Michalski. Knowledge acquisition through conceptual clustering: A theoretical framework and analgorithm for partitioning data into conjunctive concepts. Journal of Policy Analysis and InformationSystems, 4(3):219–244, 1980.

Hierarchical inference applied to Cyc

Hierarchical graphs are a frequent solution for capturing symbolic data due the importance of hierarchies for defining knowledge. In these graphs, relations among elements may contain large portions of the element’s semantics. However, knowledge discovery based on analyzing the patterns of hierarchical relations is rarely used. We outline four inference based algorithms exploiting semantic properties of hierarchically represented knowledge for producing new links, and test one of them on a generalization of Cyc’s KB. Finally, we argue why such algorithms can be useful for unsupervised learning and supervised analysis of a KBPeer Reviewe

UPCommons

https://upcommons.upc.edu/bitstream/2117/22882/1/garcia-gasulla.dario.pdf

Hierarchical inference applied to Cyc

Abstract

Similar works

Full text

Available Versions

UPCommons. Portal del coneixement obert de la UPC

UPCommons