Measuring the understandability of deduction rules for OWL

Debugging OWL ontologies can be aided with automated reasoners that generate entailments, including undesirable ones. This information is, however, only useful if developers understand why the entailments hold. To support domain experts (with limited knowledge of OWL), we are developing a system that explains, in English, why an entailment follows from an ontology. In planning such explanations, our system starts from a justification of the entailment and constructs a proof tree including intermediate statements that link the justification to the entailment. Proof trees are constructed from a set of intuitively plausible deduction rules. We here report on a study in which we collected empirical frequency data on the understandability of the deduction rules, resulting in a facility index for each rule. This measure forms the basis for making a principled choice among alternative explanations, and identifying steps in the explanation that are likely to require extra elucidation

Predicting the understandability of OWL inferences

In this paper, we describe a method for predicting the understandability level of inferences with OWL. Specifically, we present a model for measuring the understandability of a multiple-step inference based on the measurement of the understandability of individual inference steps. We also present an evaluation study which confirms that our model works relatively well for two-step inferences with OWL. This model has been applied in our research on generating accessible explanations for an entailment of OWL ontologies, to determine the most understandable inference among alternatives, from which the final explanation is generated

Generating Natural Language Explanations For Entailments In Ontologies

Building an error-free and high-quality ontology in OWL (Web Ontology Language)---the latest standard ontology language endorsed by the World Wide Web Consortium---is not an easy task for domain experts, who usually have limited knowledge of OWL and logic. One sign of an erroneous ontology is the occurrence of undesired inferences (or entailments), often caused by interactions among (apparently innocuous) axioms within the ontology. This suggests the need for a tool that allows developers to inspect why such an entailment follows from the ontology in order to debug and repair it. This thesis aims to address the above problem by advancing knowledge and techniques in generating explanations for entailments in OWL ontologies. We build on earlier work on identifying minimal subsets of the ontology from which an entailment can be drawn---known technically as justifications. Our main focus is on planning (at a logical level) an explanation that links a justification (premises) to its entailment (conclusion); we also consider how best to express the explanation in English. Among other innovations, we propose a method for assessing the understandability of explanations, so that the easiest can be selected from a set of alternatives. Our findings make a theoretical contribution to Natural Language Generation and Knowledge Representation. They could also play a practical role in improving the explanation facilities in ontology development tools, considering especially the requirements of users who are not expert in OWL

Using Insights from Psychology and Language to Improve How People Reason with Description Logics

Inspired by insights from theories of human reasoning and language, we propose additions to the Manchester OWL Syntax to improve comprehensibility. These additions cover: functional and inverse functional properties, negated conjunction, the definition of exceptions, and existential and universal restrictions. By means of an empirical study, we demonstrate the effectiveness of a number of these additions, in particular: the use of solely to clarify the uniqueness of the object in a functional property; the replacement of and with intersection in conjunction, which was particularly beneficial in negated conjunction; the use of except as a substitute for and not; and the replacement of some with including and only with noneOrOnly, which helped in certain situations to clarify the nature of these restrictions

The usability of description logics: understanding the cognitive difficulties presented by description logics

Description Logics have been extensively studied from the viewpoint of decidability and computational tractability. Less attention has been given to their usability and the cognitive difficulties they present, in particular for those who are not specialists in logic. This paper reports on a study into the difficulties associated with the most commonly used Description Logic features. Psychological theories are used to take account of these. Whilst most of the features presented no difficulty to participants, the comprehension of some was affected by commonly occurring misconceptions. The paper proposes explanations and remedies for some of these difficulties. In addition, the time to confirm stated inferences was found to depend both on the maximum complexity of the relations involved and the number of steps in the argument

Using ontologies: understanding the user experience

Drawing on 118 responses to a survey of ontology use, this paper describes the experiences of those who create and use ontologies. Responses to questions about language and tool use illustrate the dominant position of OWL and provide information about the OWL profiles and particular Description Logic features used. The paper suggests that further research is required into the difficulties experienced with OWL constructs, and with modelling in OWL. The survey also reports on the use of ontology visualization software, finding that the importance of visualization to ontology users varies considerably. This is also an area which requires further investigation. The use of ontology patterns is examined, drawing on further input from a follow-up study devoted exclusively to this topic. Evidence suggests that pattern creation and use are frequently informal processes and there is a need for improved tools. A classification of ontology users into four groups is suggested. It is proposed that the categorisation of users and user behaviour should be taken into account when designing ontology tools and methodologies. This should enable rigorous, user-specific use cases

Deductive reasoning about expressive statements using external graphical representations

Research in psychology on reasoning has often been restricted to relatively inexpressive statements involving quantifiers. This is limited to situations that typically do not arise in practical settings, such as ontology engineering. In order to provide an analysis of inference, we focus on reasoning tasks presented in external graphic representations where statements correspond to those involving multiple quantifiers and unary and binary relations. Our experiment measured participants’ performance when reasoning with two notations. The first used topology to convey information via node-link diagrams (i.e. graphs). The second used topological and spatial constraints to convey information (Euler diagrams with additional graph-like syntax). We found that topological- spatial representations were more effective than topological representations. Unlike topological-spatial representations, reasoning with topological representations was harder when involving multiple quantifiers and binary relations than single quantifiers and unary relations. These findings are compared to those for sentential reasoning tasks

The Role of Application Domain Knowledge in Using OWL DL Diagrams: A Study of Inference and Problem-Solving Tasks

Diagrammatic conceptual schemas are an important part of information systems analysis and design. For effectively communicating domain semantics, modeling grammars have been proposed to create highly expressive conceptual schemas. One such grammar is the Web Ontology Language (OWL), which relies upon description logics (DL) as a knowledge representation mechanism. While an OWL DL diagram can be useful for representing domain semantics in great detail, the formal semantics of OWL DL places a burden on diagram users. This research investigates how user’s prior knowledge of the application domain impacts solving inference tasks as well as schema-based problem-solving tasks using OWL DL diagrams. Our empirical validation shows that application domain knowledge has no effect on inference performance but enhances schema-based problem-solving performance. We contribute to the conceptual modeling literature by studying task performance for a highly expressive modeling grammar and introducing inference tasks as a new task type