Design and evaluation of an ontology-based tool for generating multiple-choice questions

© 2020 Emerald Publishing Limited. This accepted manuscript is deposited under the Creative Commons Attribution Non-commercial International Licence 4.0 (CC BY-NC 4.0). Any reuse is allowed in accordance with the terms outlined by the licence, here: https://creativecommons.org/licenses/by-nc/4.0/. To reuse the AAM for commercial purposes, permission should be sought by contacting [email protected]: The recent rise in online knowledge repositories and use of formalism for structuring knowledge, such as ontologies, has provided necessary conditions for the emergence of tools for generating knowledge assessment. These tools can be used in a context of interactive computer-assisted assessment (CAA) to provide a cost-effective solution for prompt feedback and increased learner’s engagement. The purpose of this paper is to describe and evaluate a tool developed by the authors, which generates test questions from an arbitrary domain ontology, based on sound pedagogical principles encapsulated in Bloom’s taxonomy. Design/methodology/approach: This paper uses design science as a framework for presenting the research. A total of 5,230 questions were generated from 90 different ontologies and 81 randomly selected questions were evaluated by 8 CAA experts. Data were analysed using descriptive statistics and Kruskal–Wallis test for non-parametric analysis of variance.FindingsIn total, 69 per cent of generated questions were found to be useable for tests and 33 per cent to be of medium to high difficulty. Significant differences in quality of generated questions were found across different ontologies, strategies for generating distractors and Bloom’s question levels: the questions testing application of knowledge and the questions using semantic strategies were perceived to be of the highest quality. Originality/value: The paper extends the current work in the area of automated test generation in three important directions: it introduces an open-source, web-based tool available to other researchers for experimentation purposes; it recommends practical guidelines for development of similar tools; and it proposes a set of criteria and standard format for future evaluation of similar systems.Peer reviewedFinal Accepted Versio

Ill-Formed Text and Conceptual Processing

In this paper, we discuss the problem of ill-formed (or incorrectly processed) text in the context of conceptual analysis text processing systems. We show that syntactically ill-formed text is not a major problem for such systems. Conceptually ill-formed text and conceptually ill-formed representations of text do cause interesting problems. We define conceptual ill-formedness and then present ideas for how it can handled in the context of two text processing systems, IPP and RESEARCHER

Ill-Formed Text and Conceptual Processing

In this paper, we discuss the problem of ill-formed (or incorrectly processed) text in the context of conceptual analysis text processing systems. We show that syntactically ill-formed text is not a major problem for such systems. Conceptually ill-formed text and conceptually ill-formed representations of text do cause interesting problems. We define conceptual ill-formedness and then present ideas for how it can handled in the context of two text processing systems, IPP and RESEARCHER

Neo-Thomism and the Problem of Animal Suffering

Proponents of the problem of animal suffering claim that the millions of years of apparent nonhuman animal pain and suffering provides evidence against the existence of God. Neo-Cartesianism attempts to avoid this problem mainly by denying the existence of phenomenal consciousness in nonhuman animals. However, neo-Cartesian options regarding animal minds have failed to compel many. In this essay, I explore an answer to the problem of animal suffering inspired by the medieval theologian Thomas Aquinas. Instead of focusing on phenomenal consciousness, the neo-Thomistic view of animal minds focuses on self-awareness. After proposing and providing evidence for this view, I conclude that nonhuman animal suffering is not morally significant