Design and evaluation of an ontology-based tool for generating multiple-choice questions

© 2020 Emerald Publishing Limited. This accepted manuscript is deposited under the Creative Commons Attribution Non-commercial International Licence 4.0 (CC BY-NC 4.0). Any reuse is allowed in accordance with the terms outlined by the licence, here: https://creativecommons.org/licenses/by-nc/4.0/. To reuse the AAM for commercial purposes, permission should be sought by contacting [email protected]: The recent rise in online knowledge repositories and use of formalism for structuring knowledge, such as ontologies, has provided necessary conditions for the emergence of tools for generating knowledge assessment. These tools can be used in a context of interactive computer-assisted assessment (CAA) to provide a cost-effective solution for prompt feedback and increased learner’s engagement. The purpose of this paper is to describe and evaluate a tool developed by the authors, which generates test questions from an arbitrary domain ontology, based on sound pedagogical principles encapsulated in Bloom’s taxonomy. Design/methodology/approach: This paper uses design science as a framework for presenting the research. A total of 5,230 questions were generated from 90 different ontologies and 81 randomly selected questions were evaluated by 8 CAA experts. Data were analysed using descriptive statistics and Kruskal–Wallis test for non-parametric analysis of variance.FindingsIn total, 69 per cent of generated questions were found to be useable for tests and 33 per cent to be of medium to high difficulty. Significant differences in quality of generated questions were found across different ontologies, strategies for generating distractors and Bloom’s question levels: the questions testing application of knowledge and the questions using semantic strategies were perceived to be of the highest quality. Originality/value: The paper extends the current work in the area of automated test generation in three important directions: it introduces an open-source, web-based tool available to other researchers for experimentation purposes; it recommends practical guidelines for development of similar tools; and it proposes a set of criteria and standard format for future evaluation of similar systems.Peer reviewedFinal Accepted Versio

Ontologies for automatic question generation

Assessment is an important tool for formal learning, especially in higher education. At present, many universities use online assessment systems where questions are entered manually into a question bank system. This kind of system requires the instructor’s time and effort to construct questions manually. The main aim of this thesis is, therefore, to contribute to the investigation of new question generation strategies for short/long answer questions in order to allow for the development of automatic factual question generation from an ontology for educational assessment purposes. This research is guided by four research questions: (1) How well can an ontology be used for generating factual assessment questions? (2) How can questions be generated from course ontology? (3) Are the ontological question generation strategies able to generate acceptable assessment questions? and (4) Do the topic-based indexing able to improve the feasibility of AQGen. We firstly conduct ontology validation to evaluate the appropriateness of concept representation using a competency question approach. We used revision questions from the textbook to obtain keyword (in revision questions) and a concept (in the ontology) matching. The results show that only half of the ontology concepts matched the keywords. We took further investigation on the unmatched concepts and found some incorrect concept naming and later suggest a guideline for an appropriate concept naming. At the same time, we introduce validation of ontology using revision questions as competency questions to check for ontology completeness. Furthermore, we also proposed 17 short/long answer question templates for 3 question categories, namely definition, concept completion and comparison. In the subsequent part of the thesis, we develop the AQGen tool and evaluate the generated questions. Two Computer Science subjects, namely OS and CNS, are chosen to evaluate AQGen generated questions. We conduct a questionnaire survey from 17 domain experts to identify experts’ agreement on the acceptability measure of AQGen generated questions. The experts’ agreements for acceptability measure are favourable, and it is reported that three of the four QG strategies proposed can generate acceptable questions. It has generated thousands of questions from the 3 question categories. AQGen is updated with question selection to generate a feasible question set from a tremendous amount of generated questions before. We have suggested topic-based indexing with the purpose to assert knowledge about topic chapters into ontology representation for question selection. The topic indexing shows a feasible result for filtering question by topics. Finally, our results contribute to an understanding of ontology element representation for question generations and how to automatically generate questions from ontology for education assessment

The design of an undergraduate chiropractic curriculum

Evidence is provided to support Kierkegaard's phenomenology that only what is learned through experience is truly known. It is demonstrated that the chiropractic curriculum represents a unique area of investigation and that it is possible to define curriculum; to create a functional and integrative model which subsumes elements from the traditional, cyclical and process models; and to design an integrative, problem-based, evidence-based, experiential chiropractic curriculum. A taxonomy is proposed for curriculum design in four domains which deal respectively with a) curriculum processes which include the selection, motivation and interaction of curriculum developers, curriculum definitions and models, and an algorithm for curriculum design; b) curriculum organisation which addresses philosophical, sociological, cultural and psychological foundations, curriculum paradigms and a chiropractic conceptual framework; c) curriculum development which concerns design strategies, situational analysis, intent, content, design and organisation of learning experiences and assessment of student performance; and d) curriculum application, which includes the learning climate, quality management, management of change, self-evaluation and external accreditationCurriculum and Instructional StudiesM. Ed. (Didactics

Botswana teachers' experiences of formative assessment in Standard 4 Mathematics

In 21st century education, assessment is considered a crucial role as it bridges the gap between teaching and learning. This study aimed to determine the possible effects of an intervention for Botswana Standard 4 mathematics use of formative assessment (FA) strategies in the classroom to enhance pupils’ higher-order thinking skills (HOTS). There is a recurrent educational debate in Botswana centred on pupils’ poor achievement in basic numeracy, literacy, and life skills. Specifically, Standard 4 pupils who progress to Standard 5 and 6 were perceived to be unable to handle mathematical HOTS tasks. They were considered to be weak when measured on the common achievement in large-scale assessments such as the Trends in International Mathematics and Science Study (TIMSS) 2015. With this backdrop in mind, the current study took a sequential embedded mixed, single-group, pre-test, intervention (training) and post-test design as a methodological strategy to investigate assessment practices related to higher-order thinking skills. The findings from the baseline survey (Phase I) revealed that teachers had some challenges in FA practices, including integration of HOTS tasks. The lesson from this phase’s research data is how to assist the teacher in implementing FA effectively to enhance mathematics teaching. Post-intervention classroom observation showed that teachers practised the integration and implementation of some FA strategies (Phase II). The findings in Phase III also revealed that the pupils’ post-interventional results in HOTS tasks significantly improved compared to their pre-assessment results. Additionally, the teachers’ experience and reflections were found to be favourably inclined to support formative assessment higher-order thinking skills as a strategy to enhance mathematics teaching. Throughout the phases of the study, Kivunja’s assessment feedback loop (AFL) model was underpinned as a theoretical point of departure to determine the classroom FA practice and integration of teaching HOTS in mathematics. The current study provides evidence of strategies to improve achievement levels in mathematics higher-order thinking skills in primary education in Botswana.Thesis (PhD (Assessment and Quality Assurance))--University of Pretoria, 2021.University of Pretoria Postgraduate scholarshipEducational PsychologyPhD (Assessment and Quality Assurance)Unrestricte

Bioinspired metaheuristic algorithms for global optimization

This paper presents concise comparison study of newly developed bioinspired algorithms for global optimization problems. Three different metaheuristic techniques, namely Accelerated Particle Swarm Optimization (APSO), Firefly Algorithm (FA), and Grey Wolf Optimizer (GWO) are investigated and implemented in Matlab environment. These methods are compared on four unimodal and multimodal nonlinear functions in order to find global optimum values. Computational results indicate that GWO outperforms other intelligent techniques, and that all aforementioned algorithms can be successfully used for optimization of continuous functions

Experimental Evaluation of Growing and Pruning Hyper Basis Function Neural Networks Trained with Extended Information Filter

In this paper we test Extended Information Filter (EIF) for sequential training of Hyper Basis Function Neural Networks with growing and pruning ability (HBF-GP). The HBF neuron allows different scaling of input dimensions to provide better generalization property when dealing with complex nonlinear problems in engineering practice. The main intuition behind HBF is in generalization of Gaussian type of neuron that applies Mahalanobis-like distance as a distance metrics between input training sample and prototype vector. We exploit concept of neuron’s significance and allow growing and pruning of HBF neurons during sequential learning process. From engineer’s perspective, EIF is attractive for training of neural networks because it allows a designer to have scarce initial knowledge of the system/problem. Extensive experimental study shows that HBF neural network trained with EIF achieves same prediction error and compactness of network topology when compared to EKF, but without the need to know initial state uncertainty, which is its main advantage over EKF