An item analysis on multiple-choice questions: a case of a junior high school English try-out test in Indonesia

The present study aims to analyze multiple-choice questions obtained from a trial testing conducted in a state junior high school in Indonesia. The study seeks to reveal the level of difficulty, discriminating power and distractor efficiency of the selected test items by employing item analysis. The result of the study discovers that levels of difficulty on the question items are varied. Some question tended to be easy and moderately difficult while the others are difficult to answer. It also uncovers that, in regard to discriminating power, some questions are well constructed while the others are ambiguously worded that can potentially cause the questions to fail to evaluate the students’ ability. The analysis on distractor efficiency presents information how the chosen multiple-choice questions were frequently constructed with less effective distractors that caused more high achieving students to choose wrong answers

Practice and Assessment of Reading Classes Using Moodle

This research paper details the extensive use of Computer Assisted Language Learning (CALL) for a content-based reading syllabus at Gunma University, through the software program Moodle (Modular Object-Oriented Dynamic Learning Environment ), a free and open-source software learning management system used at Gunma University. 　　The research basis of this paper is within the sphere of Action Research , as a valuable professional development tool (Nunan, 2001) based on this researcher’s perceived valuation of the system and how it could better aid students to perform better in and be more motivated towards their English language and reading studies, introduce new technological skills and abilities, and aid teachers in better preparation, teaching and assessment of reading classes. Moodle enthuses that the Lesson Module ‘enables a teacher to deliver content and/or practice activities in interesting and flexible ways...teachers can choose to increase engagement and ensure understanding by including a variety of questions, such as multiple choice, matching and short answer.’ (Moodle, 2016). Therefore, this paper will ascertain whether the syllabus achieved a greater engagement and enjoyment by the students, and ensured better comprehension and understanding of key tasks and instructions. In addition, it will detail how teachers can benefit course management by employing such technology within the classroom

Automating question generation from educational text

The use of question-based activities (QBAs) is wide-spread in education, traditionally forming an integral part of the learning and assessment process. In this paper, we design and evaluate an automated question generation tool for formative and summative assessment in schools. We present an expert survey of one hundred and four teachers, demonstrating the need for automated generation of QBAs, as a tool that can significantly reduce the workload of teachers and facilitate personalized learning experiences. Leveraging the recent advancements in generative AI, we then present a modular framework employing transformer based language models for automatic generation of multiple-choice questions (MCQs) from textual content. The presented solution, with distinct modules for question generation, correct answer prediction, and distractor formulation, enables us to evaluate different language models and generation techniques. Finally, we perform an extensive quantitative and qualitative evaluation, demonstrating trade-offs in the use of different techniques and models.Comment: Accepted to AI-2023 (Forty-third SGAI International Conference on Artificial Intelligence) as a long paper, link: http://www.bcs-sgai.org/ai202

Philosophers’ Views on the Use of Non-Essay Assessment Methods: Discussion of an E-Mail Survey

This paper presents and discusses the results of an email survey which asked participants to share their views on the efficacy of multiple choice, fill-in-the-blank, or matching questions as evaluation methods in philosophy courses. First, the structure of the survey and its contents are explained. Next, responses are broken down along the lines of student responses and teacher responses. In both cases, there was significant disagreement among respondents, though there were notable patterns emerged. Student arguments in favor of non-essay assessment emphasized the expedience; arguments against emphasized the inadequacy of such evaluation methods to the nuances of philosophical material. Teacher responses echoed student responses but included considerations of fairness, ambiguity in student answers, student motivation, and justifications for non-essay assessment in specific contexts. Finally, the author discusses respondents’ opinions on whether philosophy departments should ban non-essay questions. The author concludes by suggesting that the results of this survey merit attention as an indication of how widespread the difficulties of non-essay assessment are and as an indication of the diversity of views on the subject

Scrum2Kanban: Integrating Kanban and Scrum in a University Software Engineering Capstone Course

Using university capstone courses to teach agile software development methodologies has become commonplace, as agile methods have gained support in professional software development. This usually means students are introduced to and work with the currently most popular agile methodology: Scrum. However, as the agile methods employed in the industry change and are adapted to different contexts, university courses must follow suit. A prime example of this is the Kanban method, which has recently gathered attention in the industry. In this paper, we describe a capstone course design, which adds the hands-on learning of the lean principles advocated by Kanban into a capstone project run with Scrum. This both ensures that students are aware of recent process frameworks and ideas as well as gain a more thorough overview of how agile methods can be employed in practice. We describe the details of the course and analyze the participating students' perceptions as well as our observations. We analyze the development artifacts, created by students during the course in respect to the two different development methodologies. We further present a summary of the lessons learned as well as recommendations for future similar courses. The survey conducted at the end of the course revealed an overwhelmingly positive attitude of students towards the integration of Kanban into the course

Extending Item Response Theory to Online Homework

Item Response Theory becomes an increasingly important tool when analyzing ``Big Data'' gathered from online educational venues. However, the mechanism was originally developed in traditional exam settings, and several of its assumptions are infringed upon when deployed in the online realm. For a large enrollment physics course for scientists and engineers, the study compares outcomes from IRT analyses of exam and homework data, and then proceeds to investigate the effects of each confounding factor introduced in the online realm. It is found that IRT yields the correct trends for learner ability and meaningful item parameters, yet overall agreement with exam data is moderate. It is also found that learner ability and item discrimination is over wide ranges robust with respect to model assumptions and introduced noise, less so than item difficulty

The use of computers as substitute tutors for marketing students

The use of computers as substitute tutors is associated primarily, though not exclusively, with multiple‐choice question formats (Ellington, Percival and Race, 1993). We report the findings of a project that involved the design, testing and evaluation of a set of computer‐based tutorials employing multiple‐choice questions with 700 students on postgraduate and undergraduate introductory Marketing modules. The computer‐based tutorials were designed to meet two main objectives, namely to help students in their formative assessment and to help staff monitor any difficulties students were experiencing with module content. However, students incorporated the tutorials into their learning in ways that had not been anticipated specifically; they used the tutorials for a number of related but different purposes, and their usage patterns varied considerably