Immediate Elaborated Feedback Personalization in Online Assessment

Providing a student with feedback that is timely, most suitable and useful for her personality and the performed task is a challenging problem of online assessment within Web-based Learning Systems (WBLSs). In our recent work we suggested a general approach of feedback adaptation in WBLS and through a series of experiments we demonstrated the possibilities of tailoring the feedback that is presented to a student as a result of her response to questions of an online test, taking into account the individual learning styles (LS), certitude in a response and correctness of this response. In this paper we present the result of the most recent experimental field study where we tested two feedback adaptation strategies in real student assessment settings (73 students had to answer 15 multiple-choice questions for passing the midterm exam). The first strategy is based on the correctness and certitude of the response, while the second strategy takes student LS into account as well. The analysis of assessment results and students’ behaviour demonstrate that both strategies perform reasonably well, yet the analysis also provide some evidence that the second strategy does a better job

Process mining online assessment data

Traditional data mining techniques have been extensively applied to find interesting patterns, build descriptive and predictive models from large volumes of data accumulated through the use of different information systems. The results of data mining can be used for getting a better understanding of the underlying educational processes, for generating recommendations and advice to students, for improving management of learning objects, etc. However, most of the traditional data mining techniques focus on data dependencies or simple patterns and do not provide a visual representation of the complete educational (assessment) process ready to be analyzed. To allow for these types of analysis (in which the process plays the central role), a new line of data-mining research, called process mining, has been initiated. Process mining focuses on the development of a set of intelligent tools and techniques aimed at extracting process-related knowledge from event logs recorded by an information system. In this paper we demonstrate the applicability of process mining, and the ProM framework in particular, to educational data mining context. We analyze assessment data from recently organized online multiple choice tests and demonstrate the use of process discovery, conformance checking and performance analysis techniques

A Virtual Conversational Agent for Teens with Autism: Experimental Results and Design Lessons

We present the design of an online social skills development interface for teenagers with autism spectrum disorder (ASD). The interface is intended to enable private conversation practice anywhere, anytime using a web-browser. Users converse informally with a virtual agent, receiving feedback on nonverbal cues in real-time, and summary feedback. The prototype was developed in consultation with an expert UX designer, two psychologists, and a pediatrician. Using the data from 47 individuals, feedback and dialogue generation were automated using a hidden Markov model and a schema-driven dialogue manager capable of handling multi-topic conversations. We conducted a study with nine high-functioning ASD teenagers. Through a thematic analysis of post-experiment interviews, identified several key design considerations, notably: 1) Users should be fully briefed at the outset about the purpose and limitations of the system, to avoid unrealistic expectations. 2) An interface should incorporate positive acknowledgment of behavior change. 3) Realistic appearance of a virtual agent and responsiveness are important in engaging users. 4) Conversation personalization, for instance in prompting laconic users for more input and reciprocal questions, would help the teenagers engage for longer terms and increase the system's utility

Usefulness of feedback in e-learning from the students' perspective

Purpose: Functionality of feedback in pedagogical processes has been broadly analyzed in face-to-face learning, although to a lesser extent than in the on-line learning. Narciss (2004, 2008) distinguishes two dimensions within the feedback, the semantic dimension and the structural dimension. This article aims to analyze, from the students' perspective, the semantic dimension of feedback in a virtual learning environment (VLE). Firstly, we analyze the importance that VLE students give to feedback and its degree of personalization. Later, the usefulness that students deem feedback should have, paying special attention to each of its semantic subdimensions (Narciss, 2004). Design/methodology/approach: A survey was conducted among students of Business Administration degree of the Universitat Oberta de Catalunya (UOC). 182 students took part, separated into two groups (pilot and control). Findings: It has been proved that 90% of students give a great or very great importance to reception of feedback from their tutors, a relevance they consider to be higher than the one within a face-to-face environment. This percentage is around 75% with regard to the importance given to the level of personalization of feedback. The development of a factor analysis has revealed that usefulness of personalized feedback perceived by the students can be subsumed under two large dimensions: the one that facilitates learning (related to its semantic dimension) and the motivational one (by allowing an easier and more fluid communication with the tutor, contributing not to leave the course, etc.) The latter dimension has been also proved to be key in order to attain improvements in the students' satisfaction with the learning process. This research was funded by the Catalan government and the UOC, within the framework of projects to Improve Teaching Quality (ITQ). Originality/value: Implementing personalized feedback has a relevant impact on the student, who values it because it makes his learning process easier, richer and more significant. Moreover, it has a clear motivational effect over the student, which had not been sufficiently evidenced by other researches. Such effect needs not to be underestimated, particularly within an on-line environment, where dropout rate is usually high

Usefulness of feedback in e-learning from the students' perspective

Functionality of feedback in pedagogical processes has been broadly analyzed in face-to-face learning, although to a lesser extent than in the on-line learning. Narciss (2004, 2008) distinguishes two dimensions within the feedback, the semantic dimension and the structural dimension. This article aims to analyze, from the student's perspective, the semantic dimension of feedback in a virtual learning environment (VLE). Firstly, we analyze the importance that VLE students give to feedback and its degree of personalization. Later, the usefulness that students deem feedback should have, paying special attention to each of its semantic subdimensions (Narciss, 2004)

DEVELOPMENT OF COMPUTER-BASED DIAGNOSTIC ASSESSMENT SYSTEM: CASE STUDY OF EQUIVALENCE OF PAPER-AND-PENCIL AND COMPUTER-BASED TESTING

In the last two decades computer-based assessment has become an important part in support of teaching and learning. It is seen as a solution to implement assessment for learning in school and provide immediate feedback on students’ performance in real-time. Research literature on computer-based assessment suggests that every measurement instrument developer before implementation of a test has to provide evidence that computer-based and paper-based versions are equivalent and provide consistent measures. There is a risk that properties of computer-based assessment including unfamiliarity with the system and proficiency level of digital skills can seriously affect students’ performance. This paper focuses on computer-based diagnostic assessment system designed to support numeracy and literacy teaching and learning. The aim of this study is to confirm that literacy and numeracy learning measurement instruments elaborated in diagnostic assessment system provide consistent results as paper-based versions of both instruments. Data were collected administering four tests. Two of the assessments were computer-based literacy and numeracy diagnostic assessments and two were paper-based versions. By analyzing both versions of assessments using various statistical techniques we explore differences in students’ performance. Our results showed that at this development phase of the computer-based diagnostic assessment system the students who completed computer-based test versions showed similar or better performance than their counterparts who completed paper-based versions

Measuring the Additive Effects of Multimedia Social Cue Principles on Learners’ Cognitive Load, Emotions, Attitude, and Learning Outcomes

Multimedia principles are developed and employed to design effective multimedia instructions that foster learning. Specifically, multimedia principles such as personalization, voice, and embodiment principles are developed based on social cues to promote deep learning. Most researchers in the past have investigated the individual effects of these principles on learning. The goal of the present study was to investigate the additive effects of these abovementioned principles on learners’ perceived cognitive load, emotions, attitude, and learning outcomes (i.e. retention and transfer of knowledge). Sixty college students participated in this study. Participants were asked to complete two short instructional modules and a short learning assessment after each module. Additionally, they were asked to complete a NASA Task Load Index (TLX) questionnaire, emotion assessment, and attitude questionnaire. The results suggested that non-personalized instructions lead to higher cognitive load than the personalized instructions. Participants in the personalized voice with embodiment condition had the least feelings of disgust when learning the information and had highest retention scores. Additionally, personalized voice narrations were found to be detrimental for learning. However, if personalized voice narrations are used for instructional purposes, then it must be accompanied with an embodiment to foster learning and improve performance on transfer of knowledge. The findings of this study could be used to improve the design of the multimedia instructions that are effective in fostering learning

ADMINISTERING CLASSES IN VIRTUAL LEARNING ENVIRONMENT: FACILITATIONS, PRINCIPLES, EDUCATORS’ ROLES AND INTERACTIONS

This study investigates teachers’ competencies in the implementation of facilities, principles, educators’ roles, and interactions in the Virtual Learning Environment (VLE). Using explanatory sequential mixed method, it aims to examine whether all those competencies such as learning system tools and resources, technology quality, and workload management are considered throughout the learning process. In total, 102 students from the Faculty of Tarbiyah and Teachers’ Training at Antasari State Islamic University were involved in the survey. Fourteen of them, as well as three lecturers, were then interviewed. The results indicate that despite students’ limited access to the internet, lecturers attempt to provide all students’ facilities. However, in some departments, various learning activities, monitoring, and feedback still need to be improved. The lecturers generally execute the principle of the virtual learning environment. The students mostly believe that lecturers are aware of their roles, and manage their interactions well. In conclusion, the study suggests teachers’ competencies in the virtual learning environment and calls for further studies in this subject.

Beyond Personalization: Embracing Democratic Learning Within Artificially Intelligent Systems

This essay explains how, from the theoretical perspective of Basil Bernstein's three “conditions for democracy,” the current pedagogy of artificially intelligent personalized learning seems inadequate. Building on Bernstein's comprehensive work and more recent research concerned with personalized education, Natalia Kucirkova and Sandra Leaton Gray suggest three principles for advancing personalized education and artificial intelligence (AI). They argue that if AI is to reach its full potential in terms of promoting children's identity as democratic citizens, its pedagogy must go beyond monitoring the technological progression of personalized provision of knowledge. It needs to pay more careful attention to the democratic impact of data‐driven systems. Kucirkova and Leaton Gray propose a framework to distinguish the value of personalized learning in relation to pluralization and to guide educational researchers and practitioners in its application to socially just classrooms

A Phenomenological Study of Teachers\u27 Experiences with Educational Gamification and its Impact on Student Engagement in the Middle School Math and Science Classroom

The purpose of this phenomenological study was to understand digital gamification and its effect on student engagement based on the lived experiences of middle school math and science teachers in rural schools in the southeast region of the United States. Nick Pelling’s gamification theory guided the study herein. Gamification theory served as a tool to alter learner engagement which impacted instruction and learning. I used a criterion-based purposeful selection of 10 middle school math and science teachers with gamification experience. Participating teachers had three or more years of teaching experience and taught in regional rural schools. The hermeneutical phenomenological study resulted in the themes of gamification elements on student engagement, planning gamification lessons, and obstacles to gamification. The lived experiences of middle school math and science teachers positively addressed the gap in the correlation between gamification and enhancing student engagement