Game Theory based Peer Grading Mechanisms for MOOCs

An efficient peer grading mechanism is proposed for grading the multitude of assignments in online courses. This novel approach is based on game theory and mechanism design. A set of assumptions and a mathematical model is ratified to simulate the dominant strategy behavior of students in a given mechanism. A benchmark function accounting for grade accuracy and workload is established to quantitatively compare effectiveness and scalability of various mechanisms. After multiple iterations of mechanisms under increasingly realistic assumptions, three are proposed: Calibration, Improved Calibration, and Deduction. The Calibration mechanism performs as predicted by game theory when tested in an online crowd-sourced experiment, but fails when students are assumed to communicate. The Improved Calibration mechanism addresses this assumption, but at the cost of more effort spent grading. The Deduction mechanism performs relatively well in the benchmark, outperforming the Calibration, Improved Calibration, traditional automated, and traditional peer grading systems. The mathematical model and benchmark opens the way for future derivative works to be performed and compared

An Algorithm for Peer Review Matching in Massive Courses for Minimising Students' Frustration

Traditional pedagogical approaches are no longer sufficient to cope with the increasing challenges of Massive Open On-line Courses (MOOCs). Consequently, it is necessary to explore new paradigms. This paper describes an exploration of the adaptation of the peer review methodology for its application to MOOCs. Its main goal is to minimise the students' frustration through the reduction of the number of committed students that receive no feedback from their peers. In order to achieve this objective, we propose two algorithms for the peer review matching in MOOCs. Both reward committed students by prioritising the review of their submissions. The first algorithm uses sliding deadlines to minimise the probability of a submission not being reviewed. Our experiments show that it reduces dramatically the number of submissions from committed students that do not receive any review. The second algorithm is a simplification of the former. It is easier to implement and, despite performing worse than the first one, it also improves with respect to the baseline.This work was partially funded by the Spanish Competitiveness and Economy Ministry (Ministerio de Economía y Competitividad) project “EEE – Espacios Educativos Especulares” (TIN2011-28308-C03-01), and by the Madrid regional project “eMadrid” (S2009/TIC-1650)

Gamification: a key determinant of massive open online course (MOOC) success

Massive open online courses (MOOCs), contribute significantly to individual empowerment because they can help people learn about a wide range of topics. To realize the full potential of MOOCs, we need to understand their factors of success, here defined as the use, user satisfaction, along the individual and organizational performance resulting from the user involvement. We propose a theoretical framework to identify the determinants of successful MOOCs, and empirically measure these factors in a real MOOC context. We put forward the role of gamification and suggest that, together with information system (IS) theory, gamification proved to play a crucial role in the success of MOOCs.info:eu-repo/semantics/acceptedVersio

Once upon a tip... : a story of MOOCs and gamification

Comunicação publicada nas atas da conferência EADTU 2017, realizadas na Open University, Milton Keynes, de 25-27 de outubro de 2017.This paper discusses the future of MOOCs based on recent research and acknowledged affordances of videogame’s design. The interest in MOOCS for educational purposes has increased over the last few years, with researchers identifying key pedagogical features that make the success of these inherently powerful learning tools. However, low student motivation and high dropout rates have somehow changed the original expectations of many researchers, despite the MOOC user base doubling in 2015. So, in this study we survey recent literature looking for answers, and discuss the evidence gathered from specific MOOCs with over one thousand participants, namely, pioneering iMOOC courses at Universidade Aberta (the Portuguese Open University). Finally, we look at the gaming world and discuss some findings that may benefit the learning design of MOOCs, considering that, besides the huge appeal of these (free) courses, there are recurring shortcomings that we have to alleviate. We follow up on the tip that gamification, and other emerging strategies, such as social networking and digital storytelling, may be vital to assure a sustainable future for open education and MOOCs.info:eu-repo/semantics/publishedVersio

MOOC (Massive Open Online Courses)

Massive Open Online Courses (MOOCs) are free online courses available to anyone who can sign up. MOOCs provide an affordable and flexible way to learn new skills, advance in careers, and provide quality educational experiences to a certain extent. Millions of people around the world use MOOCs for learning and their reasons are various, including career development, career change, college preparation, supplementary learning, lifelong learning, corporate e-Learning and training, and so on

Investigating Different Types of Assessment in Massive Open Online Courses

The current technological era has largely influenced the development of learning environments. As a result, there are new opportunities for teaching, learning and assessment. The emergence of Massive Open Online Courses (MOOCs) in particular, has attracted the attention of higher education institutions and course designers. MOOCs may provide the opportunity to thousands of students to learn from anywhere and at their convenience. Assessment is a component of the learning environment that drives student learning. However, only a small proportion of existing literature on assessment investigates its use for the enhancement of educational growth as most of the literature is concerned with how to use assessment for purposes of grading and ranking (Rowntree, 1987). Assessment has a double role in learning by both motivating students to study in order to undertake it, but also providing the necessary feedback on their performance so that students can track their learning progress (Rowntree, 1987). Research in MOOCs is currently growing, focusing on different aspects such as the “questionable course quality, high dropout rate, unavailable course credits, complex copyright, limited hardware and ineffective assessments” (Chen, 2014). Assessment in MOOCs has been mostly investigated from a perspective that is looking at: how the grading load can be diminished by adopting automated techniques, the aims of each technique, and finally new potential approaches that will be able to assess high-level cognition. Summing up, researchers are currently testing tools that will be automatically scoring essays and giving feedback to learners in an effective way (see Balfour, 2013). However, the learners’ voice and standpoint about the different assessment types in the MOOCs context is inconclusive in the current literature and there is need for more research. This study explores learners’ views on assessment types in Massive Open Online Courses, whether any of these has an impact on their enrolment and completion of a course and in what aspects each type of assessment is effective in supporting their learning experience. Auto-assessment, peer-assessment and self-assessment are the types under investigation as they are frequently used in MOOCs and therefore are the most commonly discussed in literature (see Balfour, 2013, Suen, 2013, Wilkowski et al, 2014). The study draws upon literature on assessment in general and on assessment in MOOCs in particular. The concept of online communities, i.e. the learners that appear in MOOCs will also be discussed in detail. Online ethnographic approaches are employed to explore the issue in question by using online interviewing and observation methods. Thematic analysis is carried out using a sample of 12 MOOCs participants from online interviews and 13 posts of online observations. The outcome of this qualitative research study reveals that even though participants identify benefits in peer assessment, there is a preference for automated assessment since it is an already known, clear type of assessment for them. Moreover, self-assessment is not popular by participants. Learners’ comments also reveal that a clear guidance for assessment helps them to carry out peer assessment more effectively. Some learners also consider that the combination of assessment types may also have a positive effect on students’ learning as each of them serves a different purpose