Social Addictive Gameful Engineering (SAGE): A Game-based Learning and Assessment System for Computational Thinking

At an unrivaled and enduring pace, computing has transformed the world, resulting in demand for a universal fourth foundation beyond reading, writing, and arithmetic: computational thinking (CT). Despite increasingly widespread acceptance of CT as a crucial competency for all, transforming education systems accordingly has proven complex. The principal hypothesis of this thesis is that we can improve the efficiency and efficacy of teaching and learning CT by building gameful learning and assessment systems on top of block-based programming environments. Additionally, we believe this can be accomplished at scale and cost conducive to accelerating CT dissemination for all. After introducing the requirements, approach, and architecture, we present a solution named Gameful Direct Instruction. This involves embedding Parsons Programming Puzzles (PPPs) in Scratch, which is a block-based programming environment currently used prevalently in grades 6-8. PPPs encourage students to practice CT by assembling into correct order sets of mixed-up blocks that comprise samples of well-written code which focus on individual concepts. The structure provided by PPPs enable instructors to design games that steer learner attention toward targeted learning goals through puzzle-solving play. Learners receive continuous automated feedback as they attempt to arrange programming constructs in correct order, leading to more efficient comprehension of core CT concepts than they might otherwise attain through less structured Scratch assignments. We measure this efficiency first via a pilot study conducted after the initial integration of PPPs with Scratch, and second after the addition of scaffolding enhancements in a study involving a larger adult general population. We complement Gameful Direct Instruction with a solution named Gameful Constructionism. This involves integrating with Scratch implicit assessment functionality that facilitates constructionist video game (CVG) design and play. CVGs enable learner to explore CT using construction tools sufficiently expressive for personally meaningful gameplay. Instructors are enabled to guide learning by defining game objectives useful for implicit assessment, while affording learners the opportunity to take ownership of the experience and progress through the sequence of interest and motivation toward sustained engagement. When strategically arranged within a learning progression after PPP gameplay produces evidence of efficient comprehension, CVGs amplify the impact of direct instruction by providing the sculpted context in which learners can apply CT concepts more freely, thereby broadening and deepening understanding, and improving learning efficacy. We measure this efficacy in a study of the general adult population. Since these approaches leverage low fidelity yet motivating gameful techniques, they facilitate the development of learning content at scale and cost supportive of widespread CT uptake. We conclude this thesis with a glance at future work that anticipates further progress in scalability via a solution named Gameful Intelligent Tutoring. This involves augmenting Scratch with Intelligent Tutoring System (ITS) functionality that offers across-activity next-game recommendations, and within-activity just-in-time and on-demand hints. Since these data-driven methods operate without requiring knowledge engineering for each game designed, the instructor can evolve her role from one focused on knowledge transfer to one centered on supporting learning through the design of educational experiences, and we can accelerate the dissemination of CT at scale and reasonable cost while also advancing toward continuously differentiated instruction for each learner

DeepEval: An Integrated Framework for the Evaluation of Student Responses in Dialogue Based Intelligent Tutoring Systems

The automatic assessment of student answers is one of the critical components of an Intelligent Tutoring System (ITS) because accurate assessment of student input is needed in order to provide effective feedback that leads to learning. But this is a very challenging task because it requires natural language understanding capabilities. The process requires various components, concepts identification, co-reference resolution, ellipsis handling etc. As part of this thesis, we thoroughly analyzed a set of student responses obtained from an experiment with the intelligent tutoring system DeepTutor in which college students interacted with the tutor to solve conceptual physics problems, designed an automatic answer assessment framework (DeepEval), and evaluated the framework after implementing several important components. To evaluate our system, we annotated 618 responses from 41 students for correctness. Our system performs better as compared to the typical similarity calculation method. We also discuss various issues in automatic answer evaluation

Digital Disruption in Teaching and Testing

This book provides a significant contribution to the increasing conversation concerning the place of big data in education. Offering a multidisciplinary approach with a diversity of perspectives from international scholars and industry experts, chapter authors engage in both research- and industry-informed discussions and analyses on the place of big data in education, particularly as it pertains to large-scale and ongoing assessment practices moving into the digital space. This volume offers an innovative, practical, and international view of the future of current opportunities and challenges in education and the place of assessment in this context

Predicting grade progression within the Limpopo Education System

One way to improve education in South Africa is to ensure that additional support and resourcing are provided to schools and learners that are most in need of help. To this end, education officials need to understand the factors affecting learning and the schools most in need of appropriate interventions. Several theories, models and methods have been developed to attempt to address the challenges faced in the education sector. Educational Data Mining (EDM) is one which has gained prominence in addressing these challenges. EDM is a field of data mining using mathematical and machine learning models to improve learners’ performance, education administration, and policy formulation. This study explored the literature and related methodologies used within the EDM context and constructed a solution to improve learner support and planning in the Limpopo primary and secondary schools education system. The data utilized included socio-economic environment, demographic information as well as learner’s performance sourced from the Education Management Information Systems database of the Limpopo Department of Education (LDoE). Feature selection methods; Information Gain, Correlation and Asymmetrical Uncertainty were combined to determine factors that affect learning. Three machine learning classifiers, AdaboostM1 (Decision Stump), HoeffdingTree and NaïveBayes, were used to predict learners’ grade progression. These were compared using several evaluation metrics and HoeffdingTree outperformed AdaboostM1 (Decision Stump) and NaïveBayes. When the final HoeffdingTree model was applied to the test datasets, the performance was exceptionally good. It is hoped that the implementation of this model will assist the LDoE in its role of supporting learning and planning of resource allocation

Comprehension based adaptive learning systems

Conversational Intelligent Tutoring Systems aim to mimic the adaptive behaviour of human tutors by delivering tutorial content as part of a dynamic exchange of information conducted using natural language. Deciding when it is beneficial to intervene in a student’s learning process is an important skill for tutoring. Human tutors use prior knowledge about the student, discourse content and learner non-verbal behaviour to choose when intervention will help learners overcome impasse. Experienced human tutors adapt discourse and pedagogy based on recognition of comprehension and non-comprehension indicative learner behaviour. In this research non-verbal behaviour is explored as a method of computationally analysing reading comprehension so as to equip an intelligent conversational agent with the human-like ability to estimate comprehension from non-verbal behaviour as a decision making trigger for feedback, prompts or hints. This thesis presents research that combines a conversational intelligent tutoring system (CITS) with near real-time comprehension classification based on modelling of e-learner non-verbal behaviour to estimate learner comprehension during on-screen conversational tutoring and to use comprehension classifications as a trigger for intervening with hints, prompts or feedback for the learner. To improve the effectiveness of tuition in e-learning, this research aims to design, develop and demonstrate novel computational methods for modelling e-learner comprehension of on-screen information in near real-time and for adapting CITS tutorial discourse and pedagogy in response to perception of comprehension indicative behaviour. The contribution of this research is to detail the motivation for, design of, and evaluation of a system which has the human-like ability to introduce micro-adaptive feedback into tutorial discourse in response to automatic perception of e-learner reading comprehension. This research evaluates empirically whether e-learner non-verbal behaviour can be modelled to classify comprehension in near real-time and presents a near real-time comprehension classification system which achieves normalised comprehension classification accuracy of 75%. Understanding e-learner comprehension creates exciting opportunities for advanced personalisation of materials, discourse, challenge and the digital environment itself. The research suggests a benefit is gained from comprehension based adaptation in conversational intelligent tutoring systems, with a controlled trial of a comprehension based adaptive CITS called Hendrix 2.0 showing increases in tutorial assessment scores of up to 17% when comprehension based discourse adaptation is deployed to scaffold the learning experience

Eye on Collaborative Creativity : Insights From Multiple-Person Mobile Gaze Tracking in the Context of Collaborative Design

Early Career WorkshopNon peer reviewe

Expanding evidence approaches for learning in a digital world

Executive Summary: Relatively low-cost digital technology is ubiquitous in daily life and work. The Web is a vast source of information, communication, and connection opportunities available to anyone with Internet access. Most professionals and many students have a mobile device in their pocket with more computing power than early supercomputers. These technological advances hold great potential for improving educational outcomes, but by themselves hardware and networks will not improve learning. Decades of research show that high-quality learning resources and sound implementations are needed as well.The learning sciences have found that today’s technologies offer powerful capabilities for creating high-quality learning resources, such as capabilities for visualization, simulation, games, interactivity, intelligent tutoring, collaboration, assessment, and feedback. Further, digital learning resources enable rapid cycles of iterative improvement, and improvements to resources can be instantly distributed over the Internet. In addition, digital technologies are attracting exciting new talent, both from other industries and from the teacher workforce itself, into the production of digital learning resources. Yet even with so many reasons to expect dramatic progress, something more—better use of evidence— is needed to support the creation, implementation, and continuous enhancement of high-quality learning resources in ways that improve student outcomes

Doctor of Philosophy in Computer Science

dissertationThe organization of learning materials is often limited by the systems available for delivery of such material. Currently, the learning management system (LMS) is widely used to distribute course materials. These systems deliver the material in a text-based, linear way. As online education continues to expand and educators seek to increase their effectiveness by adding more effective active learning strategies, these delivery methods become a limitation. This work demonstrates the possibility of presenting course materials in a graphical way that expresses important relations and provides support for manipulating the order of those materials. The ENABLE system gathers data from an existing course, uses text analysis techniques, graph theory, graph transformation, and a user interface to create and present graphical course maps. These course maps are able to express information not currently available in the LMS. Student agents have been developed to traverse these course maps to identify the variety of possible paths through the material. The temporal relations imposed by the current course delivery methods have been replaced by prerequisite relations that express ordering that provides educational value. Reducing the connections to these more meaningful relations allows more possibilities for change. Technical methods are used to explore and calibrate linear and nonlinear models of learning. These methods are used to track mastery of learning material and identify relative difficulty values. Several probability models are developed and used to demonstrate that data from existing, temporally based courses can be used to make predictions about student success in courses using the same material but organized without the temporal limitations. Combined, these demonstrate the possibility of tools and techniques that can support the implementation of a graphical course map that allows varied paths and provides an enriched, more informative interface between the educator, the student, and the learning material. This fundamental change in how course materials are presented and interfaced with has the potential to make educational opportunities available to a broader spectrum of people with diverse abilities and circumstances. The graphical course map can be pivotal in attaining this transition