Adaptive structure metrics for automated feedback provision in intelligent tutoring systems

Paaßen B, Mokbel B, Hammer B. Adaptive structure metrics for automated feedback provision in intelligent tutoring systems. Neurocomputing. 2016;192(SI):3-13.Typical intelligent tutoring systems rely on detailed domain-knowledge which is hard to obtain and difficult to encode. As a data-driven alternative to explicit domain-knowledge, one can present learners with feedback based on similar existing solutions from a set of stored examples. At the heart of such a data-driven approach is the notion of similarity. We present a general-purpose framework to construct structure metrics on sequential data and to adapt those metrics using machine learning techniques. We demonstrate that metric adaptation improves the classification of wrong versus correct learner attempts in a simulated data set from sports training, and the classification of the underlying learner strategy in a real Java programming dataset

A Multifaceted Consideration of Motivation and Learning within ASSISTments

An approach to education gaining popularity in the modern classroom, adaptive tutoring systems offer interactive learning environments in which students can access immediate feedback and rich tutoring while teachers can achieve organized assessment for targeted interventions. Yet despite the benefits that these systems provide, a number of questions remain regarding the optimal inner workings of adaptive platforms. What is the recipe for optimal student performance within these platforms? What elements should be taken into consideration when designing these learning environments? Can facets of these platforms be harnessed to increase students’ motivation to learn and to improve both immediate and robust learning gains? This thesis combines work conducted over the past two years through versatile approaches toward the goal of enhancing student motivation and learning within the ASSISTments platform. Approaches considered include a) enhancing motivation and performance through altered feedback using hypermedia elements, b) instilling motivational messages alongside media enhanced content and feedback, c) allowing students to choose their feedback medium, thereby exerting control over their assignment, d) altering content delivery by interleaving skills to enhance solution strategy development, and e) establishing partial credit assessments to drive motivation and proper system usage while enhancing student modeling. After a brief introduction regarding the main tenants of this research, each chapter highlights a randomized controlled trial focused around one of these approaches. All studies presented have been conducted or are still running within ASSISTments. Much of this work has already been published at peer reviewed conference venues, some with stringent acceptance rates as low as 25% for full papers. Two of the studies presented here are second iterations of previously published work that are still in progress, and only preliminary analyses are available. A chapter on conclusions and future work is included to discuss the contributions that have been made to the Learning Sciences community thus far, and to briefly discuss potential directions for my continued research

A Foundation For Educational Research at Scale: Evolution and Application

The complexities of how people learn have plagued researchers for centuries. A range of experimental and non-experimental methodologies have been used to isolate and implement positive interventions for students\u27 cognitive, meta-cognitive, behavioral, and socio-emotional successes in learning. But the face of learning is changing in the digital age. The value of accrued knowledge, popular throughout the industrial age, is being overpowered by the value of curiosity and the ability to ask critical questions. Most students can access the largest free collection of human knowledge (and cat videos) with ease using their phones or laptops and omnipresent cellular and Wi-Fi networks. Viewing this new-age capacity for connection as an opportunity, educational stakeholders have delegated many traditional learning tasks to online environments. With this influx of online learning, student errors can be corrected with immediacy, student data is more prevalent and actionable, and teachers can intervene with efficiency and efficacy. As such, endeavors in educational data mining, learning analytics, and authentic educational research at scale have grown popular in recent years; fields afforded by the luxuries of technology and driven by the age-old goal of understanding how people learn. This dissertation explores the evolution and application of ASSISTments Research, an approach to authentic educational research at scale that leverages ASSISTments, a popular online learning platform, to better understand how people learn. Part I details the evolution and advocacy of two tools that form the research arm of ASSISTments: the ASSISTments TestBed and the Assessment of Learning Infrastructure (ALI). An NSF funded Data Infrastructure Building Blocks grant (#1724889, $494,644 2017-2020), outlines goals for the new age of ASSISTments Research as a result of lessons learned in recent years. Part II details a personal application of these research tools with a focus on the framework of Self Determination Theory. The primary facets of this theory, thought to positively affect learning and intrinsic motivation, are investigated in depth through randomized controlled trials targeting Autonomy, Belonging, and Competence. Finally, a synthesis chapter highlights important connections between Parts I & II, offering lessons learned regarding ASSISTments Research and suggesting additional guidance for its future development, while broadly defining contributions to the Learning Sciences community

Information Technology decision makers’ readiness for artificial intelligence governance in institutions of higher education in South Africa

Artificial intelligence (AI) can enhance the educational experience for academics and students. However, research has inadequately examined AI ethics and governance, particularly in the higher education sector of developing economies such as South Africa. AI governance ensures that envisioned AI benefits are realized while reducing AI risks. Against this backdrop of huge research deficit, the current study reports on a qualitative exploratory study that investigates the state of readiness for AI governance and AI governance maturity in South African higher education institutions. Informed by the combination of the TOE framework, the traditional IT governance model and the adapted IT governance maturity assessment model, semi-structured interviews were conducted with academic and ICT decision makers from two public and three private higher education institutions in South Africa to determine their insights on the state of readiness and maturity of AI governance. Results reveal high proliferation of AI elements in higher education information systems. However, results revealed low levels of AI governance readiness by higher education institutions. The study recommends for recognition of AI risks and taking lessons from AI regulatory frameworks advanced in developed countries

Technology-enhanced Personalised Learning: Untangling the Evidence

Technology-enhanced personalised learning is not yet common in Germany, which is why we have tasked scientists with summarising the current status of international research on the matter. This study demonstrates the great potential of technology in implementing effective personalised learning. Nevertheless, it has not been assessed yet whether the practical implementation actually works: Even in countries such as the U.S., which lead the way in using techology in classroom settings, hardly any evaluation studies have been done to prove the effectiveness of technology-enhanced personalised learning. In the light of the above, the authors make recommendations for actions to be taken in Germany to make best use of the potential of technology in providing individual support and guidance to students

Student Modeling From Different Aspects

With the wide usage of online tutoring systems, researchers become interested in mining data from logged files of these systems, so as to get better understanding of students. Varieties of aspects of students’ learning have become focus of studies, such as modeling students’ mastery status and affects. On the other hand, Randomized Controlled Trial (RCT), which is an unbiased method for getting insights of education, finds its way in Intelligent Tutoring System. Firstly, people are curious about what kind of settings would work better. Secondly, such a tutoring system, with lots of students and teachers using it, provides an opportunity for building a RCT infrastructure underlying the system. With the increasing interest in Data mining and RCTs, the thesis focuses on these two aspects. In the first part, we focus on analyzing and mining data from ASSISTments, an online tutoring system run by a team in Worcester Polytechnic Institute. Through the data, we try to answer several questions from different aspects of students learning. The first question we try to answer is what matters more to student modeling, skill information or student information. The second question is whether it is necessary to model students’ learning at different opportunity count. The third question is about the benefits of using partial credit, rather than binary credit as measurement of students’ learning in RCTs. The fourth question focuses on the amount that students spent Wheel Spinning in the tutoring system. The fifth questions studies the tradeoff between the mastery threshold and the time spent in the tutoring system. By answering the five questions, we both propose machine learning methodology that can be applied in educational data mining, and present findings from analyzing and mining the data. In the second part, we focused on RCTs within ASSISTments. Firstly, we looked at a pilot study of reassessment and relearning, which suggested a better system setting to improve students’ robust learning. Secondly, we proposed the idea to build an infrastructure of learning within ASSISTments, which provides the opportunities to improve the whole educational environment

Enhancing Personalization Within ASSISTments

ASSISTments is an online adaptive tutoring system with the ability to provide assistance to students in the form of hints and scaffolding. ASSISTments has many features to help students improve their knowledge. Researchers run studies in order to discover ways for students to learn better but ASSISTments is missing one major aspect for researchers: student level personalization. It is easy to create an assignment for a particular class or school but it would take much longer to create an assignment for each student and it would be difficult for the teacher to look through many assignment reports. One of the strongest code blocks in coding is the if-then; allowing the program to branch off to another set of code under certain circumstances. ASSISTments needed an if-then system in order for students to branch off to other parts of the assignment under certain circumstances. With this, researchers would be able to personalize assignments to give more help to lower knowledge students or allow students to get a choice of what kind of tutoring they would like to receive. With this idea in mind, the basic if-then structure was implemented into ASSISTments using problem or problem set correctness as the condition statement. Once the if-then system was created opportunities opened to create additional experiments and run studies in ASSISTments. The basic if-then was limited in using correctness only for its condition statement. This meant that a new if-then system would need to be implemented to include custom condition statements that allowed the researcher to have the assignment branch on any condition using all the information recorded in the assignment. While work was being done on the if-then system, research was being done and two papers were written on partial credit in ASSISTments. Partial credit was found out to be as accurate as knowledge tracing in determining student performance on the next problem. Once a partial credit algorithm was found, a study using if-then was analyzed. It was found that there was no statistically significant difference between students who were given a choice on their feedback and students who received no choice