Motivational and metacognitive feedback in an ITS: linking past states and experiences to current problems

Feedback is an important element in learning as it can provide learners with both information about progress as well as external motivational stimuli, providing them with an opportunity for reflection. Motivation and metacognition are strongly intertwined, with learners high in self-efficacy more likely to use a variety of self-regulatory learning strategies, as well as to persist longer on challenging tasks. Learning from past experience involves metacognitive processes as an act of reflecting upon one’s own experience and, coupled with existing knowledge, aids the acquisition and construction of further knowledge. The aim of the research was to improve the learner’s focus on the process and experience of problem solving while using an Intelligent Tutoring System (ITS), by addressing the primary question: what are the effects of including motivational and metacognitive feedback based on the learner’s past states and experiences? An existing ITS, SQL-Tutor, was used in a study with participants from first year undergraduate degrees studying a database module. The study used two versions of SQL-Tutor: the Control group used a base version providing domain feedback and the Study group used an extended version that also provided motivational and metacognitive feedback. Three sources of data collection were used: module summative assessments, ITS log files and a post-study questionnaire. The analysis included both pre-post comparisons and how the participants interacted with the system, for example their persistence in problem-solving and the degree to which they referred to past learning. Comparisons between groups showed some differing trends both in learning and behaviour in favour of the Study group, though these trends were not significantly different. The study findings showed promise for the use of motivational and metacognitive feedback based on the learners’ past states and experiences that could be used as a basis for future research work and refinement

Contextualised problem-based approach for teaching undergraduate database module

In this paper, a new approach has been used in teaching the second year undergraduate database module. The approach is a combination of contextualisation, problem-based approach, group work and continuous formative assessment. The contextualisation ensures the visibility of teaching/learning activities so that students are aware of the values of activities and how they can fit into a big picture. Problem-based approach gives the students tasks/problems to solve before the relevant lecture takes place, hence can better develop effective reasoning processes, independently learning skills and improve motivation and engagement. Group work is regularly used due to the diversity of student backgrounds and level of prior knowledge of certain topics. By having group work, students can learn from each other and easily clarify confusions among themselves before approaching the lecturer. This gives the lecture more time focusing on common issues. Formative assessment has also been used to support teaching/learning activities and to reinforce their understanding. The work in this paper has been evaluated via an end-of-year online module survey. The results show good effectiveness of the new approach, although there are still spaces for improvement

Supporting learning in intelligent tutoring systems with motivational strategies.

Motivation and affect detection are prominent yet challenging areas of research in the field of Intelligent Tutoring Systems (ITSs). Devising strategies to engage learners and motivate them to practice regularly are of great interest to researchers. In the learning and education domain, where students use ITSs regularly, motivating them to engage with the system effectively may lead to higher learning outcomes. Therefore, developing an ITS which provides a complete learning experience to students by catering to their cognitive, affective, metacognitive, and motivational needs is an ambitious yet promising area of research. This dissertation is the first step towards this goal in the context of SQL-Tutor, a mature ITS for tutoring SQL. In this research project, I have conducted a series of studies to detect and evaluate learners' affective states and employed various strategies for increasing motivation and engagement to improve learning from SQL-Tutor. Firstly, I established the reliability of iMotions to correctly identify learners' emotions and found that worked examples alleviated learners' frustration while solving problems with SQL-Tutor. Gamification is introduced as a motivational strategy to persuade learners to practice with the system. Gamification has emerged as a strong engagement and motivation strategy in learning environments for young learners. I evaluated the effects of gamified SQL-Tutor on undergraduate students and found that gamification indirectly improved learning by influencing learners’ time on task. It helped students by increasing their motivation which produce similar effects as intrinsically motivated students. Additionally, prior knowledge, gamification experience, and interest in the topic moderated the effects of gamification. Lastly, self-regulated learning support is presented as another strategy to affect learners’ internal motivation and skills. The support provided in the form of interventions improved students’ learning outcomes. Additionally, the learners' challenge-accepting behaviour, problem selection, goal setting, and self-reflection have improved with support without experiencing any negative emotions. This research project contributes to the latest trends of motivation and learning research in ITS

How much support is necessary for self-regulated learning?

Self-regulated learning is crucial for learning success, and is even of greater importance for online learning as there is less support and feedback available to students. We describe a simple intervention designed to support self-regulated learning in the context of SQL-Tutor, a mature intelligent tutoring system. SQL-Tutor logged data about all interactions students performed, including interactions with the SRL support. Frequency-based analyses did not identify any differences in behaviors of low or high scoring students. However, epistemic network analysis identified significant differences in how students use help available from SQL-Tutor. Students who scored low on the SQL test asked for high-level help (in the form of partial or full solution), copied the provided solutions and submitted them as their own. We conclude that additional support is necessary for students with weak self-regulation skills

การออกแบบแบบจําลองการให้ผลป้อนกลับบนฐานการรู้คิดของผู้เรียนในการเรียนรู้ภาษาสอบถามเชิงโครงสร้าง

A Design of a Feedback Model based on Student Metacognition in Learning Structured Query LanguageChusak Yathongchai, Thara angskun and Jitimon Angskunรับบทความ: 15 กุมภาพันธ์ 2561; ยอมรับตีพิมพ์: 15 พฤษภาคม 2561DOI: http://doi.org/10.14456/jstel.2018.5 บทคัดย่อบทความนี้นำเสนอการออกแบบแบบจำลองการให้ผลป้อนกลับตามการรู้คิดของผู้เรียน เพื่อนำไปใช้ในระบบการสอนเสริมอัจฉริยะที่ใช้รูปแบบการเรียนรู้แบบแก้ปัญหา โดยใช้เนื้อหาภาษาสอบถามเชิงโครงสร้างเป็นกรณีศึกษา บทความนี้ได้เสนอแนวทางการให้ผลป้อนกลับ 2 ด้าน คือ การคิดและการรู้คิด ในด้านการคิด ได้กำหนดรูปแบบการให้ผลป้อนกลับที่หลากหลายมีทั้งหมด 5 รูปแบบจากผลป้อนกลับ 4 ประเภท โดยใช้กลยุทธ์การให้ผลป้อนกลับแบบคงที่ 3 รูปแบบ คือ บอกความถูกต้องของคำตอบ บอกตำแหน่งที่ผิด และบอกเป็นนัย และกลยุทธ์การให้ผลป้อนกลับแบบปรับตัว 2 รูปแบบ คือ รูปแบบการให้ผลป้อนกลับตามลำดับ ได้แก่ บอกความถูกต้องของคำตอบ บอกตำแหน่งที่ผิด บอกเป็นนัย และบอกผลเฉลย โดยแต่ละโจทย์ปัญหาเริ่มต้นจากลำดับที่ 1 และรูปแบบให้ผลป้อนกลับ ตามลำดับ โดยลำดับเริ่มต้นของแต่ละโจทย์ปัญหาจะได้มาจากผลการแก้ไขโจทย์ปัญหาในข้อก่อนหน้านี้ ส่วนในด้านการรู้คิดได้ใช้คำถามสะท้อนการรู้คิดของผู้เรียน การสร้างแบบจำลองนี้ใช้ปัจจัยนำเข้าเกี่ยวกับการรู้คิด 4 ปัจจัย คือ ความเข้าใจในโจทย์ปัญหา ความยากของโจทย์ปัญหาตามความคิดของผู้เรียน ความมั่นใจในคำตอบ และความยากของโจทย์ปัญหา มีปัจจัยวิเคราะห์ประสิทธิภาพ 2 ปัจจัย คือ ระดับความพยายาม และเวลาที่ใช้แก้ปัญหา โดยแบบจำลองอยู่ในรูปของกฎที่มีจำนวน 33 กฎ ที่เป็นทางเลือกในการกำหนดรูปแบบการให้ผลป้อนกลับที่เหมาะสมตามการรู้คิดของผู้เรียนที่แตกต่างกัน ที่จะช่วยพัฒนาผู้เรียนทั้งทางด้านการคิดและการรู้คิด ผลการทดสอบประสิทธิภาพของแบบจำลองมีค่าความแม่นเฉลี่ยถ่วงน้ำหนักเท่ากับ 0.91 ส่วนค่าความเที่ยง ค่าความระลึก และค่าประสิทธิภาพโดยรวมมีค่าเฉลี่ยถ่วงน้ำหนักของทุกค่าเท่ากัน คือ 0.77 คำสำคัญ: การให้ผลป้อนกลับ  การรู้คิด  ระบบการสอนเสริมอัจฉริยะ  ภาษาสอบถามเชิงโครงสร้าง  Abstract The purpose of this research aimed to design a feedback model based on student metacognition. The design would be incorporated in an intelligent tutoring system using a problem–solving approach. Structured Query Language (SQL) teaching was used as a case study for the problem–solving approach. This research proposed feedback guidelines in both cognitive and metacognitive functions. Regarding the cognitive function, five feedback stra-tegies with four feedback types were determined. Three out of five were non–adaptive strategies called knowledge of the results, error flagging, and hints. The other two were adaptive strategies. One provided feedback in sequence of knowledge of the results, error flagging, hints, and knowledge of correct response. The other provided feedback in sequence based on results from the previous problem–solving. Regarding the metacognitive function, the reflective ques-tions were used to reflect metacognition of students. The designed model employed the four metacognitive factors as inputs. They were the learner’s perception of the problem understanding; the learner’s perception of the problem difficulty level; the confidence of answer; and the problem difficulty level. The effort level and time to solve the problem were factors used to determine the feedback strategies. There were thirty–three rules in the model used to decide the appropriate feedback strategies based on students’ metacognition. The model had potential to improve students both in terms of cognition and metacognition. The evaluation results indicated that the model had 0.91 of the weighted average of accuracy and 0.77 of the weighted average of precision, recall, and f-measure. Keywords: Feedback, Metacognition, Intelligent tutoring system, SQ

Supporting students in the analysis of case studies for professional ethics education

Intelligent tutoring systems and computer-supported collaborative environments have been designed to enhance human learning in various domains. While a number of solid techniques have been developed in the Artificial Intelligence in Education (AIED) field to foster human learning in fundamental science domains, there is still a lack of evidence about how to support learning in so-called ill-defined domains that are characterized by the absence of formal domain theories, uncertainty about best solution strategies and teaching practices, and learners' answers represented through text and argumentation. This dissertation investigates how to support students' learning in the ill-defined domain of professional ethics through a computer-based learning system. More specifically, it examines how to support students in the analysis of case studies, which is a common pedagogical practice in the ethics domain. This dissertation describes our design considerations and a resulting system called Umka. In Umka learners analyze case studies individually and collaboratively that pose some ethical or professional dilemmas. Umka provides various types of support to learners in the analysis task. In the individual analysis it provides various kinds of feedback to arguments of learners based on predefined system knowledge. In the collaborative analysis Umka fosters learners' interactions and self-reflection through system suggestions and a specifically designed visualization. The system suggestions offer learners the chance to consider certain helpful arguments of their peers, or to interact with certain helpful peers. The visualization highlights similarities and differences between the learners' positions, and illustrates the learners' level of acceptance of each other's positions. This dissertation reports on a series of experiments in which we evaluated the effectiveness of Umka's support features, and suggests several research contributions. Through this work, it is shown that despite the ill-definedness of the ethics domain, and the consequent complications of text processing and domain modelling, it is possible to build effective tutoring systems for supporting students' learning in this domain. Moreover, the techniques developed through this research for the ethics domain can be readily expanded to other ill-defined domains, where argument, qualitative analysis, metacognition and interaction over case studies are key pedagogical practices

Towards the Use of Dialog Systems to Facilitate Inclusive Education

Continuous advances in the development of information technologies have currently led to the possibility of accessing learning contents from anywhere, at anytime, and almost instantaneously. However, accessibility is not always the main objective in the design of educative applications, specifically to facilitate their adoption by disabled people. Different technologies have recently emerged to foster the accessibility of computers and new mobile devices, favoring a more natural communication between the student and the developed educative systems. This chapter describes innovative uses of multimodal dialog systems in education, with special emphasis in the advantages that they provide for creating inclusive applications and learning activities

Design of interactive visualization of models and students data

This document reports the design of the interactive visualizations of open student models that will be performed in GRAPPLE. The visualizations will be based on data stored in the domain model and student model, and aim at supporting learners to be more engaged in the learning process, and instructors in assisting the learners

Metamodel for personalized adaptation of pedagogical strategies using metacognition in Intelligent Tutoring Systems

The modeling process of metacognitive functions in Intelligent Tutoring Systems (ITS) is a difficult and time-consuming task. In particular when the integration of several metacognitive components, such as self-regulation and metamemory is needed. Metacognition has been used in Artificial Intelligence (AI) to improve the performance of complex systems such as ITS. However the design ITS with metacognitive capabilities is a complex task due to the number and complexity of processes involved. The modeling process of ITS is in itself a difficult task and often requires experienced designers and programmers, even when using authoring tools. In particular the design of the pedagogical strategies for an ITS is complex and requires the interaction of a number of variables that define it as a dynamic process. This doctoral thesis presents a metamodel for the personalized adaptation of pedagogical strategies integrating metamemory and self-regulation in ITS. The metamodel called MPPSM (Metamodel of Personalized adaptation of Pedagogical Strategies using Metacognition in intelligent tutoring systems) was synthetized from the analysis of 40 metacognitive models and 45 ITS models that exist in the literature. MPPSMhas a conceptual architecture with four levels of modeling according to the standard Meta- Object Facility (MOF) of Model-Driven Architecture (MDA) methodology. MPPSM enables designers to have modeling tools in early stage of software development process to produce more robust ITS that are able to self-regulate their own reasoning and learning processes. In this sense, a concrete syntax composed of a graphic notation called M++ was defined in order to make the MPPSM metamodel more usable. M++ is a Domain-Specific Visual Language (DSVL) for modeling metacognition in ITS. M++ has approximately 20 tools for modeling metacognitive systems with introspective monitoring and meta-level control. MPPSM allows the generation of metacognitive models using M++ in a visual editor named MetaThink. In MPPSM-based models metacognitive components required for monitoring and executive control of the reasoning processes take place in each module of an ITS can be specified. MPPSM-based models represent the cycle of reasoning of an ITS about: (i) failures generated in its own reasoning tasks (e.g. self-regulation); and (ii) anomalies in events that occur in its Long-Term Memory (LTM) (e.g. metamemory). A prototype of ITS called FUNPRO was developed for the validation of the performance of metacognitive mechanism of MPPSM in the process of the personalization of pedagogical strategies regarding to the preferences and profiles of real students. FUNPRO uses self-regulation to monitor and control the processes of reasoning at object-level and metamemory for the adaptation to changes in the constraints of information retrieval tasks from LTM. The major contributions of this work are: (i) the MOF-based metamodel for the personalization of pedagogical strategies using computational metacognition in ITS; (ii) the M++ DSVL for modeling metacognition in ITS; and (iii) the ITS prototype called FUNPRO (FUNdamentos de PROgramación) that aims to provide personalized instruction in the subject of Introduction to Programming. The results given in the experimental tests demonstrate: (i) metacognitive models generated are consistent with the MPPSM metamodel; (ii) positive perceptions of users with respect to the proposed DSVL and it provide preliminary information concerning the quality of the concrete syntax of M++; (iii) in FUNPRO, multi-level pedagogical model enhanced with metacognition allows the dynamic adaptation of the pedagogical strategy according to the profile of each student.Doctorad