Examples and tutored problems: How can self-explanation make a difference to learning?

“The final publication is available at link.springer.com”.Learning from worked examples has been shown to be superior to unsupported problem solving in numerous studies. Examples reduce the cognitive load on the learner's working memory, thus helping the student to learn faster or deal with more complex questions. Only recently researchers started investigating the worked example effect in Intelligent Tutoring Systems (ITSs). We conducted a study to investigate the effect of using worked examples in combination with supported problem-solving in SQL-Tutor. We had three conditions: Examples Only (EO), Problems Only (PO), and Alternating Examples/Problems (AEP). After completing a problem, students received a self-explanation prompt that focused on the concepts used in the problem, to make sure that students acquire conceptual knowledge. On the other hand, examples were followed by self-explanation prompts that focused on procedural knowledge. The study showed that the AEP and PO conditions outperformed EO in learning gain, while AEP outperformed PO in conceptual knowledge acquisition. Therefore, interleaving examples with supported problems is an optimal choice compared to using examples or supported problems only in SQL-Tutor

An Interaction Centred Approach to the teaching of Non-technical Skills in a Virtual Environment

In most domains involving expert knowledge, there is a number of cognitive and social factors influencing how efficient one human being is at correctly assessing and responding to certain situations. These factors, which contribute to the efficient and safe realization of a technical activity, are known as non-technical skills, and correspond to a wide range of cognitive proficiencies such as situation awareness, decision making, stress or fatigue management, but also social skills such as communication, leadership and team working. Different studies have shown the impact such skills can have in the successful resolving of a number of critical situations, even more so in our domains of interest which are medical surgery or driving. In this paper, we take a look at the difficulties raised by the teaching of the technical and non-technical skills mobilized during a critical situation, in the context of TEL within virtual environments. We present the advantages of using a combined enactive and situated learning approach to this problematic, and then take an ill-defined perspective to raise some important designing issues in this respect. We show that some aspects of this problem have not been encompassed yet in the ill-defined domains literature, and should be further studied in any attempt at teaching behaviours inducing technical and non-technical skills in a virtual world

Supporting students in the analysis of case studies for professional ethics education

Intelligent tutoring systems and computer-supported collaborative environments have been designed to enhance human learning in various domains. While a number of solid techniques have been developed in the Artificial Intelligence in Education (AIED) field to foster human learning in fundamental science domains, there is still a lack of evidence about how to support learning in so-called ill-defined domains that are characterized by the absence of formal domain theories, uncertainty about best solution strategies and teaching practices, and learners' answers represented through text and argumentation. This dissertation investigates how to support students' learning in the ill-defined domain of professional ethics through a computer-based learning system. More specifically, it examines how to support students in the analysis of case studies, which is a common pedagogical practice in the ethics domain. This dissertation describes our design considerations and a resulting system called Umka. In Umka learners analyze case studies individually and collaboratively that pose some ethical or professional dilemmas. Umka provides various types of support to learners in the analysis task. In the individual analysis it provides various kinds of feedback to arguments of learners based on predefined system knowledge. In the collaborative analysis Umka fosters learners' interactions and self-reflection through system suggestions and a specifically designed visualization. The system suggestions offer learners the chance to consider certain helpful arguments of their peers, or to interact with certain helpful peers. The visualization highlights similarities and differences between the learners' positions, and illustrates the learners' level of acceptance of each other's positions. This dissertation reports on a series of experiments in which we evaluated the effectiveness of Umka's support features, and suggests several research contributions. Through this work, it is shown that despite the ill-definedness of the ethics domain, and the consequent complications of text processing and domain modelling, it is possible to build effective tutoring systems for supporting students' learning in this domain. Moreover, the techniques developed through this research for the ethics domain can be readily expanded to other ill-defined domains, where argument, qualitative analysis, metacognition and interaction over case studies are key pedagogical practices

Intelligent Mentoring Systems for Making Meaning from Work Experience

This position paper presents a forward-looking view on addressing a long standing professional learning challenge faced by higher educational institutions, namely assisting students to make meaning from work-based experience and develop as reflexive professionals. We suggest that a synergetic approach, building on existing research in professional lifelong learning and intelligent learning environments and taking advantage of new opportunities provided by emerging technologies, will underpin a new breed of intelligent mentoring systems for professional learning. They will foster the learners’ meaning making process, as well as assist tutors in their roles as coaches/mentors

Reflective Experiential Learning: Using Active Video Watching for Soft Skills Training

Learning by watching videos has become the dominant way of learning for millennials. However, watching videos is a passive form of learning which usually results in a low level of engagement. As the result, video-based learning often results in poor learning outcomes. One of the proven strategies to increase engagement is to integrate interactive activities such as quizzes and assessment problems into videos. Although this strategy increases engagement, it requires changing existing videos and therefore substantial effort from the teacher. We have developed the Active Video Watching (AVW) system that enables the teacher to use existing videos from YouTube without modifications. The teacher is required to define a set of aspects for videos, which serve as reflective scaffolds in order to increase engagement and focus learners’ thinking. AVW provides a Personal Space for individual learners to link their personal experiences while watching videos. The comments collected can be used by the individuals to reflect on their own thoughts or to be shared with other learners in the Social Space. We conducted a study with postgraduate students on presentation skills. The results show that the level of engagement with AVW was high, and that the aspects were effective as reflection prompts. We plan to conduct further studies related to other types of soft skills, and also to further extend AVW to provide individualized feedback to students

Clustering student interaction data using Bloom's Taxonomy to find predictive reading patterns

In modern educational technology we have the ability to capture click-stream interaction data from a student as they work on educational problems within an online environment. This provides us with an opportunity to identify student behaviours within the data (captured by the online environment) that are predictive of student success or failure. The constraints that exist within an educational setting provide the ability to associate these student behaviours to specific educational outcomes. This information could be then used to inform environments that support student learning while improving a student’s metacognitive skills. In this dissertation, we describe how reading behaviour clusters were extracted in an experiment in which students were embedded in a learning environment where they read documents and answered questions. We tracked their keystroke level behaviour and then applied clustering techniques to find pedagogically meaningful clusters. The key to finding these clusters were categorizing the questions as to their level in Bloom’s educational taxonomy: different behaviour patterns predicted success and failure in answering questions at various levels of Bloom. The clusters found in the first experiment were confirmed through two further experiments that explored variations in the number, type, and length of documents and the kinds of questions asked. In the final experiment, we also went beyond the actual keystrokes and explored how the pauses between keystrokes as a student answers a question can be utilized in the process of determining student success. This research suggests that it should be possible to diagnose learner behaviour even in “ill-defined” domains like reading. It also suggests that Bloom’s taxonomy can be an important (even necessary) input to such diagnosis

Un modèle pour la génération d'indices par une plateforme de tuteurs par traçage de modèle

La présente thèse décrit des travaux de recherche effectués dans le domaine des systèmes tutoriels intelligents (STI). Plus particulièrement, elle s'intéresse aux tuteurs par traçage de modèle (MTT). Les MTTs ont montré leur efficacité pour le tutorat de la résolution de tâches bien définies. Par contre, les interventions pédagogiques qu'ils produisent doivent être incluses, par l'auteur du tuteur, dans le modèle de la tâche enseignée. La recherche effectuée répond à cette limite en proposant des méthodes et algorithmes permettant la génération automatique d'interventions pédagogiques. Une méthode a été développée afin de permettre à la plateforme Astus de générer des indices par rapport à la prochaine étape en examinant le contenu du modèle de la tâche enseignée. De plus, un algorithme a été conçu afin de diagnostiquer les erreurs des apprenants en fonction des actions hors trace qu'ils commettent. Ce diagnostic permet à Astus d'offrir une rétroaction par rapport aux erreurs sans que l'auteur du tuteur ait à explicitement modéliser les erreurs. Cinq expérimentations ont été effectuées lors de cours enseignés au département d'informatique de l'Université de Sherbrooke afin de valider de façon empirique les interventions générées par Astus. Le résultat de ces expérimentations montre que 1) il est possible de générer des indices par rapport à la prochaine étape qui sont aussi efficaces et aussi appréciés que ceux conçus par un enseignant et que 2) la plateforme Astus est en mesure de diagnostiquer un grand nombre d'actions hors trace des apprenants afin de fournir une rétroaction par rapport aux erreurs

Augmented Conversation and Cognitive Apprenticeship Metamodel Based Intelligent Learning Activity Builder System

This research focused on a formal (theory based) approach to designing Intelligent Tutoring System (ITS) authoring tool involving two specific conventional pedagogical theories—Conversation Theory (CT) and Cognitive Apprenticeship (CA). The research conceptualised an Augmented Conversation and Cognitive Apprenticeship Metamodel (ACCAM) based on apriori theoretical knowledge and assumptions of its underlying theories. ACCAM was implemented in an Intelligent Learning Activity Builder System (ILABS)—an ITS authoring tool. ACCAM’s implementation aims to facilitate formally designed tutoring systems, hence, ILABS―the practical implementation of ACCAM― constructs metamodels for Intelligent Learning Activity Tools (ILATs) in a numerical problem-solving context (focusing on the construction of procedural knowledge in applied numerical disciplines). Also, an Intelligent Learning Activity Management System (ILAMS), although not the focus of this research, was developed as a launchpad for ILATs constructed and to administer learning activities. Hence, ACCAM and ILABS constitute the conceptual and practical contributions that respectively flow from this research. ACCAM’s implementation was tested through the evaluation of ILABS and ILATs within an applied numerical domain―the accounting domain. The evaluation focused on the key constructs of ACCAM―cognitive visibility and conversation, implemented through a tutoring strategy employing Process Monitoring (PM). PM augments conversation within a cognitive apprenticeship framework; it aims to improve the visibility of the cognitive process of a learner and infers intelligence in tutoring systems. PM was implemented via an interface that attempts to bring learner’s thought process to the surface. This approach contrasted with previous studies that adopted standard Artificial Intelligence (AI) based inference techniques. The interface-based PM extends the existing CT and CA work. The strategy (i.e. interface-based PM) makes available a new tutoring approach that aimed fine-grain (or step-wise) feedbacks, unlike the goal-oriented feedbacks of model-tracing. The impact of PM—as a preventive strategy (or intervention) and to aid diagnosis of learners’ cognitive process—was investigated in relation to other constructs from the literature (such as detection of misconception, feedback generation and perceived learning effectiveness). Thus, the conceptualisation and implementation of PM via an interface also contributes to knowledge and practice. The evaluation of the ACCAM-based design approach and investigation of the above mentioned constructs were undertaken through users’ reaction/perception to ILABS and ILAT. This involved, principally, quantitative approach. However, a qualitative approach was also utilised to gain deeper insight. Findings from the evaluation supports the formal (theory based) design approach—the design of ILABS through interaction with ACCAM. Empirical data revealed the presence of conversation and cognitive visibility constructs in ILATs, which were determined through its behaviour during the learning process. This research identified some other theoretical elements (e.g. motivation, reflection, remediation, evaluation, etc.) that possibly play out in a learning process. This clarifies key conceptual variables that should be considered when constructing tutoring systems for applied numerical disciplines (e.g. accounting, engineering). Also, the research revealed that PM enhances the detection of a learner’s misconception and feedback generation. Nevertheless, qualitative data revealed that frequent feedbacks due to the implementation of PM could be obstructive to thought process at advance stage of learning. Thus, PM implementations should also include delayed diagnosis, especially for advance learners who prefer to have it on request. Despite that, current implementation allows users to turn PM off, thereby using alternative learning route. Overall, the research revealed that the implementation of interface-based PM (i.e. conversation and cognitive visibility) improved the visibility of learner’s cognitive process, and this in turn enhanced learning—as perceived

Revisiting Ill-Definedness and the Consequences for ITSs

ITSs for ill-defined domains have attracted a lot of attention recently, which is well-deserved, as such ITSs are hard to develop. The first step towards such ITSs is reaching a wide agreement about the terminology used in the area. In this paper, we discuss the two important dimensions of ill-definedness: the domain and the instructional task. By the domain we assume declarative domain knowledge, or the domain theory, while the instructional task is the task the student is learning, in terms of problem-solving skills. It is possible to have a well-defined domain and still have ill-defined instructional tasks in the same domain. We look deeper at the features of ill-defined tasks, which all contribute to their ill/well defined nature. The paper discusses model-tracing and constraint-based modeling, in terms of their suitability for ill-defined tasks and domains. We show that constraint-based modeling can be used in both well- and ill-defined domains, and illustrate our conclusion using several instructional tasks

Astus, une plateforme pour créer et étudier les systèmes tutoriels intelligents « par traçage de modèle »

Cette thèse s’intéresse aux systèmes tutoriels intelligents (STI), un type d’environnement informatique pour l’apprentissage humain (EIAH) qui se distingue des autres (p. ex. les exerciseurs et les hypermédias éducatifs) en offrant un mécanisme d’évaluation plus sophistiqué. Parmi les différentes familles de STI, ce sont les STI « par traçage de modèle » (MTT) qui ont le plus fait leurs preuves. Les MTT sont critiqués, premièrement parce qu’ils évaluent l’apprenant de façon serrée (c.-à-d. qui positionne l’action de l’apprenant par rapport à une ou plusieurs méthodes pour effectuer la tâche), ce qui n’est possible que pour des tâches bien définies. Par conséquent, on leur reproche d’encourager un apprentissage superficiel. Deuxièmement, parce que les efforts de création qu’ils requièrent sont jugés prohibitifs, ce qui a mené à l’apparition d’autres familles de STI, comme les STI « par contraintes » et les STI « par traçage d’exemples » et ceux basés sur l’apprentissage automatique. Par cette thèse, nous voulons contribuer à renouveler l’intérêt pour les MTT en améliorant le rapport entre les efforts de création et l’efficacité potentielle des interventions, et en établissant plus clairement leur rôle pédagogique. Pour ce faire, nous proposons la plateforme Astus qui permet d’explorer l’espace qui existe entre les MTT créés avec les plateformes existantes, et des MTT dédiés ayant recours à des connaissances didactiques sophistiquées (p. ex. des dialogues) qui exigent des efforts de création encore plus importants. La plateforme Astus se distingue des plateformes existantes parce qu’elle génère des interventions plutôt que de recourir à des interventions prémâchées et qu’elle supporte les tâches s’effectuant dans des environnements qui ont une dimension physique. La génération des interventions dépend : d’un modèle de la tâche qui s’inscrit dans le paradigme du tuteur, c’est-à-dire qui représente une abstraction et une généralisation des instructions d’un tuteur humain; d’un modèle de l’UI qui permet des interventions riches comme une démonstration (c.-à-d. déplacements du pointeur et simulation des clics et des saisies); de langages dédiés et d’outils qui réduisent les efforts de création des auteurs; de mécanismes d’extension qui permettent d’adapter la génération en fonction d’une stratégie pédagogique particulière. Le paradigme du tuteur, parce qu’il favorise une communication transparente entre le système et l’apprenant, met en évidence les avantages et les désavantages de l’approche pédagogique des MTT, essentiellement une évaluation précise (c.-à-d. qui permet de produire des indices sur la prochaine étape et des rétroactions sur les erreurs), mais serrée. En s’inscrivant explicitement le paradigme du tuteur, entre autres en évitant de tirer profit de la nature de domaines particuliers ou de propriétés de tâches particulières pour assouplir l’évaluation, la plateforme Astus se démarque plus nettement des autres familles de STI que les autres MTT. Par conséquent, elle établit plus clairement le rôle pédagogique des MTT. Cinq expérimentations (menées par Luc Paquette) à petite échelle ont été réalisées auprès d’étudiants au baccalauréat au département d’informatique (un laboratoire pour la manipulation d’arbres binaires de recherche et un pour la conversion de nombres en virgule flottante). Ces expérimentations indiquent que les interventions générées sont efficaces. Au-delà de ces résultats, c’est le processus entourant ces expérimentations, parce qu’il est comparable au processus des chercheurs potentiellement intéressés par la plateforme Astus, qui montre que la version présentée dans cette thèse est plus qu’un prototype et qu’elle peut être utilisée à l’interne dans un contexte réel