Towards the Discovery of Learner Metacognition from Reflective Writing

Modern society demands renewed attention on the competencies required to best equip students for a dynamic and uncertain future. We present exploratory work based on the premise that metacognitive and reflective competencies are essential for this task. Bringing the concepts of metacognition and reflection together into a conceptual model within which we conceived of them as both a set of similar features, and as a spectrum ranging from the unconscious inner-self through to the conscious, external, social self. This model was used to guide exploratory computational analysis of 6,090 instances of reflective writing authored by undergraduate students. We found the conceptual model useful in informing the computational analysis, which in turn showed potential for automating the discovery of metacognitive activity in reflective writing, an approach that holds promise for the generation of formative feedback for students as they work towards developing core 21st century competencies

Exploring Long-Term Impacts of Self-Regulated Learning Interventions in K-12 Contexts: A Systematic Review

While research illustrates the benefits of interventions designed to improve self-regulated learning (SRL) and academic achievement, far fewer studies have examined the durability of these effects. This review synthesizes research on the lasting effects of 17 comprehensive SRL interventions on variables related to metacognition, cognition, motivation, and achievement in K-12 populations. Results reveal common patterns of design, domain-specificity, intervention complexity, and style of measurement instrument. Intervention effects tend to be durable regarding achievement and SRL but were mixed when presented across multiple measures of SRL. Overall findings imply that SRL interventions can lead to enduring effects on achievement and better achievement outcomes than content-strategy instruction alone and can be implemented successfully in a variety of contexts and subjects

Supporting Adult Learners\u27 Metacognitive Development with a Sociotechnical System

Metacognition is defined as thinking about and reflecting on one\u27s cognitive processes. In learning contexts, strong metacognition leads to retention, academic success, and deep learning. While we know a lot about the metacognition of learners in grades K-12 and college, there are limited studies on adult learners\u27 (24 and older) metacognitive awareness, how to support it, or the role technology can play, particularly since e-learning is quickly becoming the central mode of learning for adult learners. Thus, I have the following motivating research question: How can we support adult learners\u27 metacognitive development in e-learning environments? To better understand adult learners\u27 needs, I conducted a content analysis of adults\u27 learning ePortfolios and surveyed a cross-section of adult learners to determine their metacognitive awareness. Based on those findings and the literature on designing learning technologies for adult learners, I iteratively designed and developed a web-based application with adult learning, social learning, and persuasive design elements. During two sections of an online course, a treatment group used the intervention and a control group did not. Both groups completed a pre-/post-self report of their metacognitive awareness, developed a learning portfolio that was rated by two raters for evidence of metacognition, and participated in interviews. This research shows that (a) adult learners are adept at planning and monitoring their learning but need more support in managing information and evaluating their learning; (b) a web-based intervention with social-persuasive design elements supports adult learners in metacognitive development; and (c) social and persuasive design elements, when aligned with adult learning principles, support adult learners\u27 narrative identity, which I argue is a key factor in supporting their metacognitive development. This research aims to provide designers, educators, and learners with a better understanding of adult learners needs and offers design principles and guidelines for development of sociotechnical systems that can promote their metacognitive development in e-learning environments

Vexique: Vocabulary enhancement software for French immersion students

This dissertation developed and tested online writing analytics software, Vexique, designed to improve French immersion (FI) high school students’ lexical richness by promoting use of alternative vocabulary in phrasing an essay. Vexique’s features were grounded in the Hulstijn and Laufer (2001) Involvement Load Hypothesis for second language (L2) vocabulary acquisition, whereby giving more attention to processing lexical information encourages vocabulary acquisition and retention in L2. Forty-five Grade 12 FI students participated. Students wrote two argumentative essays. Vexique provided quantified feedback of vocabulary and usage on the first essay that afforded making lexical improvements prior to submission. To test effects of the software’s analytics, students wrote a second essay without feedback. Lexical richness increased after learners received prompts about their first essay. Results showed statistically detectable benefits to lexical richness indicated by lexical density and diversity. Results also indicated no statistically detectable difference in repetitive content words in the second essay

An Investigation of the Relationships Between the Scoring Rubrics Inventory and the Metacognitive Awareness Inventory as Reported by Secondary School Core-Subject Teachers

The promise of increased student achievement through educational reform is delivered still-born if teachers do not know how to implement complex instructional practices and sophisticated analysis of student performance. Metacognitive awareness is crucial to the adoption and application of proven educational initiatives. Teachers who successfully implement criterion-referenced assessment instruction, scoring rubrics, transfer to their students the metacognitive knowledge and skills of how to learn. This study is predicated on the research assumptions that metacognition and its attendant skills are critical to the successful implementation of scoring rubrics. A researcher-developed instrument, the Scoring Rubrics Inventory (SRI) and the Metacognitive Awareness Inventory (MAI) were distributed to core-subject teachers from three large public schools in Southwest Louisiana. From a population of sixty-eight (N=68) voluntary participants, eighteen teacher-participants self-reported as high implementers of scoring rubrics, thirty-nine as mid-level implementers and eleven as low-level implementers. From this population, twelve subjects were randomly selected (four high, four mid-level, and four lowlevel implementers) by an outside rater for double-blind observations and interviews. Pearson Product Moment correlations of the SRI and the MAI revealed five significant pairings using an alpha level of .05. The statistical results, coupled with the observation and interview findings from the sample-subjects established the consistency and stability of the Scoring Rubrics Inventory. Further, the totality of the results reported here support the research hypothesis of the study: H1: There is a significant correlation between the metacognitive awareness of secondary school core-subject teachers and the successful implementation of scoring rubrics. The results of the study indicated that secondary school core-subject teachers who successfully implement scoring rubrics possess a metacognitive awareness that transcends professional development training. The findings also suggested that teacher-participants who do not implement scoring rubrics either cannot or lack commitment to the innovation. Implications for teacher educators and school leaders indicated the need to: identify those persons who require additional professional development training; include operational strategies and modeling of successful implementation during training; and maintain a consistent training program in scoring rubrics. Recommendations for future research were offered

Prompting Self-Monitoring of Learning in Self-Paced Computer Based Training: The Effect on Self-Regulation and Learning

The purpose of this dissertation was to investigate the effects of prompting students to monitor their use of learning strategies and comprehension while completing self-paced, work-related training in a computer-based learning environment. Study participants included 94 enlisted military volunteers, randomly assigned to one of three groups in the spring of 2012. Changes in strategy use and comprehension were evaluated within and between groups receiving either immediate, delayed or no prompts using multiple methods of measurement, both during and after training. Prompts asked participants to rate their level of agreement to statements regarding their strategy use and comprehension of lesson content. Dependent variables included declarative knowledge and self-regulation. Declarative knowledge was measured using multiple end-of-lesson tests and a comprehensive end-of-course test. Self-regulation of strategy use was measured using a post-treatment self-report instrument and strategy use scores derived from an evaluation of learner notes. Independent variables included prompts to self-monitor performance; prior knowledge was used as a covariate in all analyses. Multivariate analysis of covariance was used to investigate the effects of the prompts on the combination of self-regulation and comprehension scores at the end of training. Mixed model repeated measures analysis of covariance was used to investigate changes in self-regulation and strategy use during training. Analysis of results revealed no statistically significant effects of the prompting treatments on combined scores of self-regulation and comprehension by the end of the treatment between groups. Furthermore, there were no significant effects of the prompts on strategy use or comprehension over time between groups. Findings from this study suggest the addition of prompts in computer-based learning events may not be effective for all learners or learning tasks. In contrast to similar experiments with college students, the prompts failed to influence participant strategy use and learning. Although groups receiving prompts invested more time in training, the additional time did not lead to improved overall strategy use or comprehension scores in comparison to the group that did not receive prompts. By the end of training, average comprehension scores among groups was equivalent and, on average, below passing (80%). The lack of effect on strategy use may have been a result of participants\u27 low prior knowledge, proficiency with learning strategies, task complexity and the value participants assigned to the learning task. Findings from this study expand the existing body of knowledge regarding the self-regulation of learning in computer-based learning environments, particularly with regard to the population of working adults, whose self-regulation of learning in the workplace has not been extensively investigated. Additionally, this study provides an example of how to employ multiple measures of self-regulation to more fully describe self-regulatory processes in computer-based learning environments, an approach researchers investigating self-regulation have called for

Evaluating the use of automated writing evaluation programs as tools for formative feedback in English as a Second Language postgraduate students

Automated Writing Evaluation (AWE) programs use natural language processing techniques to analyse texts and score them according to pre-defined parameters. Most of these programs also offer feedback functions, providing students with feedback on various aspects of writing. These programs have been increasingly used in English as a second language (ESL) and English as a foreign language (EFL) classrooms to provide formative feedback to students, especially in the context of academic writing. However, existing research into the use of AWE programs has focused more on the end product of revision with AWE programs and whether the use of these can provide quantifiable gains in error reduction and holistic scores. Little research has investigated how these programs can be integrated into the writing process or what pedagogical approaches result in these technologies being incorporated into the classroom in ways that help the students develop their writing skills. This results in two major gaps in current literature on the use of AWE programs: 1) there is little information regarding how students engage with the feedback they receive and how they decide which feedback to use, and 2) scores and error rates can only give a superficial, post-hoc understanding of the effects of AWE in revision, but tells little about the depth of the changes made. The research showcased in this thesis seeks to address these gaps by presenting the results of two studies designed to improve our knowledge of how students use AWE programs. In the first study, screen captures and think-aloud protocols were used to record the interactions of 11 ESL postgraduate students with an AWE program during four revision sessions. The recordings were analysed to identify self-regulation strategies and decision-making strategies used by students when engaging with the AWE feedback. In the second study, a web program was created to collect texts before and after the use of an AWE program for revision, collecting a total of 30 texts suitable for analysis. The texts were compared before and after revision with an AWE program to understand the extent of the changes made during revision by analysing uptake rates and linguistic markers of proficiency to quantify the effects AWE feedback had on the revision of these texts. Results from both studies suggest students are selective in their use of AWE feedback, drawing from a variety of sources of previous knowledge about English grammar and academic writing conventions to decide whether to accept or reject AWE feedback and this selectiveness results in low feedback uptake and little changes to the texts. These sources include feedback from teachers and mentors, previous exposure to academic texts and knowledge gained from previous English and composition classes. Successful integration of AWE programs into ESL/EFL classrooms should therefore take into account how students engage with the AWE feedback and use that knowledge in pedagogical strategies that scaffold student use of AWE programs and help them develop the cognitive and metacognitive skills need to successfully navigate the feedback they receive

Investigating Learning Difficulties at Romanian Language and Literature Subject in Perspective of Learning to Learn Competence Development

AbstractLearning how to learn can be particularly difficult for students with learning difficulties. Each student may encounter certain problems at one or more school subjects in a certain time, problems that seem to be insurmountable and may become constant throughout schooling. This study investigates the frequencies of students’ learning difficulties from 5-12th grades on the cognitive, metacognitive and non-cognitive dimensions in Romanian language and literature subject. Participants in the study were 81 Romanian language and literature teachers from Cluj county, who were asked about the students’ learning difficulties in order to design modalities to overcome them

What Do We Think We Think We Are Doing?: Metacognition and Self-Regulation in Programming

Metacognition and self-regulation are popular areas of interest in programming education, and they have been extensively researched outside of computing. While computing education researchers should draw upon this prior work, programming education is unique enough that we should explore the extent to which prior work applies to our context. The goal of this systematic review is to support research on metacognition and self-regulation in programming education by synthesizing relevant theories, measurements, and prior work on these topics. By reviewing papers that mention metacognition or self-regulation in the context of programming, we aim to provide a benchmark of our current progress towards understanding these topics and recommendations for future research. In our results, we discuss eight common theories that are widely used outside of computing education research, half of which are commonly used in computing education research. We also highlight 11 theories on related constructs (e.g., self-efficacy) that have been used successfully to understand programming education. Towards measuring metacognition and self-regulation in learners, we discuss seven instruments and protocols that have been used and highlight their strengths and weaknesses. To benchmark the current state of research, we examined papers that primarily studied metacognition and self-regulation in programming education and synthesize the reported interventions used and results from that research. While the primary intended contribution of this paper is to support research, readers will also learn about developing and supporting metacognition and self-regulation of students in programming courses