Accuracy of chemistry performance evaluation of BSc four-year programme students : a case study

The ability to make realistic judgements of one’s performance is a demonstration of the possession of strong metacognitive skills. Metacognition involves the monitoring of one’s progress during learning, and the ability to modify learning strategies for increased effectiveness. Poor-performing students are at risk because they generally exhibit high levels of overconfidence when evaluating their performance, and may fail to adjust their learning strategies in time. This study aims to explore the accuracy with which students in the BSc Four-year programme (BFYP) of the University of Pretoria evaluate their performance in a stoichiometry test, as well as the influence of teaching on test performance and on accuracy of performance evaluation. The factors that students rely on when making performance evaluations as well as shifts in the reliance on these factors after teaching are explored. Finally, the study examines the relationship between bias in performance evaluation and the self-protection, self-enhancement motivational factors and gender. Data were collected by means of a three-tier stoichiometry test instrument, administered as pre- and posttest, as well as a questionnaire administered simultaneously with the pretests to a sample of 91 students. Each test item comprised a stoichiometry question, a confidence rating and a free-response explanation for the choice of confidence rating. The confidence rating was interpreted as an indication of expected performance. The test instrument allowed for the investigation of bias in performance evaluation in the pre- and posttests, the exploration of factors that students rely on when making performance evaluations and how the reliance on these factors shifted in the posttests. The questionnaires were used to collect data on self-enhancement, self-protection and gender. The study shows that the majority of the students were overconfident in the evaluation of their performance in both the pre- and posttests. Performance improved significantly in the posttest but accuracy of performance evaluation did not. Students were categorised as overconfident (OC), realistic (R) or under-confident (UC) based on the difference between actual and expected performance. Five subgroups were defined on the basis of accuracy of performance evaluation in the pre- and posttests. The five subgroups, labelled first by their pretest and then their posttest category, were the OC-OC (50 students), OC-R (13 students), R-R (11 students), R-OC (15 students) and the R-UC (2 students) subgroups. The results indicated no significant difference between the pre-knowledge and ability of the students in the four main subgroups. The students differed significantly in terms of performance in the posttest, their pre- and posttest average confidence scores and in performance gain. A significant difference was not found with regard to performance in the CMY 143 end of semester examination. These findings confirmed that we were dealing with four discrete subgroups with different characteristics. The OC-R subgroup achieved the highest learning gain by a significant margin. Moderate learning gains were demonstrated by the R-R and OC-OC subgroups and the R-OC subgroup did not achieve any learning gain at all. Careful analysis of qualitative data revealed that accuracy in the evaluation of posttest performance was associated with both a reduction in the prevalence of vague subjective judgments and with higher performance gain. Similarly, an increase in the tendency to base metacognitive monitoring on vague global judgments of performance in the posttest was associated with reduced accuracy of self-evaluation and lower learning gain. The tendency by the four performance evaluation subgroups to self-enhance or self-protect was not found to be statistically different. P-values greater than 0.05 in the pre- and posttests indicated that males and females were not significantly different in their accuracy of performance evaluation. The study suggests that an element of bias in performance evaluation may be beneficial to learning. Inaccuracy in self-evaluation in the pretest did not hamper learning for both the OC-OC and OC-R subgroups. Students who were over-optimistic about their performance in the pretest may have been less intimidated by the challenges of the new content material than those who were better calibrated (R-R and R-OC subgroups). Students who remained overconfident in the posttest, i.e. in the OC-OC subgroup did not gain from the learning experience as much as those who entered overconfident but became better calibrated. Those who entered tentatively as realists and then, with a little exposure, became unrealistic in their performance evaluation were shown to be the most vulnerable based on their lack of learning gain. Furthermore, increasing content knowledge alone may not be enough to raise the metacognitive ability of students. Finally, chemistry educators should be aware that students often make vague subjective judgements of performance even on a topic like stoichiometry, which requires predominantly procedural knowledge and formal reasoning. Our study has shown that this deficiency, when associated with poor accuracy of self-evaluation, may hamper learning gain. CopyrightDissertation (MSc)--University of Pretoria, 2012.Chemistryunrestricte

Factors underlying metacognitive judgements in foundation chemistry

Poor performing students have been found to generally report unjustified high levels of confidence in their academic performance. The aim of this study was to elucidate the reasons students provide as justifications for their perceived academic performance in a chemistry test. As part of a larger mixed methods study, students enrolled in a foundation chemistry course (N = 91) were required to justify their level of confidence in the accuracy of their responses during a pre- and posttest set on a specific chemistry topic. The qualitative data was analysed for the purposes of the current study and is presented in this paper. The students’ responses shed light on the explanations students give as justifications for their confidence judgements and the influence of teaching on these explanations.http://www.ejmste.comam2020Science, Mathematics and Technology Educatio

Manifestations of metacognitive activity during the collaborative planning of chemistry practical investigations

This paper elaborates a process followed to characterise manifestations of cognitive regulation during the collaborative planning of chemistry practical investigations. Metacognitive activity was defined as the demonstration of planning, monitoring, control and evaluation of cognitive activities by students while carrying out the chemistry task. Inherent in collaborative learning is the social aspect of metacognition, which in this study was evidenced in social cognitive regulation (notably of intra- and interpersonal metacognitive regulations) as groups of students went about planning their practical investigations. Discussions of two of the learning groups (n =4; n = 3) as they planned the extended practical investigation were recorded, transcribed and analysed for indicators of any inherent metacognitive activity. The process of characterising the manifestations of metacognition resulted in the development of a coding system which specifies not only the regulatory strategies at play but the type of regulation (self or other), the area of regulation (cognition, task performance or behaviour) as well as the depth of regulatory contributions (high or low). The fine-grained coding system allowed for a finer theoretical elucidation of the social nature of metacognition. The implications of this study for metacognition and chemistry education research are highlighted.The National Research Foundation (South Africa), Canon Collins Trust and Legal administered Ros Moger Terry Furlong Scholarship, Association for Commonwealth Universities and University of Pretoria postgraduate research support.http://www.tandfonline.com/loi/tsed202018-12-21hj2017ChemistryScience, Mathematics and Technology Educatio

Metacognitive monitoring and learning gain in foundation chemistry

The ability to make realistic judgements of one’s performance is a demonstration of the possession of strong metacognitive skills. In this study we investigate the relationship between accuracy of selfevaluation as an expression of metacognitive skill, and learning gain in stoichiometry. The context is an academic development programme at a South African University offered for under-prepared students enrolled for science and engineering. These students generally exhibit unrealistically high levels of confidence in performance and this could potentially place them at risk by negatively affecting decisions regarding time management and self-regulation. We investigated whether overconfidence before instruction is corrected upon exposure to teaching. A three-tier stoichiometry test was used to collect qualitative and quantitative data before and after instruction. Findings indicate that the majority of the students were overconfident in the evaluation of their performance in both the pre- and posttests. Overconfidence was not a debilitating disposition when demonstrated in the pretest provided that it was corrected during teaching and learning. The most vulnerable students were those that judged their performance or lack thereof realistically in the pretest but became overconfident during the teaching and learning of stoichiometry. Our results suggest that under-prepared students are slow in developing accurate metacognitive monitoring skills within a classroom environment that did not include instruction focused on the development of such skills. We recommend a proactive and constructive response by educators which may reduce the incidence of failure and preserve the positive contribution of confidence, albeit excessively positive.NRF funding, Graça Machel Scholarship for women.http://www.rsc.org/cerphb2014ai201

The influence of an inquiry-based approach on grade four learners' understanding of the particulate nature of matter in the gaseous phase : a case study

A qualitative pre-/post-test case study was conducted to explore the influence of inquiry-based education in eliciting learners’ understanding of the particulate nature of matter in the gaseous phase. Two grade four classes (n=116) were conveniently and purposively sampled from two farm schools in Pretoria, South Africa. Data was collected through pre-test, initial group interviews, post-test, final group interviews and field notes. The pre-intervention data indicated that the continuous, continuousanimistic and the continuous empty models of matter in the gaseous phase dominated learners’ understanding in all the four classes. A considerable decrease in the continuous model was observed in the inquiry classes rather than in the lecture classes post-intervention.The South African National Research Foundationhttps://www.ejmste.compm2021Science, Mathematics and Technology Educatio

An inquiry-based practical curriculum for organic chemistry as preparation for industry and postgraduate research

This paper describes the development of a new practical curriculum for third-year organic chemistry to replace the recipe-based approach typically used in undergraduate teaching laboratories. The new curriculum consists of an inquiry-based project set in a simulated industrial context preceded by two scaffolding experiments to prepare students for the task. The industrial project requires students to evaluate experimentally three multi-step synthetic routes to a given target based on cost, technical challenge and environmental impact in order to make a recommendation as to which route the ‘company’ should use to synthesize the compound. The project equips students with technical skills suitable for both postgraduate research and industry, and develops metacognition and understanding through the use of the jig-saw cooperative learning strategy and reflection. The students were found to engage with the practical work at a deep intellectual level, demonstrating that contextualized inquiry-based laboratory teaching afforded an improved quality of learning. In addition, the reported practical curriculum made a difficult subject accessible and even popular, to some measure grew the students’ ability in all desired graduate attributes and resulted in the establishment of a professional identity for individual students.Education Innovation grant from the University of Pretoria.http://saci.co.za/journalam201

Manifestations of metacognitive activity in an upper undergraduate organic chemistry laboratory

This study was carried out to investigate how metacognitive activity, particularly cognitive regulation, manifests in the collaborative planning of chemistry practical investigations by senior undergraduate students in a simulated industrial project. The participating students worked in home groups to evaluate three synthetic routes for a given compound, and decided on a preferred route while considering the criteria of cost, technical challenge, and environmental impact. This is consistent with the jigsaw learning technique. During the planning session, the students who were evaluating the same synthetic routes convened in specialist groups to draw up detailed experimental procedures for their routes. Audio recordings of the two specialist and the four home group discussions were purposively selected, transcribed, and analysed for manifestations of metacognitive regulation. This study started with a partial theory of what constitutes cognitive regulation in collaborative group discussions, and as the research progressed, verbal indicators of each component of cognitive regulation were inductively determined from analysing the pilot study data. These were then compiled into a coding scheme. The coding scheme was further refined following recommendations of an analytic audience. The students' self-reports were collected through retrospective stimulated recall interviews and were used to triangulate the findings inferred from the group discussions. This study has made important theoretical and methodological contributions. The coding scheme proved to be both conceptually and methodologically useful in that it allowed for fine-grained coding. The system of coding interrogated not only the manifestations of metacognitive regulation at play (planning, monitoring, control and evaluation), but facilitated an in-depth look at the types of regulation, i.e. self or other, the areas where students applied their efforts towards regulation (cognition, behaviour and task performance) as well as the depth of cognitive regulation (low or high). The coding scheme went beyond serving as a tool for characterising manifestations of metacognitive activity, it developed into a framework which provides a finer theoretical elucidation of the social nature of metacognition. I show in this thesis how in group work metacognitive activity was found to be predominantly other-regulatory, manifesting mostly as control and monitoring, with much fewer instances of planning and evaluation. These observations were made across all groups despite the differences in social context. The low occurrence of planning, evaluation and high-level regulation seemed to suggest a hierarchy in terms of the level of difficulty of metacognitive regulation. An even deeper look revealed that individual patterns of regulation differed in terms of individual dispositions and personal goals. Investigating the transferability of the individual patterns of regulation increased the originality of this study. Both the personal characteristics (extrovert vs introvert) and the personal style of regulation (assertive vs tentative) were found to be transferable and not group dependent. The findings of this study show that peer interaction in collaborative tasks can facilitate achievement of collective conceptual understanding and learning gain through inter-individual regulation in social contexts. However, students find planning, evaluation and high-level regulation challenging, especially in social contexts. I suggest that concerted efforts should be made to teach students to make the most of group work by identifying and introducing instructional strategies that develop the desirable skills of egalitarian collaboration and the more difficult aspects of cognitive regulation and high-level engagement. Strategies such as metacognitive prompts, teaching students about team development techniques and exposing students to collaborative ill-structured tasks could be helpful in this regard. Some suggestions have also been made in terms of directions for future research.Thesis (PhD)--University of Pretoria, 2016.ChemistryPhDUnrestricte

Teachers' and learners' perceptions of stoichiometry using POGIL : a case study in South Africa

This study explored the perceptions of teachers and learners regarding the use of Process Oriented Guided Learning Inquiry Learning (POGIL) to teach stoichiometry. A qualitative case study was carried out at two conveniently and purposively sampled township schools in Pretoria, South Africa. For this purpose, two Grade 11 physical sciences classes were identified where 48 learners and their respective teachers, who had previously been trained to teach using POGIL, gave consent to participate in the study. Data were collected using lesson observations, focus group interviews for learners and interviews for teachers. All data were transcribed and analysed with the aid of ATLAS.ti software for qualitative data analysis. The findings from the observations indicate that the learners were excited, motivated, and actively engaged in their work. They assisted one another by attempting to answer questions supported with justification. The findings from the focus group interviews indicate that the learners were excited to learn using POGIL and wished to use the method in other subjects, including mathematics. The learners had noticed improvements in their grades and understanding of abstract topics. The findings from the teacher interviews indicated that they appreciated POGIL because they found it useful in reducing misconceptions, increasing learner participation, increasing understanding and achievement, and that their learners were more engaged. The results indicate that POGIL increased learners’ interest, participation and active learning, while seemingly also improving the learners’ understanding of and achievement in abstract topics like stoichiometry. POGIL may be useful in increasing active learning and participation of learners which may lead to increased understanding and achievement.This study was funded by the National Research Foundation, South Africa.The National Research Foundation, South Africa.https://www.ejmste.comam2022Science, Mathematics and Technology Educatio