Projecte: PID2020-UB/043Poster que presenta resultados del proyecto PID2020-UB/043, sobre cohorte completa de estudiantes del máster de formación del profesorado en la UB. En contraste con literatura previa, encontramos en nuestros resultados algunas diferencias entre sexos en relación con la confianza (o autocompetencia) manifestada por el alumnado participante.PID2020-UB/04

Alvarez-Brinquis, Mireya

Carbó, Maria

ElKhayat, Maryam

Fierro, José Daniel

Garcia-Mila, Mercè

Gri, Tània

Jarque, Maria Josep

Pérez-Clemente, Gemma

Pérez-Sedano, Esther

Remesal Ortiz, Ana

Vega, Fàtima

English

Garcia-Milà, Mercè

Gri, Tania

Diposit Digital de la Universitat de Barcelona

CBM (confidence-based marking) offers an alternative grading algorithm for learning tests: individuals’ personal self-efficacy (or self-confidence) is challenged at responding each multiple choice item. The learners’ grades are adjusted based on the correctness/error of their answer in connection with their declared self-confidence (high, middle, low). Very soon authors reckon a formative potential in it (Gardner-Medwin, 2007). The final grading range transcends the traditional 0-10 grading scheme, and so learners need to relearn to re-interpret their result, which has potentially a formative (+motivational +cognitive, hence metacognitive) effect.Remesal, A.* (aremesal@ub.edu); Álvarez-Brinquis, M.; Carbó, M.; ElKhayat, M.; Fierro, J.D., Garcia-Mila, M.; Gri, T.; Jarque, M.J.; Pérez-Clemente, G.; Pérez-Sedano, E.; Vega, F.Department of Cognition, Development and Educational PsychologyChallenging the traditional grading scheme for metacognitive engagement at teacher educationWhat isCBM ?OurTeachingprogramTest CBM1Prior Knowl.Topic112%45%13%30%Total sample, CBM results patternsDecrease Increase Stable ValleyRQ-1: Are there differences in CBM results regarding curricular area?PURPOSE >> carry out a formative use of CBM to consistently foster reflexive self-assessment and metacognition.CONTEXT >> Master for Secondary Teachers Education at the Universitat de Barcelona. 1-year program. Compulsory module of Human Development in Adolescence and Instructional Psychology (4 months in 20 sessions).INSTRUCTIONAL DESIGN >> students respond to 10 CBM-items starting each topic (of three). Each test tackles general and usual misconceptions revolving the program’s contents. After responding to the CBM-test, they receive direct result with an interpretation guide and a reflection questionnaire to foster metacognitive reflection.RQ-3: Are there differences in confidence regarding curricular area?PARTICIPANTS >> initially 458 students, full cohort in the master for Sec.Teacher. Final sample (after excluding criteria) n= 327 (71%). Excluding criteria for validation of the sample: (1) completion of all 3 tests, (2) no more than one missing answer per test.33%20%12%21%14%Curricular areasLanguage, n 107Soc.Science, n 66Nat.Science, n 38Arts, n 69Other areas, n 4740%60%SexMen, n 132 Women, n 195CBM algorithm if right… if wrong…High confidence +3 -6Middleconfidence +2 -2Low confidence +1 0Topic3Test CBM2Prior Knowl.Topic212 weeks, 20 sessionsTest CBM2Prior Knowl.OurstudyGUIDING INSTRUMENTS >>Students receive a general CBM-interpretation guide starting the module, plus automatic feedback after responding each test, with their CBM-result. The interpretation guide models likely results following CBM algorithm in terms of “levels of desirable performance”.Students voluntarily respond a reflection questionnaire after responding CBM tests.DESIRABLE LEVELS OF CBM PERFORMANCELevel 0 >> CBM result in negative rangeLevel 1 >> CBM result [1-10] (1*/#items/)Level 2 >> CBM result [11-20] (2*/#items/)Level 3 >> CBM result [21-30] (3*/#items/)QUESTIONS FOR GUIDED REFLECTION:• How do you feel about the result? (M.C.answer)• Does it meet your expectation? Is it below? Above? (M.C.answer)• Why do you think this result came about? (free answer)• What can you do to improve? (free answer)RQ-2: Are there differences in CBM-results regarding sex?RQ-4: Are there differences in confidence regarding sex?PURPOSE >> to evaluate the instructional experience with CBM.METHODOLOGY >> Mixed methodology, with quantitative and qualitative data.DATA >> Logfiles, CBM results, CBM single responses, reflective open answers (voluntary).ResultsNo difference per curricular area nor sexExample of one group of students’progression along the module22,96%31,12%3,57%46,43% 44,90%25,51%29,08%21,43%54,59%2,55% 2,55%16,33%0,00%10,00%20,00%30,00%40,00%50,00%60,00%T1 T2 T3Womenlevel 0 level 1 level 2 level 329,55% 28,03%5,30%40,91%53,79%41,67%26,52%16,67%43,18%3,79% 0,76%9,09%0%10%20%30%40%50%60%T1 T2 T3Menlevel 0 level 1 level 2 level 3χ2 = 120.6025, df(6), p** < 0.00001χ2 = 53.2339, df(6), p** < 0.00001Inter-sexT1: χ2 = 17.5478, df(3), p** = .000545T2: χ2 = 3.8432, df(3), p = .2789T3: χ2 = 12.1318, df(3), p** = .006931,91% 29,79%2,13%53,19%27,66%34,04%14,89%55,32%2,13% 2,13%14,89%0%10%20%30%40%50%60%T1 T2 T3Other areaslevel 0 level 1 level 2 level 320,56%32,71%3,74%51,40% 48,60%30,84%27,10%15,89%52,34%2,80% 2,80%13,08%0%10%20%30%40%50%60%T1 T2 T3Languageslevel 0 level 1 level 2 level 321,05%26,32%5,26%52,63%47,37%23,68%23,68% 23,68%60,53%2,63% 2,63%10,53%0%10%20%30%40%50%60%70%T1 T2 T3Nat.Sciencelevel 0 level 1 level 2 level 321,21% 24,24%6,06%42,42%50,00%40,91%33,33%25,76%39,39%6,06%0,00%13,64%0%10%20%30%40%50%60%T1 T2 T3Soc.Sciencelevel 0 level 1 level 2 level 336,23% 33,33%4,35%39,13%44,93%33,33%23,19% 20,29%47,83%1,45% 1,45%14,49%0%10%20%30%40%50%60%T1 T2 T3Artslevel 0 level 1 level 2 level 3Total sampleχ2 = 166.7, df(6), p** < .0000126%30%4%43%49%32%28%20%50%3% 2%13%0%10%20%30%40%50%60%T1 T2 T3CBM results, Total samplelevel 0 level 1 level 2 level 3 χ2 = 20.9247, df(4), p** = 0.000029χ2 = 28.1766, df(4), p** < .00001χ2 = 38.1169, df(4), p** < .00001χ2 = 54.6564, df(4), p** < .00001χ2 = 14.3131, df(4), p** = 0.00636RQ1 + RQ20,00%5,00%10,00%15,00%20,00%Languages Soc.Science Nat.Science Arts Other areasPatterns curricular areaDecrease Increase Stable ValleyRQ3 + RQ49127841155788867969357 3294080200400600800100012001400T1 T2 T3Total sample, progression of confidence & H-E HH HM HL EL EM EH551460708453528604227 1912430100200300400500600700800T1 T2 T3WomenHH HM HL EL EM EH361 324447335 339 365130 138 1650100200300400500600700800T1 T2 T3MenHH HM HL EL EM EH303262367247278326122 1031400100200300400T1 T2 T3LanguagesHH HM HL EL EM EH102 101138105 105 12333 27 340100200300400T1 T2 T3Nat.ScienceHH HM HL EL EM EH202 172247156 171 18585 62 810100200300400T1 T2 T3Soc.ScienceHH HM HL EL EM EH156 135223161203 21180 80 950100200300400T1 T2 T3ArtsHH HM HL EL EM EH149114180119 110 12437 57 580100200300400T1 T2 T3Other areasHH HM HL EL EM EHHits/confidence χ2 = 21.2875, df(4), p** = 0.000278Total sampleHits/Confidence χ2 = 21.4133, df(4), p** = .000262Conclusions:1) No, we found no differences in CBM results in terms of curricular aera. Students’ disciplinary background does not affect.2) No, we found no differences in self-confidence in terms of curricular area. Students’ disciplinary background does not affect.3) Yes, we found differences in CBM results in terms of sex. Women get significantly better results at third test, after two previous trials and after significantly worse results in first trial.4) Yes, we found differences in self-confidence in terms of sex. Women fluctuate  stronger and gain significantly better self-confidence by the third attempt compared to men.These results are all positive as much as they offer new directions of research and instructional design for improving formative practices not only in teacher education but also in other areas. In fact, prior studies do not offer evidence on gender-differences (Gardner-Medwin, 2006), however our results point to these kind of differences and hence to the interest of considering particular instructional strategies to:1) Support women in their loss of self-competence and confidence2) Support men in a more self-critical reflection processQualitative results of students’ guided reflection are still to be analysed. In coherence with these first analysis of quantitative data, we want to inquire further on gender-differences and on other contextual conditions such as previous education level to access the Teacher Education masters program (Bachelor, Postgraduate, Doctorate)Some referencesBarr, D. A., & Burke, J. R. (2013). Using confidence-based marking in a laboratory setting: A tool for student self-assessment and learning. Journal of Chiropractic Education, 27(1), 21-26.Gardner-Medwin, A. (2006).Confidence-based Marking: towards deeper learning and better exams. In Bryan, C. & Clegg, K. (eds) Innovative assessment in Higher Education. pp.141-149. Routledge.Remesal, A., Vega, F., Alvarez-Brinquis, M., & Pérez-Clemente, G. (2019). Confidence based marking for SRL in Secondary Teacher Education: students' voice. EARLI 2019. Aachen, Germany, 12th-16th, August, 2019.LEVEL 0LEVEL 1LEVEL 2LEVEL 3

Challenging the traditional grading scheme for metacognitive engagement at teacher education.

https://diposit.ub.edu/dspace/bitstream/2445/201272/1/2022_SIG1-4%20Remesal%20et%20al%20CBM%20POSTER.pdf

Challenging the traditional grading scheme for metacognitive engagement at teacher education.

Abstract

Similar works

Full text

Available Versions

Diposit Digital de la Universitat de Barcelona