Team-Based Learning Analytics: An Empirical Case Study

Many medical schools that have implemented team-based learning (TBL) have also incorporated an electronic learning architecture, commonly referred to as a learning management system (LMS), to support the instructional process. However, one LMS feature that is often overlooked is the LMS's ability to record data that can be used for further analysis. In this article, the authors present a case study illustrating how one medical school used data that are routinely collected via the school's LMS to make informed decisions. The case study started with one instructor's observation that some teams in one of the undergraduate medical education learning modules appeared to be struggling during one of the team activities; that is, some teams seemed unable to explain or justify their responses to items on the team readiness assurance test (tRAT). Following this observation, the authors conducted 4 analyses. Their analyses demonstrate how LMS-generated and recorded data can be used in a systematic manner to investigate issues in the real educational environment. The first analysis identified a team that performed significantly poorer on the tRAT. A subsequent analysis investigated whether the weaker team's poorer performance was consistent over a whole module. Findings revealed that the weaker team performed poorer on the majority of the TBL sessions. Further investigation using LMS data showed that the weaker performance was due to the lack of preparation of one individual team member (rather than a collective poor tRAT performance). Using the findings obtained from this case study, the authors hope to convey how LMS data are powerful and may form the basis of evidence-based educational decision making

Effects of graded versus ungraded individual readiness assurance scores in team-based learning : a quasi-experimental study

Pre-class preparation is a crucial component of team-based learning (TBL). Lack of preparation hinders both individual learning and team performance during TBL. The purpose of the present study was to explore how the grading of the individual readiness assurance test (iRAT) can affect pre-class preparation, iRAT performance and performance in the end-of-year examination. Using a quasi-experimental design, Year 1 and 2 students' download frequency for their pre-class materials, performance on iRAT and examination were examined under two conditions; (1) under which the iRAT was graded and (2) under which the iRAT was ungraded. Medical students (N = 220) from three cohorts were included in the study. Differences between both conditions were tested by means of six separate ANCOVAs, using medical school entry test scores as the covariate to account for potential cohort effects. Results revealed that students were downloading more pre-class materials prior to their TBL sessions, and were performed significantly better on iRAT when their performance was graded, even after controlling for cohort effects. Analysis of covariance demonstrated that performance on iRAT also appeared to affect performance on their examination scores. The results of the study suggest that grading has a positive effect on students' iRAT scores. Implications for TBL are discussed

Effects of graded versus ungraded individual readiness assurance scores in team-based learning: A quasi-experimental study

Pre-class preparation is a crucial component of team-based learning (TBL). Lack of preparation hinders both individual learning and team performance during TBL. The purpose of the present study was to explore how the grading of the individual readiness assurance test (iRAT) can affect pre-class preparation, iRAT performance and performance in the end-of-year examination. Using a quasi-experimental design, Year 1 and 2 students’ download frequency for their pre-class materials, performance on iRAT and examination were examined under two conditions; (1) under which the iRAT was graded and (2) under which the iRAT was ungraded. Medical students (N = 220) from three cohorts were included in the study. Differences between both conditions were tested by means of six separate ANCOVAs, using medical school entry test scores as the covariate to account for potential cohort effects. Results revealed that students were downloading more pre-class materials prior to their TBL sessions, and were performed significantly better on iRAT when their performance was graded, even after controlling for cohort effects. Analysis of covariance demonstrated that performance on iRAT also appeared to affect performance on their examination scores. The results of the study suggest that grading has a positive effect on students’ iRAT scores. Implications for TBL are discussed

Assessing How Students Learn in Team-Based Learning: Validation of the Knowledge Re-Consolidation Inventory

Purpose: The purpose of the present study was to establish the construct validity of a new instrument to measure psychological learning processes associated with Team-based learning (TBL), the Knowledge Re-Consolidation Inventory (KRCI). The instrument was designed to measure six factors: (1) self-guided preparation, (2) knowledge consolidation, (3) retrieval practice, (4) peer elaboration, (5) feedback, and (6) transfer of knowledge. Method: Two samples were taken, consisting in total of 197 first- and second-year medical students from Singapore. To establish the construct validity of the KRCI, two confirmatory factor analyses were conducted (CFA). First, an exploration sample (N=90) was taken from the second-year medical students to conduct a preliminary CFA, and it resulted in elimination of items with poor psychometric properties. A confirmatory sample (N=107) was then taken from the first-year medical students to conduct a second CFA to cross-validate the KRCI with reduced items. Results: From the original 38 items, 16 remained. The resulting model fitted the data well. The second CFA with the cross-validation sample replicated the findings of the first analysis and supported the factorial structure of the hypothesised six-factor model. Tests of factorial invariance demonstrate that the factorial structure of the KRCI was stable across measurements. Discussion: The results of the study suggest that the KRCI is a valid and reliable instrument capable of measuring the six psychological mechanisms underlying TBL.Published versio