Cumulative assessment:Strategic choices to influence students' study effort

BACKGROUND: It has been asserted that assessment can and should be used to drive students’ learning. In the current study, we present a cumulative assessment program in which test planning, repeated testing and compensation are combined in order to influence study effort. The program is aimed at helping initially low-scoring students improve their performance during a module, without impairing initially high-scoring students’ performance. We used performance as a proxy for study effort and investigated whether the program worked as intended. METHODS: We analysed students’ test scores in two second-year (n = 494 and n = 436) and two third-year modules (n = 383 and n = 345) in which cumulative assessment was applied. We used t-tests to compare the change in test scores of initially low-scoring students with that of initially high-scoring students between the first and second subtest and again between the combined first and second subtest and the third subtest. During the interpretation of the outcomes we took regression to the mean and test difficulty into account. RESULTS: Between the first and the second subtest in all four modules, the scores of initially low-scoring students increased more than the scores of initially high-scoring students decreased. Between subtests two and three, we found a similar effect in one module, no significant effect in two modules and the opposite effect in another module. CONCLUSION: The results between the first two subtests suggest that cumulative assessment may positively influence students’ study effort. The inconsistent outcomes between subtests two and three may be caused by differences in perceived imminence, impact and workload between the third subtest and the first two. Cumulative assessment may serve as an example of how several evidence-based assessment principles can be integrated into a program for the benefit of student learning

The development and validation of a short form of the STERLinG:A practical, valid and reliable tool to evaluate teacher competencies to encourage reflective learning

Background: To optimize response rates, it is important to have brief, comprehensive instruments.Aims: We have developed and validated a short form of an instrument for measuring students' perceptions of teachers' competencies to encourage students' reflective learning in small groups (the STERLinG).Methods: Based on statistical and content criteria, the original 36-item STERLinG was reduced to 15 items: three scales with five items each. This mini-STERLinG was validated. Confirmatory factor analysis was performed and internal consistencies were calculated.Results: The instrument was completed by 501 respondents (63%). The original instrument structure was confirmed with 62.6% explained variance. Reliabilities were high with 0.91 for the entire mini-STERLinG and 0.87, 0.85 and 0.81 for its subscales.Conclusions: The mini-STERLinG was found to be a feasible, valid and reliable instrument.</p

Time spent on clerkship activities by students in relation to their perceptions of learning environment quality

Context Students' perceptions of their learning environment are of great importance to their learning process. In this study we assessed the time allocated by students to clerkship activities and the relationship between students' allocations of time and their perceptions of the quality of their clinical learning environment. Methods Participants were 133 undergraduate students from eight hospitals taking part in four clerkship rotations. All students recorded the time they spent on eight clerkship activities over 2 weeks and completed the Postgraduate Hospital Educational Environment Measure (PHEEM). Partial correlation analyses were undertaken to examine the relationship between the amount of time students spent on each activity and their PHEEM scores. Results Students spent nearly 8 hours a day on clerkship activities. Most time was spent on observing doctors (40%), followed by participating in consultations without direct supervision (12%). The time students spent on observing doctors (r = 0.206, P <0.05) and in consultations without direct supervision (r = 0.211, P <0.05) was significantly related to the students' PHEEM scores. There was a significant relationship at the P <0.10 level between the time spent on directly supervised activities and students' PHEEM scores (r = 0.165, P <0.10). Conclusions The results suggest that the time spent on activities involving direct patient contact is positively related to students' perceptions of the quality of their learning environment. None of the activities were significantly negatively related to the students' perceptions of their clinical learning environment. Future research should examine the optimal time allocations required to enhance the perceived quality of the clinical learning environment

Influence of open- and closed-book tests on medical students' learning approaches

CONTEXT Two learning approaches are consistently distinguished in the literature: deep and surface learning. The deep learning approach is considered preferable. Open-book tests are expected to stimulate deep learning and to offer a possible way of handling the substantial growth in medical knowledge. In this study we test the hypothesis that open-book tests stimulate deep learning more than closed-book tests. METHODS Medical students in Years 2 (n = 423) and 3 (n = 306) participated in this study. They evaluated their preparation for open- and closed-book tests using the test for Deep Information Processing (DIP). This questionnaire consists of 24 items divided into three subscales: Critical Reading; Broaden One's Context, and Structuring. A paired t-test was used to analyse the data. RESULTS Both cohorts scored significantly higher when preparing for closed-book tests for the overall DIP score and on the Broaden One's Context and Structuring scales. Year 3 students also scored significantly higher on the Critical Reading scale when preparing for closed-book tests. Gender differences were found: women used deeper learning approaches than men. CONCLUSIONS Our hypothesis was not supported. In fact, the opposite was found: closed-book tests stimulated a deep learning approach more than open-book tests. Three possible explanations are: deep learning is particularly necessary for remembering and recalling knowledge; students feel more confident when preparing for closed-book tests, and students are more motivated to study for closed-book tests. The debate on the concept of deep learning in higher education should probably be renewed

The effect of curriculum sample selection for medical school

In the Netherlands, students are admitted to medical school through (1) selection, (2) direct access by high pre-university Grade Point Average (pu-GPA), (3) lottery after being rejected in the selection procedure, or (4) lottery. At Radboud University Medical Center, 2010 was the first year we selected applicants. We designed a procedure based on tasks mimicking the reality of early medical school. Applicants took an online course followed by an on-site exam, resembling courses and exams in early medical school. Based on the exam scores, applicants were selected or rejected. The aim of our study is to determine whether curriculum sample selection explains performance in medical school and is preferable compared to selection based on performance in secondary school. We gathered data on the performance of students of three consecutive cohorts (2010-2012, N = 954). We compared medical school performance (course credits and grade points) of selected students to the three groups admitted in other ways, especially lottery admissions. In regression analyses, we controlled for out of context cognitive performance by adjusting for pu-GPA. Selection-admitted students outperformed lottery-admitted students on most outcome measures, unadjusted as well as adjusted for pu-GPA (p aecurrency 0.05). They had higher grade points than non-selected lottery students, both unadjusted and adjusted for pu-GPA (p aecurrency 0.025). Adjusted for pu-GPA, selection-admitted students and high-pu-GPA students performed equally. We recommend this selection procedure as it adds to secondary school cognitive performance for the general population of students, is efficient for large numbers of applicants and not labour-intensive

Influences of deep learning, need for cognition and preparation time on open- and closed-book test performance

Objectives The ability to master discipline-specific knowledge is one of the competencies medical students must acquire. In this context, 'mastering' means being able to recall and apply knowledge. A way to assess this competency is to use both open- and closed-book tests. Student performance on both tests can be influenced by the way the student processes information. Deep information processing is expected to influence performance positively. The personal preferences of students in relation to how they process information in general (i.e. their level of need for cognition) may also be of importance. In this study, we examined the inter-relatedness of deep learning, need for cognition and preparation time, and scores on open- and closed-book tests. Methods This study was conducted at the University Medical Centre Groningen. Participants were Year 2 students (n = 423). They were asked to complete a questionnaire on deep information processing, a scale for need for cognition on a questionnaire on intellectualism and, additionally, to write down the time they spent on test preparation. We related these measures to the students' scores on two tests, both consisting of open- and closed-book components and used structural equation modelling to analyse the data. Results Both questionnaires were completed by 239 students (57%). The results showed that need for cognition positively influenced both open- and closed-book test scores (beta-coefficients 0.05 and 0.11, respectively). Furthermore, study outcomes measured by open-book tests predicted closed-book test results better than the other way around (beta-coefficients 0.72 and 0.11, respectively). Conclusions Students with a high need for cognition performed better on open- as well as closed-book tests. Deep learning did not influence their performance. Adding open-book tests to the regularly used closed-book tests seems to improve the recall of knowledge that has to be known by heart. Need for cognition may provide a valuable addition to existing theories on learning

How students and specialists appreciate the mini-clinical evaluation exercise (mini-CEX) in Indonesian clerkships

Background Cultural differences might challenge the acceptance of the implementation of assessment formats that are developed in other countries. Acceptance of assessment formats is essential for its effectiveness; therefore, we explored the views of students and specialists on the practicality and impact on learning of these formats. This study was conducted to explore Indonesian students' and specialists' appreciation of the implementation of the Mini-Clinical Evaluation Exercise (Mini-CEX) in Indonesian clerkships. Methods This study was conducted at the Universitas Gadjah Mada, Indonesia. Participants were 52 students and 21 specialists in neurology and 78 students and 50 specialists in internal medicine. They were asked to complete a 19-item questionnaire that covered the characteristics of the mini-CEX such as its practicality, and the impact on learning and professional development. We used a Mann-Whitney U test to analyse the data. Results In total, 124 students (46 from neurology and 78 from internal medicine) and 38 specialists (13 from neurology and 25 from internal medicine) participated in this study. Students and specialists were positive about the practicality of the mini-CEX and the impact of this assessment format on learning and on professional development. The Mann-Whitney U test showed that there were no significant differences between students' and specialists' opinions on the mini-CEX, except for 2 items: specialists' appreciation of direct observation (mean rank = 93.16) was statistically significantly higher than students' appreciation of it (mean rank = 77.93; z = 2.065; p <0.05), but students' appreciation of the item that students' past mini-CEX results affected their recent mini-CEX outcomes (mean rank = 85.29) was significantly higher than specialists' appreciation of it (mean rank = 69.12; z = 2140; p <0.05). Conclusion Students and specialists were positive about the mini-CEX in Indonesian clerkships, although it was developed and validated in another culture. We found only small differences between their appreciations, which could be explained by the patterns of specialist-student interaction in Indonesian culture as large power distance and low individualism country