Identifying low test-taking effort during low-stakes tests with the new Test-taking Effort Short Scale (TESS) – development and psychometrics

Background: Low-stakes tests are becoming increasingly important in international assessments of educational progress, and the validity of these results is essential especially as these results are often used for benchmarking. Test scores in these tests not only mirror students’ ability but also depend on their test-taking effort. One way to obtain more valid scores from participating samples is to identify test-takers with low test-taking effort and to exclude them from further analyses. Self-assessment is a convenient and quick way of measuring test-taking effort. We present the newly developed Test-taking Effort Short Scale (TESS), which comprises three items measuring attainment value/intrinsic value, utility value, and perceived benefits, respectively. Methods: In a multicenter validation study with N = 1837 medical students sitting a low-stakes progress test we analyzed item and test statistics including construct and external validity. Results: TESS showed very good psychometric properties. We propose an approach using stanine norms to determine a cutoff value for identifying participants with low test-taking effort. Conclusion: With just three items, TESS is shorter than most established self-assessment scales; it is thus suited for administration after low-stakes progress testing. However, further studies are necessary to establish its suitability for routine usage in assessment outside progress testing

Peer Teaching in Paediatrics - Medical Students as Learners and Teachers on a Paediatric Course

Background: Peer assisted learning is known as an effective educational strategy in medical teaching. We established a peer assisted teaching program by student tutors with a focus on clinical competencies for students during their practical training on paediatric wards. It was the purpose of this study to investigate the effects of a clinical skills training by tutors, residents and consultants on students evaluations of the teaching quality and the effects of a peer teaching program on self assessed clinical competencies by the students

Peer Teaching in Paediatrics - Medical Students as Learners and Teachers on a Paediatric Course

Background: Peer assisted learning is known as an effective educational strategy in medical teaching. We established a peer assisted teaching program by student tutors with a focus on clinical competencies for students during their practical training on paediatric wards. It was the purpose of this study to investigate the effects of a clinical skills training by tutors, residents and consultants on students evaluations of the teaching quality and the effects of a peer teaching program on self assessed clinical competencies by the students

The formation of physician altruism

We study how patient-regarding altruism is formed by medical education. We elicit and structurally estimate altruistic preferences using experimental data from a large sample of medical students ( = 733) in Germany at different progress stages in their studies. The estimates reveal substantial heterogeneity in altruistic preferences of medical students. Patient-regarding altruism is highest for freshmen, significantly declines for students in the course of medical studies, and tends to increase again for last year students, who assist in clinical practice. Also, patient-regarding altruism is higher for females and positively associated to general altruism. Altruistic medical students have gained prior practical experience in healthcare, have lower income expectations, and are more likely to choose surgery and pediatrics as their preferred specialty

A study on effects of and stance over tuition fees

Aim: Regarding tuition fees (that in Germany already have been abrogated) putative drawbacks like prolonged study duration have been suspected while benefits are not clearly proven. We investigated whether tuition fees (500 Euro per semester) affected the course of studies of Cologne medical students and asked for students’ stance over tuition fees.Methods: Of 1,324 students we analyzed the rate of those passing their first medical exam (“Physikum”) within minimum time and students’ discontinuation rate, respectively. Regression analysis tested for putative influences of tuition fees and demographic factors. In an additional online survey 400 students answered questions regarding the load by and their stance over tuition fees.Results: We find that fees did not affect rate of Cologne students passing their first medical exam within minimum time or students’ discontinuation rate. According to the online survey, at times of tuition fees significantly more students did not attend courses as scheduled. Time spent on earning money was significantly increased. 51% of students who had to pay tuition fees and 71% of those who never had to stated tuition fees to be not justified. More than two thirds of students did not recognize any lasting benefit from tuition fees.Conclusion: Tuition fees did not affect discontinuation rate or study duration of Cologne medical students. However, they obviously influenced the study course due to an increased need to pursue a sideline. Cologne medical students rather refused tuition fees and did not recognize their advantages in terms of enhanced quality of studies

A validity argument for progress testing: Examining the relation between growth trajectories obtained by progress tests and national licensing examinations using a latent growth curve approach

Background: Progress testing is a longitudinal assessment that aims at tracking students' development of knowledge. This approach is used in many medical schools internationally. Although progress tests are longitudinal in nature, and their focus and use of developmental aspects is a key advantage, individual students' learning trajectories themselves play, to date, only a minor role in the use of the information obtained through progress testing. Methods: We investigate in how far between-person differences in initial levels of performance and within-person rate of growth can be regarded as distinct components of students' development and analyze the extent to which these two components are related to performances on national licensing examinations using a latent growth curve model. Results: Both, higher initial levels of performances and steepness of growth are positively related to long-term outcomes as measured by performance on national licensing examinations. We interpret these findings as evidence for progress tests' suitability to monitor students' growth of knowledge across the course of medical training. Conclusions: This study indicates that individual development as obtained by formative progress tests is related to performance in high-stakes assessments. Future studies may put more focus on the use of between-persons differences in growth of knowledge

Progress testing anytime and anywhere - Does a mobile-learning approach enhance the utility of a large-scale formative assessment tool?

Background The widespread use of mobile devices among students favors the use of mobile learning scenarios at universities. In this study, we explore whether a time- and location-independent variant of a formative progress test has an impact on the students' acceptance, its validity and reliability and if there is a difference in response processes between the two exam conditions. Methods Students were randomly assigned to two groups of which one took the test free of local or temporal fixations, while the other group took the test at the local testing center under usual examination conditions. Beside the generated test data, such as test score, time-on-test, and semester status, students also evaluated the settings. Results While there was no significant effect on the test score between the two groups, students in the mobile group spent more time on the test and were more likely to use the help of books or online resources. The results of the evaluation show that the acceptability among students is increased by a mobile version of the formative progress test. Conclusions The results suggest that the acceptance and motivation to participate in formative tests is enhanced by lifting local and temporal restrictions. The mobile version nonetheless does not have an impact on the students' performance

Computer Versus Paper-Does It Make Any Difference in Test Performance?

Construct: In this study, we examine the differences in test performance between the paper-based and the computer-based version of the Berlin formative Progress Test. In this context it is the first study that allows controlling for students' prior performance. Background: Computer-based tests make possible a more efficient examination procedure for test administration and review. Although university staff will benefit largely from computer-based tests, the question arises if computer-based tests influence students' test performance. Approach: A total of 266 German students from the 9th and 10th semester of medicine (comparable with the 4th-year North American medical school schedule) participated in the study (paper = 132, computer = 134). The allocation of the test format was conducted as a randomized matched-pair design in which students were first sorted according to their prior test results. The organizational procedure, the examination conditions, the room, and seating arrangements, as well as the order of questions and answers, were identical in both groups. Results: The sociodemographic variables and pretest scores of both groups were comparable. The test results from the paper and computer versions did not differ. The groups remained within the allotted time, but students using the computer version (particularly the high performers) needed significantly less time to complete the test. In addition, we found significant differences in guessing behavior. Low performers using the computer version guess significantly more than low-performing students in the paper-pencil version. Conclusions: Participants in computer-based tests are not at a disadvantage in terms of their test results. The computer-based test required less processing time. The reason for the longer processing time when using the paper-pencil version might be due to the time needed to write the answer down, controlling for transferring the answer correctly. It is still not known why students using the computer version (particularly low-performing students) guess at a higher rate. Further studies are necessary to understand this finding

Comparison of the evaluation of formative assessment at two medical faculties with different conditions of undergraduate training, assessment and feedback

Introduction: Both formative and summative assessments have their place in medical curricula: formative assessment to accompany the learning process and summative assessment to ensure that minimum standards are achieved. Depending on the conditions of undergraduate training, assessment and feedback, students place more or less importance on formative assessment, and thus the fulfilment of its function may be questionable. This study describes how the low-stakes formative Berlin Progress Test (BPT) is embedded at two medical faculties with partially different framework conditions and what effects these have on the students' testing efforts and the evaluation of the test, especially the perception of its benefits and (intangible) costs, such as non-participation in contemporaneous activities and emotional impairments. Methods: In this study, the proportion of non-serious BPT participants at two medical faculties (total sample: N-F1=1,410, N-F2=1,176) in winter term 2015/16 was determined both by the number of unanswered questions on the test itself and in a survey using a standardized instrument (N-F1,=415, N-F2=234). Furthermore, open questions were asked in this survey about perceived benefits and perceived costs, which were analyzed with qualitative and quantitative methods. Results: The BPT is generally better accepted at Faculty 2. This can be seen in the higher proportion of serious test takers, the lower perceived costs and the higher reported benefit, as well as the higher proportion of constructive comments. Faculty 2 students better understood the principle of formative testing and used the results of the BPT as feedback on their own knowledge progress, motivation to learn and reduction of exam fear. Discussion: When medical faculties integrate formative assessments into the curriculum, they have to provide a framework in which these assessments are perceived as an important part of the curriculum. Otherwise, it is questionable whether they can fulfil their function of accompanying the learning process