First results of computerized adaptive testing for an online physics test

Tests are an essential tool to assess students’ ability. In online education these tests are mostly of static nature with the same items for each student. In contrast computerized adaptive testing concepts take into account the information about the test user automatically collected in an online test. The aim is a comparably precise test result with fewer test items (questions). An implementation of such a computerized adaptive test (CAT) is presented here. The adaptation process is based on the precise knowledge of the item parameters, e.g. difficulty, in the item pool. An estimation of the knowledge level of the test user has to be performed in real time after each answer. With this information the next item can be selected accordingly. This leads to a highly individualized test for each test user. For all items the parameters were determined with methods of the item response theory (IRT) in the framework of the probabilistic test theory. For that real test results of former first year students in engineering science had been analyzed. The prototype of such a CAT has been developed. It focusses on a physics test for prospective students in the STEM fields. In fall 2021 the pilot phase was conducted with first year students in engineering science. The CAT shows that the same precision can be achieved with a mean of 9.3 items compared to 12 in the static test. The acceptance among the students is high. The correlation between the static test and the CAT is satisfactory

A Comparison of Aggregation Rules for Selecting Anchor Items in Multigroup DIF Analysis

This study addresses the topic of how anchoring methods for differential item functioning (DIF) analysis can be used in multigroup scenarios. The direct approach would be to combine anchoring methods developed for two‐group scenarios with multigroup DIF‐detection methods. Alternatively, multiple tests could be carried out. The results of these tests need to be aggregated to determine the anchor for the final DIF analysis. In this study, the direct approach and three aggregation rules are investigated. All approaches are combined with a variety of anchoring methods, such as the “all‐other purified” and “mean p‐value threshold” methods, in two simulation studies based on the Rasch model. Our results indicate that the direct approach generally does not lead to more accurate or even to inferior results than the aggregation rules. The min rule overall shows the best trade‐off between low false alarm rate and medium to high hit rate. However, it might be too sensitive when the number of groups is large. In this case, the all rule may be a good compromise. We also take a closer look at the anchor selection method “next candidate,” which performed rather poorly, and suggest possible improvements

German Physicians and Medical Students Do Not Represent the Population They Serve

Medical professionals who represent the communities they serve are in a better position to understand patients‘ social circumstances and communicate in a more patient-centered way. International studies show limited diversity and underrepresentation of certain social groups in the population of physicians and medical students. We designed an observational study to investigate the cultural and socio-economic diversity of physicians and medical applicants in comparison to the general population in Germany. We invited 15,195 physicians in Hamburg and 11,287 medical applicants in Germany to participate in an online survey between June and August 2022. The lower three quintiles of objective socio-economic background (SEB) were vastly underrepresented in all subsamples of the study and in particular amongst applicants and students admitted in Hamburg: 57.9% of physicians and 73.8% of medical students in Hamburg originate from the top quintile of SEB. The Turkish and Polish communities were particularly underrepresented in the group of physicians from Hamburg and medical applicants and students in Germany (p = 0.02; p < 0.001). In line with existing evidence, the vast majority of physicians and medical students come from the most affluent households when entering medical school. Widening participation strategies are needed to facilitate fairer access to the study of medicine in Germany

Can Different Admissions to Medical School Predict Performance of Non-Technical Skill Performance in Simulated Clinical Settings?

Non-technical skills (NTS) in medical care are essential to ensure patient safety. Focussing on applicants’ NTS during medical school admission could be a promising approach to ensure that future physicians master NTS at a high level. Next to pre-university educational attainment, many selection tests have been developed worldwide to facilitate and standardise the selection process of medical students. The predictive validity of these tests regarding NTS performance in clinical settings has not been investigated (yet). Therefore, we explored the predictive validities and prognosis of the Hamburg MMI (HAM-Int), HAM-Nat, PEA, and waiting as well as other quota (as example) designated by the Federal Armed Forces) for NTS performance in clinical emergency medicine training of medical students. During 2017 and 2020, N = 729 second, third, and fourth year students were enrolled within the study. The mean age of participants was 26.68 years (SD 3.96) and 49% were female students. NTS of these students were assessed during simulation scenarios of emergency training with a validated rating tool. Students admitted via waiting quota and designated by the Armed Forces performed significantly better than students admitted by excellent PEA (p = 0.026). Non-EU students performed significantly inferior (p = 0.003). Our findings provide further insight to explain how and if admission to medical school could predict NTS performance of further physicians