International medical electives in selected African countries: A phenomenological study on host experience

Objectives: To explore the host experience on international medical electives at a selection of hospitals in low- and middle-income countries in Africa. Outcomes of the study may inform and improve the preparation of global health curriculum, pre-elective training and debriefing for international medical electives. Methods: A descriptive phenomenological study was undertaken, involving semi-structured interviews with ten elective hosts at seven study sites in three African countries. Purposive convenience sampling augmented by snowballing was utilised to recruit study participants. The data were thematically analysed and interpreted with reflexivity to generate an accurate aggregate of the experience of participants in hosting international medical electives. Results: Six main themes emerged from the thematic analysis of interview data: international medical student contribution to host hospitals, host professional and personal fulfilment, barriers to student learning experience, international medical student preparedness, hope for reciprocity and barriers to cultural immersion and patient care. Conclusions: Study participants described the experience of hosting international medical elective students as overwhelmingly positive. However, they highlighted issues such as barriers to students’ learning experience and the lack of reciprocity between host and sending institutions as areas which could be addressed to optimize the experience for both hosts and international medical students. An understanding of the host experience provides stakeholders with a clearer idea of what is important in preparation, organisation and evaluation of the elective experience. This study provides the impetus for further research to examine the effectiveness of introducing appropriate pre-departure training and post-elective debriefing to students embarking on international medical electives

Improving the validity of Script Concordance Testing by optimising and balancing items

Background: A script concordance test (SCT) is a modality for assessing clinical reasoning. Concerns had been raised about the plausible validity threat to SCT scores if students deliberately avoided the extreme answer options to obtain higher scores. The aims of the study were firstly to investigate whether students’ avoidance of the extreme answer options could result in higher scores, and secondly to determine whether a ‘balanced approach’ by careful construction of SCT items (to include extreme as well as median options as model responses) would improve the validity of an SCT. Methods: Using the paired sample t-test, the actual average student scores for 10 SCT papers from 2012–2016 were compared with simulated scores. The latter were generated by recoding all ‘2’ responses to ‘1’ and ‘+2’ responses to ‘+1’ for the whole and bottom 10% of the cohort (simulation 1), and scoring as if all students had chosen ‘0’ for their responses (simulation 2). The actual average and simulated average scores in 2012 (before the ‘balanced approach’) were compared with those from 2013–2016, when papers had a good balance of modal responses from the expert reference panel. Results: In 2012, a score increase was seen in simulation 1 in the third-year cohort, from 50.2 to 55.6% (t [10] = 4.818; p = 0.001). Since 2013, with the ‘balanced approach’, the actual SCT scores (57.4%) were significantly higher than scores in both simulation 1 and simulation 2 (46.7% and 23.9% respectively). Conclusions: When constructing SCT examinations, apart from the rigorous pre-examination optimisation, it is desirable to achieve a balance between items that attract extreme responses and those that attract median response options. This could mitigate the validity threat to SCT scores, especially for the low-performing students who have previously been shown to only select median responses and avoid the extreme responses

Commentary: Expert responses in script concordance tests: A response process validity investigation

No abstract available

Rasch Analysis on OSCE Data: An Illustrative Example

BackgroundThe Objective Structured Clinical Examination (OSCE) is a widely used tool for the assessment of clinical competence in health professional education. The goal of the OSCE is to make reproducible decisions on pass/fail status as well as students’ levels of clinical competence according to their demonstrated abilities based on the scores.  This paper explores the use of the polytomous Rasch model in evaluating the psychometric properties of OSCE scores through a case study.  Method  The authors analysed an OSCE data set (comprised of 11 stations) for 80 fourth year medical students based on the polytomous Rasch model in an effort to answer two research questions:  (1) Do the clinical tasks assessed in the 11 OSCE stations map on to a common underlying construct, namely clinical competence? (2) What other insights can Rasch analysis offer in terms of scaling, item analysis and instrument validation over and above the conventional analysis based on classical test theory?ResultsThe OSCE data set has demonstrated a sufficient degree of fit to the Rasch model (χ2 = 17.060, DF=22, p=0.76) indicating that the 11 OSCE station scores have sufficient psychometric properties to form a measure for a common underlying construct, i.e. clinical competence.  Individual OSCE station scores with good fit to the Rasch model (p > 0.1 for all χ2 statistics) further corroborated the characteristic of unidimensionality of the OSCE scale for clinical competence.  A Person Separation Index (PSI) of 0.704 indicates sufficient level of reliability for the OSCE scores.  Other useful findings from the Rasch analysis that provide insights, over and above the analysis based on classical test theory, are also exemplified using the data set.  ConclusionThe polytomous Rasch model provides a useful and supplementary approach to the calibration and analysis of OSCE examination data.

Using a clinical deterioration scenario (video) as an approach for interprofessional learning

For effective management of clinical deterioration, health professionals must work together to recognise and respond to the patients changing health status. This paper provides an overview of the clinical deterioration video, used as a learning tool to engage medical, nursing and physiotherapy students and junior clinicians for the purpose of identifying the deteriorating patient and to support the development of interprofessional teamwork skills. This paper describes the perspectives of educators who have used this resource, a description of the resource, its evaluation, and access to the resource. The educators felt that the clinical deterioration video provides a learning environment in which learners develop an insight into the essential knowledge and skills required to support interprofessional practice and good patient outcomes

Migratory behaviour and survival rates of wild northern Atlantic salmon (Salmo salar) post-smolts: effects of environmental factors

This is the accepted version (authors' final draft post review) of the paper, reprinted with permission. Published version available at http://dx.doi.org/10.1111/j.1095-8649.2009.02423.xTo study smolt behaviour and survival of a northern Atlantic salmon (Salmo salar) population during river descent, sea entry and fjord migration, 120 wild S. salar were tagged with acoustic tags and registered at four automatic listening station arrays in the mouth of the North Norwegian River Alta and throughout the Alta Fjord. An estimated 75% of the post-smolts survived from the river mouth, through the estuary and the first 17 km of the fjord. Survival rates in the fjord varied with body length, and ranged from 97.0–99.5% per km. On average, the post-smolts spent 1.5 days (36 h, range 11–365 h) travelling from the river mouth to the last fjord array, 31 km from the river mouth. The migratory speed was slower (1.8 bl sec-135 ) in the first 4 km after sea entry compared to the next 27 km (3.0 bl sec-136 ). Post-smolts entered the fjord more often during the high or ebbing tide (70%). There was no clear diurnal migration pattern within the river and fjord, but most of the post-smolts entered the fjord at night (66%, 2000–0800 hours), despite the 24 h daylight at this latitude. The tidal cycle, wind-induced currents and the smolts‟ own movements seemed to influence migratory speeds and routes in different parts of the fjord. A large variation in migration patterns, both in river and fjord, might indicate that individuals in stochastic estuarine and marine environments are exposed to highly variable selection regimes resulting in different responses to environmental factors on both temporal and spatial scales. Post-smolts in northern Alta Fjord had similar early marine survival rates to those observed previously in southern fjords; however fjord residency in the north was shorter

Rasch analysis on OSCE data: An illustrative example

Background: The Objective Structured Clinical Examination (OSCE) is a widely used tool for the assessment of clinical competence in health professional education. The goal of the OSCE is to make reproducible decisions on pass/fail status as well as students’ levels of clinical competence according to their demonstrated abilities based on the scores. This paper explores the use of the polytomous Rasch model in evaluating the psychometric properties of OSCE scores through a case study. Method: The authors analysed an OSCE data set (comprised of 11 stations) for 80 fourth year medical students based on the polytomous Rasch model in an effort to answer two research questions: (1) Do the clinical tasks assessed in the 11 OSCE stations map on to a common underlying construct, namely clinical competence? (2) What other insights can Rasch analysis offer in terms of scaling, item analysis and instrument validation over and above the conventional analysis based on classical test theory? Results: The OSCE data set has demonstrated a sufficient degree of fit to the Rasch model (χ2 = 17.060, DF=22, p=0.76) indicating that the 11 OSCE station scores have sufficient psychometric properties to form a measure for a common underlying construct, i.e. clinical competence. Individual OSCE station scores with good fit to the Rasch model (p \u3e 0.1 for all χ2 statistics) further corroborated the characteristic of unidimensionality of the OSCE scale for clinical competence. A Person Separation Index (PSI) of 0.704 indicates sufficient level of reliability for the OSCE scores. Other useful findings from the Rasch analysis that provide insights, over and above the analysis based on classical test theory, are also exemplified using the data set. Conclusion: The polytomous Rasch model provides a useful and supplementary approach to the calibration and analysis of OSCE examination data

Using Rasch modeling to investigate the psychometric properties of the OSCE

Background It is vital that undergraduate medical students demonstrate competency in clinical skills upon graduation. To confidently state that students are competent in these skills, medical schools must assess students using tools that provide an accurate measure of their abilities. As with many medical schools, the School of Medicine at the University of Notre Dame Australia uses the Objective Structured Clinical Examination (OSCE). Research findings strongly support the OSCE as a reliable and valid measure of students’ clinical competence. However, traditional item analyses of OSCE papers were not providing the School with enough insight into how individual stations were performing and whether they were successfully measuring stated learning objectives. As such, item response theory, specifically the Rasch measurement model was adopted which has provided more in-depth analyses, the findings from which are broad-reaching in terms of improving assessment practices within the School. Objective This paper discusses the findings from the first phase of a three-part study where the overall objective is to use results from psychometric analyses based on the Rasch measurement model to inform and improve assessment practices. Phase 1 involves running Rasch analyses on an end-of-year OSCE paper to determine internal construct validity (among other things). Phase 2 involves using the findings from these analyses to inform station design for future papers, along with the professional development of staff and curriculum designers. Phase 3 will involve the implementation of further testing to determine if interventions in assessment practices have been successful, that is, if the internal construct validity of the OSCE has improved. Methods At the end of 2008, students in their final year of the four-year course sat a 10 station OSCE paper. Each of these stations was 15 minutes in duration and covered a range of clinical skills and disciplines. Using RUMM2020 (Andrich, Sheridan & Luo, 2004), a Rasch analysis was conducted whereby each OSCE station was considered an item. The Rasch unidimensional measurement model was employed for item and test analysis to serve as a quality monitoring and quality assurance procedure. The internal construct validity was assessed in terms of the characteristic of unidimensionality. The investigation of model fit was done concurrently for the test as a whole as well as at the individual item level. Results There was strong evidence of internal construct validity in terms of unidimensionality for the OSCE paper. For the OSCE exam as a whole, there was an excellent fit between the empirical data and the Rasch unidimensional measurement model (χ2 = 10.36, 20, p=0.96). Each individual station has also displayed excellent fit to the model. The task difficulty for each station ranges from -0.716 to +0.876 logits. The overall person separation index, however, is moderate (PSI=0.67). The traditional test statistics in terms of Point Biserial index provides further evidence of the quality of the OSCE examination. Conclusion The application of the Rasch unidimensional measurement model for the validation of the OSCE assessment has proven successful in this phase of the study. Findings show that Rasch provides in-depth insight into the psychometric properties of an OSCE which may be of interest to other medical schools. Suggestions and implications for continued use and refinement of OSCE testing are also discussed