Personal domains assessed in multiple mini interviews (MMIs) for healthcare student selection: A narrative synthesis systematic review

To examine the personal domains multiple mini interviews (MMIs) are being designed to assess, explore how they were determined and contextualise such domains in current and future healthcare student selection processes DESIGN: A systematic review of empirical research reporting on MMI model design was conducted from database inception to November 2017. Twelve electronic bibliographic databases. Evidence was extracted from original studies, and integrated in a narrative synthesis guided by the PRISMA statement for reporting systematic reviews. Personal domains were clustered into themes using a modified Delphi technique. A total of 584 articles were screened. 65 unique studies (80 articles) matched our inclusion criteria of which seven were conducted within nursing/midwifery faculties. Six in 10 studies featured applicants to medical school. Across selection processes, we identified 32 personal domains assessed by MMIs, the most frequent being: communication skills (84%), teamwork/collaboration (70%), and ethical/moral judgement (65%). Domains capturing ability to cope with stressful situations (14%), make decisions (14%), and resolve conflict in the workplace (13%) featured in fewer than ten studies overall. Intra- and inter-disciplinary inconsistencies in domain profiles were noted, as well as differences by entry level. MMIs deployed in nursing and midwifery assessed compassion and decision-making more frequently than in all other disciplines. Own programme philosophy and professional body guidance were most frequently cited (~50%) as sources for personal domains; a blueprinting process was reported in only 8% of studies. Nursing, midwifery and allied healthcare professionals should develop their theoretical frameworks for MMIs to ensure they are evidence-based and fit-for-purpose. We suggest a re-evaluation of domain priorities to ensure that students who are selected, not only have the capacity to offer the highest standards of care provision, but are able to maintain these standards when facing clinical practice and organisational pressures

The predictive validity of multiple mini interviews (MMIs) in nursing and midwifery programmes: year three findings from a cross-discipline cohort study

Education literature worldwide is replete with studies evaluating the effectiveness of Multiple Mini Interviews (MMIs) in admissions to medicine but <1% of published studies have been conducted in selection to nursing and midwifery programmes. To examine the predictive validity of MMIs using end of programme clinical and academic performance indicators of pre-registration adult, child, and mental health nursing and midwifery students. A cross-sectional cohort study at one university in the United Kingdom. A non-probability consecutive sampling strategy whereby all applicants to the September 2015 pre-registration adult, child, mental health nursing and midwifery programmes were invited to participate. Of the 354 students who commenced year one, 225 (64%) completed their three-year programme and agreed to take part (adult 120, child 32, mental health nursing 30 and midwifery 43). All applicants were interviewed using MMIs with six and seven station, four-minute models deployed in nursing and midwifery student selection respectively. Associations between MMI scores and the cross-discipline programme performance indicators available for each student at this university at the end of year three: clinical practice (assessed by mentors) and academic attainment (dissertation mark) were explored using multiple linear regression adjusting for applicant age, academic entry level, discipline and number of MMI stations. In the adjusted models, students with higher admissions MMI score (at six and seven stations) performed better in clinical practice (p < 0.001) but not in academic attainment (p = 0.122) at the end of their three-year programme. These findings provide the first report of the predictive validity of MMIs for performance in clinical practice using six and seven station models in nursing and midwifery programmes. Further evidence is required from both clinical and academic perspectives from larger, multi-site evaluations. [Abstract copyright: Crown Copyright © 2019. Published by Elsevier Ltd. All rights reserved.

The reliability and validity of multiple mini interviews (MMIs) in values based recruitment to nursing, midwifery and paramedic practice: Findings from an evaluation study

Background: Universities in the United Kingdom (UK) are required to incorporate values based recruitment (VBR) into their healthcare student selection processes. This reflects an international drive to strengthen the quality of healthcare service provision. This paper presents novel findings in relation to the reliability and predictive validity of multiple mini interviews (MMIs); one approach to VBR widely being employed by universities. Objectives: To examine the reliability (internal consistency) and predictive validity of MMIs using end of Year One practice outcomes of under-graduate pre-registration adult, child, mental health nursing, midwifery and paramedic practice students. Design: Cross-discipline evaluation study. Setting: One university in the United Kingdom. Participants: Data were collected in two streams: applicants to A) The September 2014 and 2015 Midwifery Studies programmes; B) September 2015 adult; Child and Mental Health Nursing and Paramedic Practice programmes. Fifty-seven midwifery students commenced their programme in 2014 and 69 in 2015; 47 and 54 agreed to participate and completed Year One respectively. 333 healthcare students commenced their programmes in September 2015. Of these, 281 agreed to participate and completed their first year (180 adult, 33 child and 34 mental health nursing and 34 paramedic practice students). Methods: Stream A featured a seven station four-minute model with one interviewer at each station and in Stream B a six station model was employed. Cronbach’s alpha was used to assess MMI station internal consistency and Pearson’s moment correlation co-efficient to explore associations between participants’ admission MMI score and end of Year one clinical practice outcomes (OSCE and mentor grading). Results: Stream A: Significant correlations are reported between midwifery applicant’s MMI scores and end of Year One practice outcomes. A multivariate linear regression model demonstrated that MMI score significantly predicted end of Year One practice outcomes controlling for age and academic entry level: coefficients 0.195 (p = 0.002) and 0.116 (p = 0.002) for OSCE and mentor grading respectively. In Stream B no significant correlations were found between MMI score and practice outcomes measured by mentor grading. Internal consistency for each MMI station was ‘excellent’ with values ranging from 0.966–0.974 across Streams A and B. Conclusion: This novel, cross-discipline study shows that MMIs are reliable VBR tools which have predictive validity when a seven station model is used. These data are important given the current international use of different MMI models in healthcare student selection processes

Developing a robust tool: advancing the multiple mini interview in pre-registration student midwife selection in a UK setting

Background: Published research has shown the multiple mini interview (MMI) to be a reliable assessment instrument in medical and nursing student selection internationally. Objectives: To develop, pilot and examine the reliability of MMIs in pre-registration student midwife selection in one Higher Education Institution a UK setting. Design and setting: BSc (Hons) Midwifery Studies students at a Higher Education Institution in the UK volunteered to participate in ‘mock’ MMI circuits during the first week of their programme. DeVellis’s framework for questionnaire development underpinned the generation of interview scenarios. Participants’ responses to scenario questions were rated on a 7 point scale. Internal consistency was calculated for each station. Results: An eight station model was piloted. Communication skills were assessed at each station as a generic attribute. Station specific attributes assessed included compassion and empathy, respect for difference and diversity, honesty and integrity, intellectual curiosity and reflective nature, advocacy, respect for privacy and dignity, team working and initiative, the role of the midwife and motivation to become a midwife. Cronbach’s alpha scores for each station ranged from 0.91 – 0.97. Conclusion: The systematic development of the MMI model and scenarios resulted in ‘excellent’ reliability across all stations. These findings endorse the MMI technique as a reliable alternative to the personal interview in informing final decisions in pre-registration student midwife selection

Feasibility of an automated interview grounded in Multiple Mini Interview (MMI) methodology for selection into the health professions: : an international multi-methods evaluation

Objectives: Global, Covid-driven restrictions around face-to-face interviews for healthcare student selection have forced admissions staff to rapidly adopt adapted online systems before supporting evidence is available. We have developed, what we believe is, the first automated interview grounded in Multiple Mini-Interview (MMI) methodology. This study aimed to explore test re-test reliability, acceptability, and usability of the system.Design, setting and participants: Multi-method feasibility study in Physician Associate (PA) programmes from two UK and one US university during 2019 - 2020.Primary, secondary outcomes: Feasibility measures (test-retest reliability acceptability and usability) were assessed using intra-class correlation (ICC), descriptive statistics, thematic and content analysis.Methods: Volunteers took (T1), then repeated (T2), the automated MMI, with a seven-day interval (+/- 2) then completed an evaluation questionnaire. Admissions staff participated in focus group discussions.Results: Sixty-two students and seven admission staff participated; 34 students and four staff from UK and 28 students and three staff from US universities.Good-excellent test-retest reliability was observed with T1 and T2 ICC between 0.62-0.81 (p<0.001) when assessed by individual total scores (range 80.6-119), station total scores 0.6-0.91, p<0.005, individual site (all ICC≥ 0.76 p<0.001) and mean test retest across sites 0.82 p<0.001 (95% CI 0.7-0.9).Admissions staff reported potential to reduce resource costs and bias through a more objective screening tool for pre-selection or to replace some MMI stations in a ‘hybrid model’. Maintaining human interaction through ‘touch points’ was considered essential.Users positively evaluated the system, stating it was intuitive with an accessible interface. Concepts chosen for dynamic probing needed to be appropriately tailored.Conclusion: These preliminary findings suggest that the system is reliable, generating consistent scores for candidates and is acceptable to end-users provided human touchpoints are maintained. Thus, there is evidence for the potential of such an automated system to augment healthcare student selection.Strengths and limitations of this study• The underpinning iterative theoretical approach enabled a responsive, dynamic design and development process for a new technology with no known precedent.• The conceptual leap from face-to-face or videoconference facilitated MMIs to a fully automated interview and assessment system may present barriers to stakeholders irrespective of the technology and its’ features.• The multi-method design provided for a diverse set of insights which have been essential to informing the progression of the technology.• We were unable to assess for potential differential performance within sub-groups, as would require a larger sample size

Developing and evaluating the multiple mini interview in student midwife selection.

The multiple mini interview (MMI) is an admissions instrument designed to replace the personal interview in health care student selection. Its effectiveness has been evaluated in medical student recruitment processes (Eva et al., 2004a, Roberts et al., 2014). At the commencement of this research no reliability, validity, or acceptability studies had been published specifically in relation to student midwife selection. Study objectives: to develop, pilot and evaluate the reliability, validity and acceptability of MMIs in student midwife selection in a Higher Education Institution (HEI) in the United Kingdom (UK). A dual paradigmatic dialectical enquiry was used in a multi-method case study. A literature review and qualitative work were conducted to identify the desirable personal qualities of a student midwife. This was followed by the systematic development of a customised eight station, five-minute MMI model. Sixty-two students from the BSc Midwifery Studies, September 2011 and 2012 cohorts, at the University of Surrey, volunteered to undertake ‘mock’ MMIs in the first week of their programme. Fifty-seven participants were followed up having completed their first year. Predictive validity was assessed using students’ end of year OSCE and mentor grading; station reliability, including inter-station and internal consistency, were also examined. Interviewers’ (n=9) and candidates’ (n=62) views of MMIs were obtained from a focus group and semi-structured questionnaires respectively. The literature review revealed that acknowledgement of the importance of ‘emotionality’ or an emotional dimension in the relationship between a woman and her midwife was missing from key professional, regulatory (Nursing and Midwifery Council, 2009, 2010, International Confederation of Midwives, 2011) and government documentation (Department of Health, DH,2012). According to the Department of Health (DH, 2013), selection to all National Health Service (NHS) funded training posts should incorporate recruitment for the NHS Constitution values (DH, 2012). In the absence of any mention of ‘emotionality’ it is suggested that this requires more specific recognition in considerations of what is important to appraise at selection. No statistically significant associations were found between students’ MMI score and their subsequent performance in clinical practice. The University Registrar would not consent to ‘live’ selection using MMIs in the absence of midwifery-specific evidence; participants were therefore students who had already been accepted onto an undergraduate midwifery programme. This has been addressed in an on-going longitudinal follow up-study. Reliability (internal consistency) was ‘excellent’ with Cronbach’s alpha scores between 0.91-0.97 across eight stations. Inter-station reliability findings suggested that each station measured different independent constructs with only a moderate positive correlation between two stations, kindness, compassion and respect for privacy and dignity (p<0.01). All other stations indicated little or no relationship offering additional support to the reliability of the scales. Candidates stated that undertaking MMIs would not discourage them from applying to the University as they felt they were a fair assessment instrument. They suggested that the multi- interview format was a positive feature which allowed them to recover from a poor performance at any one station. Overall, 23 participants (37%) reported a preference for MMIs compared to 22 (35%) who preferred a one-to-one personal interview format; 44 participants (71%) found the personal interview more daunting than MMIs. Interviewers appreciated the parity of opportunity afforded to candidates through the standardisation of the interview process. They were willing to adopt MMIs in future selection processes provided any anticipated complications were resolved. MMIs were shown to be reliable in the context and model defined. The insightful information obtained has informed a ‘roll out’ to MMIs across all health care student selection at the University of Surrey as well as being used by Health Education England as a case study example (HEE, 2014

Developing and evaluating the multiple mini interview in student midwife selection.

The multiple mini interview (MMI) is an admissions instrument designed to replace the personal interview in health care student selection. Its effectiveness has been evaluated in medical student recruitment processes (Eva et al., 2004a, Roberts et al., 2014). At the commencement of this research no reliability, validity, or acceptability studies had been published specifically in relation to student midwife selection. Study objectives: to develop, pilot and evaluate the reliability, validity and acceptability of MMIs in student midwife selection in a Higher Education Institution (HEI) in the United Kingdom (UK). A dual paradigmatic dialectical enquiry was used in a multi-method case study. A literature review and qualitative work were conducted to identify the desirable personal qualities of a student midwife. This was followed by the systematic development of a customised eight station, five-minute MMI model. Sixty-two students from the BSc Midwifery Studies, September 2011 and 2012 cohorts, at the University of Surrey, volunteered to undertake ‘mock’ MMIs in the first week of their programme. Fifty-seven participants were followed up having completed their first year. Predictive validity was assessed using students’ end of year OSCE and mentor grading; station reliability, including inter-station and internal consistency, were also examined. Interviewers’ (n=9) and candidates’ (n=62) views of MMIs were obtained from a focus group and semi-structured questionnaires respectively. The literature review revealed that acknowledgement of the importance of ‘emotionality’ or an emotional dimension in the relationship between a woman and her midwife was missing from key professional, regulatory (Nursing and Midwifery Council, 2009, 2010, International Confederation of Midwives, 2011) and government documentation (Department of Health, DH,2012). According to the Department of Health (DH, 2013), selection to all National Health Service (NHS) funded training posts should incorporate recruitment for the NHS Constitution values (DH, 2012). In the absence of any mention of ‘emotionality’ it is suggested that this requires more specific recognition in considerations of what is important to appraise at selection. No statistically significant associations were found between students’ MMI score and their subsequent performance in clinical practice. The University Registrar would not consent to ‘live’ selection using MMIs in the absence of midwifery-specific evidence; participants were therefore students who had already been accepted onto an undergraduate midwifery programme. This has been addressed in an on-going longitudinal follow up-study. Reliability (internal consistency) was ‘excellent’ with Cronbach’s alpha scores between 0.91-0.97 across eight stations. Inter-station reliability findings suggested that each station measured different independent constructs with only a moderate positive correlation between two stations, kindness, compassion and respect for privacy and dignity (p<0.01). All other stations indicated little or no relationship offering additional support to the reliability of the scales. Candidates stated that undertaking MMIs would not discourage them from applying to the University as they felt they were a fair assessment instrument. They suggested that the multi- interview format was a positive feature which allowed them to recover from a poor performance at any one station. Overall, 23 participants (37%) reported a preference for MMIs compared to 22 (35%) who preferred a one-to-one personal interview format; 44 participants (71%) found the personal interview more daunting than MMIs. Interviewers appreciated the parity of opportunity afforded to candidates through the standardisation of the interview process. They were willing to adopt MMIs in future selection processes provided any anticipated complications were resolved. MMIs were shown to be reliable in the context and model defined. The insightful information obtained has informed a ‘roll out’ to MMIs across all health care student selection at the University of Surrey as well as being used by Health Education England as a case study example (HEE, 2014

What is the value of Values Based Recruitment for nurse education programmes?

Aim A discussion of issues associated with Values Based Recruitment for nurse education programmes. Background Values Based Recruitment is a mandatory element in selection processes of students for Higher Education healthcare courses in England, including all programmes across nursing. Students are selected on the basis that their individual values align with those presented in the Constitution of the National Health Service. However, there are issues associated with the use of values as selection criteria that have been insufficiently addressed. These are discussed. Design Discussion paper. Data Sources This article is based on documents published on the website of the executive body responsible for the implementation of a policy regarding Values Based Recruitment in Higher Education Institutions up until June 2017 and our evaluation of the conceptualisation of Values Based Recruitment, underpinned by contemporary theory and literature. Implications for nursing Values Based Recruitment influences who is accepted onto a nurse education programme, but there has been limited critical evaluation regarding the effectiveness of employing values as selection criteria. Values are subject to interpretation and evidence regarding whether or how Values Based Recruitment will improve practice and care is lacking. The issues discussed in this article show that Higher Education Institutions offering nursing courses, whether in England or in other countries, should be critical and reflective regarding the implementation of Values Based Recruitment methods. Conclusion We call for a debate regarding the meaning and implications of Values Based Recruitment and further research regarding its validity and effectiveness.<br /

What is the value of Values Based Recruitment for nurse education programmes?

Aim A discussion of issues associated with Values Based Recruitment for nurse education programmes. Background Values Based Recruitment is a mandatory element in selection processes of students for Higher Education healthcare courses in England, including all programmes across nursing. Students are selected on the basis that their individual values align with those presented in the Constitution of the National Health Service. However, there are issues associated with the use of values as selection criteria that have been insufficiently addressed. These are discussed. Design Discussion paper. Data Sources This article is based on documents published on the website of the executive body responsible for the implementation of a policy regarding Values Based Recruitment in Higher Education Institutions up until June 2017 and our evaluation of the conceptualisation of Values Based Recruitment, underpinned by contemporary theory and literature. Implications for nursing Values Based Recruitment influences who is accepted onto a nurse education programme, but there has been limited critical evaluation regarding the effectiveness of employing values as selection criteria. Values are subject to interpretation and evidence regarding whether or how Values Based Recruitment will improve practice and care is lacking. The issues discussed in this article show that Higher Education Institutions offering nursing courses, whether in England or in other countries, should be critical and reflective regarding the implementation of Values Based Recruitment methods. Conclusion We call for a debate regarding the meaning and implications of Values Based Recruitment and further research regarding its validity and effectiveness.<br /