The Role of Leadership and Culture in Creating Meaningful Assessment: A Mixed Methods Case Study

With increased demands for institutional accountability and improved student learning, involvement in assessment has become a fundamental role of higher education faculty (Rhodes, 2010). However, faculty members and administrators often question whether assessment efforts do indeed improve student learning (Hutchings, 2010). This mixed methods case study of a faculty inquiry project explored how factors linked to organizational context (Kezar, 2013) are related to commitment to assessment and to use of assessment data by faculty members. Results indicated key best practices, such as developing faculty leaders and communities of practice to exchange ideas. The study provides insights for institutional administrators and faculty members seeking to develop a culture of assessment

The Pandemic Leadership Model: A Study of Medical Student Values During COVID-19

Background: Leadership training in medical school continues to grow. Little information exists to guide leadership program development. Concurrently, the COVID-19 pandemic provides a real-world crucible of leadership, allowing insight into qualities and characteristics medical students value. We aim to determine what students value in leadership during a pandemic and the implicit framework students use. Methods: We conducted a cross-sectional, qualitative study using a five-item novel survey instrument developed by a consensus group of experts from family medicine, leadership development, medical education, and survey research to elicit student perceptions of effective and ineffective leadership qualities and examples during the COVID-19 pandemic at the University of Michigan Medical School. We used thematic analysis to identify overarching themes to build a model of leadership integrated with existing theory. Results: 162 students participated across all years of medical school. We identified themes of Communication, Other-Orientation, Personal Characteristics, Decisive Action, and Use of Information. These five themes were then built into the model of Pandemic Leadership within the context of complexity leadership theory and collective leadership theory. This model represents qualities and characteristics students value in good leaders during a crisis. Conclusion: This study is unique in its focus on student perceptions of leadership qualities during a real-world laboratory for leadership. We hope that this information, along with the pandemic leadership model, can serve as the first step toward relevant leadership training programs in medical education. Leadership training programs in medical education would likely benefit from grounding in the student values identified by this study

Enhancing Nonverbal Communication Through Virtual Human Technology: Protocol for a Mixed Methods Study

Background: Communication is a critical component of the patient-provider relationship; however, limited research exists on the role of nonverbal communication. Virtual human training is an informatics-based educational strategy that offers various benefits in communication skill training directed at providers. Recent informatics-based interventions aimed at improving communication have mainly focused on verbal communication, yet research is needed to better understand how virtual humans can improve verbal and nonverbal communication and further elucidate the patient-provider dyad. Objective: The purpose of this study is to enhance a conceptual model that incorporates technology to examine verbal and nonverbal components of communication and develop a nonverbal assessment that will be included in the virtual simulation for further testing. Methods: This study will consist of a multistage mixed methods design, including convergent and exploratory sequential components. A convergent mixed methods study will be conducted to examine the mediating effects of nonverbal communication. Quantitative (eg, MPathic game scores, Kinect nonverbal data, objective structured clinical examination communication score, and Roter Interaction Analysis System and Facial Action Coding System coding of video) and qualitative data (eg, video recordings of MPathic–virtual reality [VR] interventions and student reflections) will be collected simultaneously. Data will be merged to determine the most crucial components of nonverbal behavior in human-computer interaction. An exploratory sequential design will proceed, consisting of a grounded theory qualitative phase. Using theoretical, purposeful sampling, interviews will be conducted with oncology providers probing intentional nonverbal behaviors. The qualitative findings will aid the development of a nonverbal communication model that will be included in a virtual human. The subsequent quantitative strand will incorporate and validate a new automated nonverbal communication behavior assessment into the virtual human simulation, MPathic-VR, by assessing interrater reliability, code interactions, and dyadic data analysis by comparing Kinect responses (system recorded) to manually scored records for specific nonverbal behaviors. Data will be integrated using building integration to develop the automated nonverbal communication behavior assessment and conduct a quality check of these nonverbal features. Results: Secondary data from the MPathic-VR randomized controlled trial data set (210 medical students and 840 video recordings of interactions) were analyzed in the first part of this study. Results showed differential experiences by performance in the intervention group. Following the analysis of the convergent design, participants consisting of medical providers (n=30) will be recruited for the qualitative phase of the subsequent exploratory sequential design. We plan to complete data collection by July 2023 to analyze and integrate these findings. Conclusions: The results from this study contribute to the improvement of patient-provider communication, both verbal and nonverbal, including the dissemination of health information and health outcomes for patients. Further, this research aims to transfer to various topical areas, including medication safety, informed consent processes, patient instructions, and treatment adherence between patients and providers

Enhancing Nonverbal Communication Through Virtual Human Technology: Protocol for a Mixed Methods Study

Background: Communication is a critical component of the patient-provider relationship; however, limited research exists on the role of nonverbal communication. Virtual human training is an informatics-based educational strategy that offers various benefits in communication skill training directed at providers. Recent informatics-based interventions aimed at improving communication have mainly focused on verbal communication, yet research is needed to better understand how virtual humans can improve verbal and nonverbal communication and further elucidate the patient-provider dyad. Objective: The purpose of this study is to enhance a conceptual model that incorporates technology to examine verbal and nonverbal components of communication and develop a nonverbal assessment that will be included in the virtual simulation for further testing. Methods: This study will consist of a multistage mixed methods design, including convergent and exploratory sequential components. A convergent mixed methods study will be conducted to examine the mediating effects of nonverbal communication. Quantitative (eg, MPathic game scores, Kinect nonverbal data, objective structured clinical examination communication score, and Roter Interaction Analysis System and Facial Action Coding System coding of video) and qualitative data (eg, video recordings of MPathic–virtual reality [VR] interventions and student reflections) will be collected simultaneously. Data will be merged to determine the most crucial components of nonverbal behavior in human-computer interaction. An exploratory sequential design will proceed, consisting of a grounded theory qualitative phase. Using theoretical, purposeful sampling, interviews will be conducted with oncology providers probing intentional nonverbal behaviors. The qualitative findings will aid the development of a nonverbal communication model that will be included in a virtual human. The subsequent quantitative strand will incorporate and validate a new automated nonverbal communication behavior assessment into the virtual human simulation, MPathic-VR, by assessing interrater reliability, code interactions, and dyadic data analysis by comparing Kinect responses (system recorded) to manually scored records for specific nonverbal behaviors. Data will be integrated using building integration to develop the automated nonverbal communication behavior assessment and conduct a quality check of these nonverbal features. Results: Secondary data from the MPathic-VR randomized controlled trial data set (210 medical students and 840 video recordings of interactions) were analyzed in the first part of this study. Results showed differential experiences by performance in the intervention group. Following the analysis of the convergent design, participants consisting of medical providers (n=30) will be recruited for the qualitative phase of the subsequent exploratory sequential design. We plan to complete data collection by July 2023 to analyze and integrate these findings. Conclusions: The results from this study contribute to the improvement of patient-provider communication, both verbal and nonverbal, including the dissemination of health information and health outcomes for patients. Further, this research aims to transfer to various topical areas, including medication safety, informed consent processes, patient instructions, and treatment adherence between patients and providers. International Registered Report Identifier (IRRID): DERR1-10.2196/4660

Descripciones de las prácticas de muestreo en cinco enfoques de investigación cualitativa en educación y ciencias de la salud

Obwohl vereinzelte Vorschläge zu Samplegrößen vorliegen, steht Literatur hierzu nach wie vor eher selten zur Verfügung. Um die Ergebnisse aus qualitativer Forschung transparenter und verlässlicher zu machen, sind jedoch praktische Vorschläge für die Konzeption von Studien und für die Budgetierung von Ressourcen erforderlich. Der vorliegende Artikel untersucht deshalb qualitative Samplegrößen und -praktiken innerhalb der Erziehungs- und Gesundheitswissenschaften für unterschiedliche Designs: Fallstudien, Ethnografie, Grounded-Theory-Methodologie, narrative Forschungsansätze und Phänomenologie. Einbezogen wurden die 51 am häufigsten zitierten Artikel aus diesen Bereichen entlang vorgängiger Kategorien und wesentlicher Charakteristika, die sich im Laufe der Studie ergaben. Insgesamt fand sich eine durchschnittliche Samplegröße von 87 Fällen, in weniger als der Hälfte der Artikel wurde die Samplingstrategie explizit benannt. Im Beitrag werden die Ergebnisse für jeden Ansatz beschrieben und Empfehlungen gegeben, die Forschenden, Gutachter/innen, Fördereinrichtungen und Studierenden helfen sollen, qualitative Samplingpraktiken besser einordnen und bewerten zu können.URN: http://nbn-resolving.de/urn:nbn:de:0114-fqs1502256Although recommendations exist for determining qualitative sample sizes, the literature appears to contain few instances of research on the topic. Practical guidance is needed for determining sample sizes to conduct rigorous qualitative research, to develop proposals, and to budget resources. The purpose of this article is to describe qualitative sample size and sampling practices within published studies in education and the health sciences by research design: case study, ethnography, grounded theory methodology, narrative inquiry, and phenomenology. I analyzed the 51 most highly cited studies using predetermined content categories and noteworthy sampling characteristics that emerged. In brief, the findings revealed a mean sample size of 87. Less than half of the studies identified a sampling strategy. I include a description of findings by approach and recommendations for sampling to assist methodologists, reviewers, program officers, graduate students, and other qualitative researchers in understanding qualitative sampling practices in recent studies.URN: http://nbn-resolving.de/urn:nbn:de:0114-fqs1502256Aunque existen recomendaciones para la determinación de tamaños de la muestra cualitativa, la literatura parece contener pocos casos de investigación sobre el tema. Es necesaria una guía práctica para la determinación de tamaños de muestra para realizar rigurosa investigación cualitativa, para desarrollar propuestas y para presupuestar recursos. El propósito de este artículo es describir prácticas de muestreo y de tamaño de la muestra cualitativa en los estudios publicados en educación y ciencias de la salud por el diseño de investigación: estudio de caso, etnografía, metodología de teoría fundamentada, investigación narrativa y fenomenología. He analizado los 51 estudios más altamente citados usando las categorías de contenido predeterminadas y las características más notables de muestreo que surgieron. En resumen, los resultados revelaron un tamaño de muestra media de 87. Menos de la mitad de los estudios identificaron una estrategia de muestreo. Incluyo una descripción de los resultados por enfoque y recomendaciones para el muestreo a fin de ayudar a metodólogos, revisores, responsables de programa, estudiantes de posgrado y otros investigadores cualitativos para lograr comprensión de las prácticas de muestreo cualitativo en estudios recientes.URN: http://nbn-resolving.de/urn:nbn:de:0114-fqs150225

The Development, Design, and Test of a Self-Assessment Instrument of Mixed Methods Research Proficiency

Mixed methods research is the collection, analysis, and integration of qualitative and quantitative data. The use of mixed methods research has been growing exponentially in fields such as health sciences and education. Demand for training is also increasing. However, a small portion of mixed methods literature addresses teaching and learning mixed methods. While that small body of literature focuses on teaching and pedagogy, this study took the perspective of the proficiency to conduct mixed methods. This study examined mixed methods proficiency, using a conceptual framework from second language acquisition, the Common European Framework. Little is known about the underlying knowledge, skills, and existential competence to develop mixed methods proficiency. An instrument is needed to gather baseline information. This study employed an exploratory sequential mixed methods design for the purpose of developing a typology and instrument. It consisted of three phases: 1) an initial qualitative exploration of proficiency involving interviews (n = 8) with leading mixed methods methodologists and an examination of curricula (n = 25); 2) an intermediate instrument development phase using building integration to develop the brief Mixed Methods Skills Self-Assessment instrument; and 3) a quantitative follow-up phase that administered the instrument to researchers using mixed methods (n = 264) to examine reliability of scores, evidence of validity, and correlates among the data. The integrated qualitative and quantitative results indicated that mixed methods proficiency is comprised of underlying professional experiences, knowledge, skills (know how), and personal characteristics of researchers. The instrument yielded some evidence of validity and adequate reliability. It can be completed in seven minutes, on average, and may be used in workshops, courses, and professional development plans. The study also tested a path model of proficiency. The model indicated that professional experiences are important to develop knowledge and skills. Both underlying declarative knowledge about mixed methods and professional experiences are important develops to skills (know how). Working on a mixed methods project seems to be the most critical aspect to develop proficiency

The Development, Design, and Test of a Self-Assessment Instrument of Mixed Methods Research Proficiency

Mixed methods research is the collection, analysis, and integration of qualitative and quantitative data. The use of mixed methods research has been growing exponentially in fields such as health sciences and education. Demand for training is also increasing. However, a small portion of mixed methods literature addresses teaching and learning mixed methods. While that small body of literature focuses on teaching and pedagogy, this study took the perspective of the proficiency to conduct mixed methods. This study examined mixed methods proficiency, using a conceptual framework from second language acquisition, the Common European Framework. Little is known about the underlying knowledge, skills, and existential competence to develop mixed methods proficiency. An instrument is needed to gather baseline information. This study employed an exploratory sequential mixed methods design for the purpose of developing a typology and instrument. It consisted of three phases: 1) an initial qualitative exploration of proficiency involving interviews (n = 8) with leading mixed methods methodologists and an examination of curricula (n = 25); 2) an intermediate instrument development phase using building integration to develop the brief Mixed Methods Skills Self-Assessment instrument; and 3) a quantitative follow-up phase that administered the instrument to researchers using mixed methods (n = 264) to examine reliability of scores, evidence of validity, and correlates among the data. The integrated qualitative and quantitative results indicated that mixed methods proficiency is comprised of underlying professional experiences, knowledge, skills (know how), and personal characteristics of researchers. The instrument yielded some evidence of validity and adequate reliability. It can be completed in seven minutes, on average, and may be used in workshops, courses, and professional development plans. The study also tested a path model of proficiency. The model indicated that professional experiences are important to develop knowledge and skills. Both underlying declarative knowledge about mixed methods and professional experiences are important develops to skills (know how). Working on a mixed methods project seems to be the most critical aspect to develop proficiency

A software feature for mixed methods analysis: The MAXQDA Interactive Quote Matrix

Mixed methods research relies on the integration of quantitative and qualitative data to develop meta-inferences. Despite an increase in techniques to facilitate integration at the methods and reporting levels, there is less practical guidance on how to conduct mixed methods integrative data analysis. Qualitative data analysis software is rapidly advancing to facilitate integrative analysis in mixed methods research studies. However, examples on how these features support analysis are relatively limited. The purpose of this article is to promote the use of qualitative data analysis software for integrative analysis in mixed methods research by describing the value of the major features available in software (i.e., code queries, crosstabulations, and quote matrices). In addition to a brief overview of these features, this article demonstrates the use of MAXQDA's Interactive Quote Matrix with a real-life example of a completed mixed methods study and delineates the step-by-step process on how to use this feature to develop meta-inferences