Practical Recommendations for Mixed Methods Sampling in Psychological Intervention Research: A Mixed Methods Case Study

Sampling is integral to the research process and, if not appropriately addressed, can affect the meta-inferences of the mixed methods study. Sampling is also closely related to recruitment, retention, and additional methodological components. Sampling issues are magnified in social and health sciences intervention research due to the temporal placement of data collection and analysis. Limited research has examined sampling based on researchers’ rationales and decision-making across mixed methods psychological intervention research. This study explored this phenomenon to develop and refine a list of practical recommendations for sampling in mixed methods that were tested using content validity. Using an exploratory sequential mixed methods case study design, the first phase consisted of a qualitative case study using two data sources, a mixed method research-systematic methodological review (MMR-SMR), and semi-structured interviews with researchers who have conducted a mixed methods psychological intervention study. Forty studies were identified through the MMR-SMR and coded using a codebook. Semi-structured interviews were conducted with researchers (N = 10), and several overarching themes were identified. Through building integration, the qualitative findings informed the development of a list of preliminary recommendations that was refined using a modified e-Delphi study for the quantitative phase. Experts (i.e., mixed methods research methodologists) were asked to rate each recommendation\u27s relevancy. Agreement consensus was established based on median and item-content validity index (I-CVI) values to test a component of content validity across each recommendation. Participants rated recommendations across Round 1 (N = 10) and Round 2 (N = 9). Recommendations were modified based on participant ratings and open-ended responses. The final list consisted of 20 recommendations, each demonstrating adequate evidence of content validity. These recommendations span various categories, including recruitment, retention, sampling across mixed methods research designs, data collection, integrating mixed methods samples, and temporal placement of qualitative strand. Multiple audiences, including researchers, mixed methods research methodologists, and grant and journal reviewers, can use the list of recommendations to guide sampling decisions in mixed methods psychological intervention research. Advisor: Wayne A. Babchu

Police surveillance of cell phone location data: Supreme Court versus public opinion

The Fourth Amendment to the US Constitution protects individuals from unreasonable searches and seizures. As technology evolves, courts must examine Fourth Amendment concerns implicated by the introduction of new and enhanced police surveillance techniques. Recent Supreme Court cases have demonstrated a trend towards reconsidering the mechanical application of traditional Fourth Amendment doctrine to define the scope of constitutional protections for modern technological devices and personal data. The current research examined whether public opinion regarding privacy rights in electronic communications is in accordance with these Supreme Court rulings. Results suggest that cell phone location data is perceived as more private and deserving of protections than other types of location data, but the privacy of other types of information recorded on cell phones is valued even more than location data. These results have implications for the police and courts considering how the Fourth Amendment will apply to smart phone technologies

Applying Visual Methods to Document the History of Psychological Testing: A Qualitative Approach

The history of psychological testing is critical to many areas of applied psychology. Assessment forms a mainstay of clinical practice, second only to psychotherapy (Meyer et al., 2001). In industrial/organizational psychological practice, employee selection depends on testing to assess applicant qualifications. In educational contexts, testing is central to the evaluation of academic performance and college readiness, in addition to determining eligibility for various types of special educational services. The history of testing is deeply rooted in myriad psychological specialties (Carlson & Geisinger, 2021). This fact prompted a qualitative examination and integration of three distinct historical threads identified by the proposal authors: (a) the history of psychological testing in the U.S. over the past century, (b) the work of Oscar Buros and the Institute of Mental Measurements that proved instrumental in the dissemination of reference materials concerning commercial testing products (Brayfield, 1979; Dyer, 1968; Gough, 1980), and (c) the lives of Oscar and Luella Buros who—among other things—established an enterprise that continues to provide critical evaluations of commercially available tests that protect consumers of those products (Cizek et al., 2007; Reynolds, 2010). This poster presentation examines critical events in the history of psychological testing from the 1930s to present-day through the lens of the Buros Institute/Center and the lives of Oscar and Luella Buros. The research method employed demonstrates the utility of a nascent qualitative approach to integrate three strands of history concerning psychological testing. The procedure emphasized visual methods (e.g., Ownby, 2017; Reavey, 2020) to systematically collect qualitative data, primarily photographs. Photographs (e.g., photo documentation) were gathered from archival and current sources, including the Luella Buros Collection curated by the Anthropology Department of a mid-Western university. Next, we aligned depictions of major historical events with depictions of events in the history of the Buros Institute/Center and the lives of the Buroses. The poster included images selected from a pool of more than 12,000 by consensus of the researchers. The poster displays visual representations (e.g., photographs) grouped by decades over the past century. The researchers link visual depictions (e.g., the first Mental Measurements Yearbook [MMY] in 1938, staff at work producing early MMYs, the house in NJ where the Buros Institute was established) with salient events across the histories of psychological testing. A qualitative analysis/summary demonstrates associations between and among the three specified historical strands. References & credits are in the attached supplemental file

Enhancing Nonverbal Communication Through Virtual Human Technology: Protocol for a Mixed Methods Study

Background: Communication is a critical component of the patient-provider relationship; however, limited research exists on the role of nonverbal communication. Virtual human training is an informatics-based educational strategy that offers various benefits in communication skill training directed at providers. Recent informatics-based interventions aimed at improving communication have mainly focused on verbal communication, yet research is needed to better understand how virtual humans can improve verbal and nonverbal communication and further elucidate the patient-provider dyad. Objective: The purpose of this study is to enhance a conceptual model that incorporates technology to examine verbal and nonverbal components of communication and develop a nonverbal assessment that will be included in the virtual simulation for further testing. Methods: This study will consist of a multistage mixed methods design, including convergent and exploratory sequential components. A convergent mixed methods study will be conducted to examine the mediating effects of nonverbal communication. Quantitative (eg, MPathic game scores, Kinect nonverbal data, objective structured clinical examination communication score, and Roter Interaction Analysis System and Facial Action Coding System coding of video) and qualitative data (eg, video recordings of MPathic–virtual reality [VR] interventions and student reflections) will be collected simultaneously. Data will be merged to determine the most crucial components of nonverbal behavior in human-computer interaction. An exploratory sequential design will proceed, consisting of a grounded theory qualitative phase. Using theoretical, purposeful sampling, interviews will be conducted with oncology providers probing intentional nonverbal behaviors. The qualitative findings will aid the development of a nonverbal communication model that will be included in a virtual human. The subsequent quantitative strand will incorporate and validate a new automated nonverbal communication behavior assessment into the virtual human simulation, MPathic-VR, by assessing interrater reliability, code interactions, and dyadic data analysis by comparing Kinect responses (system recorded) to manually scored records for specific nonverbal behaviors. Data will be integrated using building integration to develop the automated nonverbal communication behavior assessment and conduct a quality check of these nonverbal features. Results: Secondary data from the MPathic-VR randomized controlled trial data set (210 medical students and 840 video recordings of interactions) were analyzed in the first part of this study. Results showed differential experiences by performance in the intervention group. Following the analysis of the convergent design, participants consisting of medical providers (n=30) will be recruited for the qualitative phase of the subsequent exploratory sequential design. We plan to complete data collection by July 2023 to analyze and integrate these findings. Conclusions: The results from this study contribute to the improvement of patient-provider communication, both verbal and nonverbal, including the dissemination of health information and health outcomes for patients. Further, this research aims to transfer to various topical areas, including medication safety, informed consent processes, patient instructions, and treatment adherence between patients and providers. International Registered Report Identifier (IRRID): DERR1-10.2196/4660

Enhancing Nonverbal Communication Through Virtual Human Technology: Protocol for a Mixed Methods Study

Background: Communication is a critical component of the patient-provider relationship; however, limited research exists on the role of nonverbal communication. Virtual human training is an informatics-based educational strategy that offers various benefits in communication skill training directed at providers. Recent informatics-based interventions aimed at improving communication have mainly focused on verbal communication, yet research is needed to better understand how virtual humans can improve verbal and nonverbal communication and further elucidate the patient-provider dyad. Objective: The purpose of this study is to enhance a conceptual model that incorporates technology to examine verbal and nonverbal components of communication and develop a nonverbal assessment that will be included in the virtual simulation for further testing. Methods: This study will consist of a multistage mixed methods design, including convergent and exploratory sequential components. A convergent mixed methods study will be conducted to examine the mediating effects of nonverbal communication. Quantitative (eg, MPathic game scores, Kinect nonverbal data, objective structured clinical examination communication score, and Roter Interaction Analysis System and Facial Action Coding System coding of video) and qualitative data (eg, video recordings of MPathic–virtual reality [VR] interventions and student reflections) will be collected simultaneously. Data will be merged to determine the most crucial components of nonverbal behavior in human-computer interaction. An exploratory sequential design will proceed, consisting of a grounded theory qualitative phase. Using theoretical, purposeful sampling, interviews will be conducted with oncology providers probing intentional nonverbal behaviors. The qualitative findings will aid the development of a nonverbal communication model that will be included in a virtual human. The subsequent quantitative strand will incorporate and validate a new automated nonverbal communication behavior assessment into the virtual human simulation, MPathic-VR, by assessing interrater reliability, code interactions, and dyadic data analysis by comparing Kinect responses (system recorded) to manually scored records for specific nonverbal behaviors. Data will be integrated using building integration to develop the automated nonverbal communication behavior assessment and conduct a quality check of these nonverbal features. Results: Secondary data from the MPathic-VR randomized controlled trial data set (210 medical students and 840 video recordings of interactions) were analyzed in the first part of this study. Results showed differential experiences by performance in the intervention group. Following the analysis of the convergent design, participants consisting of medical providers (n=30) will be recruited for the qualitative phase of the subsequent exploratory sequential design. We plan to complete data collection by July 2023 to analyze and integrate these findings. Conclusions: The results from this study contribute to the improvement of patient-provider communication, both verbal and nonverbal, including the dissemination of health information and health outcomes for patients. Further, this research aims to transfer to various topical areas, including medication safety, informed consent processes, patient instructions, and treatment adherence between patients and providers

Assessing Quality in Mixed Methods Research: A Case Study Operationalizing the Legitimation Typology

Quality in mixed methods research (MMR) has been an ongoing topic of discussion over the past two decades. One of the obstacles of assessing quality in a MMR study is developing credible inferences from the integration of quantitative and qualitative approaches (Plano Clark & Ivankova, 2016). Some researchers have designed a variety of strategies for assessing quality of a mixed methods study as a whole (Teddlie, & Tashakkori, 2003; Onwuegbuzie & Johnson, 2006; Tashakkori, & Teddlie, 2003), but there is no general consensus among researchers on which methods to use. The aim of this intrinsic, exploratory case study was to investigate how researchers assess quality of a MMR study, particularly using the legitimation typology, and ultimately, operationalize the legitimation typology to increase its applicability in MMR. Individual semi-structured interviews were conducted with the co-developer of the legitimation typology, researchers who had applied the legitimation typology to their empirical study, and mixed methods scholars who have written about or share knowledge on the legitimation typology. Data analysis revealed eight themes: (1) role of validity in MMR, (2) importance of integration, (3) value added to discordant data, (4) versatility of the legitimation typology, (5) role of colleagues/mentors in MMR, (6) researchers’ application/interpretations of legitimation types, (7) clarifications to the legitimation typology, and (8) researcher recommendations. Based on these findings, several recommendations to the current 2017 legitimation typology are proposed. Advisor: Wayne A. Babchu

Practical Recommendations for Mixed Methods Sampling in Psychological Intervention Research: A Mixed Methods Case Study

Sampling is integral to the research process and, if not appropriately addressed, can affect the meta-inferences of the mixed methods study. Sampling is also closely related to recruitment, retention, and additional methodological components. Sampling issues are magnified in social and health sciences intervention research due to the temporal placement of data collection and analysis. Limited research has examined sampling based on researchers’ rationales and decision-making across mixed methods psychological intervention research. This study explored this phenomenon to develop and refine a list of practical recommendations for sampling in mixed methods that were tested using content validity. Using an exploratory sequential mixed methods case study design, the first phase consisted of a qualitative case study using two data sources, a mixed method research-systematic methodological review (MMR-SMR), and semi-structured interviews with researchers who have conducted a mixed methods psychological intervention study. Forty studies were identified through the MMR-SMR and coded using a codebook. Semi-structured interviews were conducted with researchers (N = 10), and several overarching themes were identified. Through building integration, the qualitative findings informed the development of a list of preliminary recommendations that was refined using a modified e-Delphi study for the quantitative phase. Experts (i.e., mixed methods research methodologists) were asked to rate each recommendation\u27s relevancy. Agreement consensus was established based on median and item-content validity index (I-CVI) values to test a component of content validity across each recommendation. Participants rated recommendations across Round 1 (N = 10) and Round 2 (N = 9). Recommendations were modified based on participant ratings and open-ended responses. The final list consisted of 20 recommendations, each demonstrating adequate evidence of content validity. These recommendations span various categories, including recruitment, retention, sampling across mixed methods research designs, data collection, integrating mixed methods samples, and temporal placement of qualitative strand. Multiple audiences, including researchers, mixed methods research methodologists, and grant and journal reviewers, can use the list of recommendations to guide sampling decisions in mixed methods psychological intervention research

Enhancing Nonverbal Communication Through Virtual Human Technology: Protocol for a Mixed Methods Study.

BACKGROUND: Communication is a critical component of the patient-provider relationship; however, limited research exists on the role of nonverbal communication. Virtual human training is an informatics-based educational strategy that offers various benefits in communication skill training directed at providers. Recent informatics-based interventions aimed at improving communication have mainly focused on verbal communication, yet research is needed to better understand how virtual humans can improve verbal and nonverbal communication and further elucidate the patient-provider dyad. OBJECTIVE: The purpose of this study is to enhance a conceptual model that incorporates technology to examine verbal and nonverbal components of communication and develop a nonverbal assessment that will be included in the virtual simulation for further testing. METHODS: This study will consist of a multistage mixed methods design, including convergent and exploratory sequential components. A convergent mixed methods study will be conducted to examine the mediating effects of nonverbal communication. Quantitative (eg, MPathic game scores, Kinect nonverbal data, objective structured clinical examination communication score, and Roter Interaction Analysis System and Facial Action Coding System coding of video) and qualitative data (eg, video recordings of MPathic-virtual reality [VR] interventions and student reflections) will be collected simultaneously. Data will be merged to determine the most crucial components of nonverbal behavior in human-computer interaction. An exploratory sequential design will proceed, consisting of a grounded theory qualitative phase. Using theoretical, purposeful sampling, interviews will be conducted with oncology providers probing intentional nonverbal behaviors. The qualitative findings will aid the development of a nonverbal communication model that will be included in a virtual human. The subsequent quantitative strand will incorporate and validate a new automated nonverbal communication behavior assessment into the virtual human simulation, MPathic-VR, by assessing interrater reliability, code interactions, and dyadic data analysis by comparing Kinect responses (system recorded) to manually scored records for specific nonverbal behaviors. Data will be integrated using building integration to develop the automated nonverbal communication behavior assessment and conduct a quality check of these nonverbal features. RESULTS: Secondary data from the MPathic-VR randomized controlled trial data set (210 medical students and 840 video recordings of interactions) were analyzed in the first part of this study. Results showed differential experiences by performance in the intervention group. Following the analysis of the convergent design, participants consisting of medical providers (n=30) will be recruited for the qualitative phase of the subsequent exploratory sequential design. We plan to complete data collection by July 2023 to analyze and integrate these findings. CONCLUSIONS: The results from this study contribute to the improvement of patient-provider communication, both verbal and nonverbal, including the dissemination of health information and health outcomes for patients. Further, this research aims to transfer to various topical areas, including medication safety, informed consent processes, patient instructions, and treatment adherence between patients and providers. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/46601.http://deepblue.lib.umich.edu/bitstream/2027.42/176916/2/Perez 2023 Published Protocol.pdfPublished onlineDescription of Perez 2023 Published Protocol.pdf : Published versio