Performance of a cognitive load inventory during simulated handoffs: Evidence for validity.

BackgroundAdvancing patient safety during handoffs remains a public health priority. The application of cognitive load theory offers promise, but is currently limited by the inability to measure cognitive load types.ObjectiveTo develop and collect validity evidence for a revised self-report inventory that measures cognitive load types during a handoff.MethodsBased on prior published work, input from experts in cognitive load theory and handoffs, and a think-aloud exercise with residents, a revised Cognitive Load Inventory for Handoffs was developed. The Cognitive Load Inventory for Handoffs has items for intrinsic, extraneous, and germane load. Students who were second- and sixth-year students recruited from a Dutch medical school participated in four simulated handoffs (two simple and two complex cases). At the end of each handoff, study participants completed the Cognitive Load Inventory for Handoffs, Paas' Cognitive Load Scale, and one global rating item for intrinsic load, extraneous load, and germane load, respectively. Factor and correlational analyses were performed to collect evidence for validity.ResultsConfirmatory factor analysis yielded a single factor that combined intrinsic and germane loads. The extraneous load items performed poorly and were removed from the model. The score from the combined intrinsic and germane load items associated, as predicted by cognitive load theory, with a commonly used measure of overall cognitive load (Pearson's r = 0.83, p < 0.001), case complexity (beta = 0.74, p < 0.001), level of experience (beta = -0.96, p < 0.001), and handoff accuracy (r = -0.34, p < 0.001).ConclusionThese results offer encouragement that intrinsic load during handoffs may be measured via a self-report measure. Additional work is required to develop an adequate measure of extraneous load

Assessment of Medical Students’ Shared Decision-Making in Standardized Patient Encounters

BackgroundShared decision-making, in which physicians and patients openly explore beliefs, exchange information, and reach explicit closure, may represent optimal physician-patient communication. There are currently no universally accepted methods to assess medical students' competence in shared decision-making.ObjectiveTo characterize medical students' shared decision-making with standardized patients (SPs) and determine if students' use of shared decision-making correlates with SP ratings of their communication.DesignRetrospective study of medical students' performance with four SPs.ParticipantsSixty fourth-year medical students.MeasurementsObjective blinded coding of shared decision-making quantified as decision moments (exploration/articulation of perspective, information sharing, explicit closure for a particular decision); SP scoring of communication skills using a validated checklist.ResultsOf 779 decision moments generated in 240 encounters, 312 (40%) met criteria for shared decision-making. All students engaged in shared decision-making in at least two of the four cases, although in two cases 5% and 12% of students engaged in no shared decision-making. The most commonly discussed decision moment topics were medications (n = 98, 31%), follow-up visits (71, 23%), and diagnostic testing (44, 14%). Correlations between the number of decision moments in a case and students' communication scores were low (rho = 0.07 to 0.37).ConclusionsAlthough all students engaged in some shared decision-making, particularly regarding medical interventions, there was no correlation between shared decision-making and overall communication competence rated by the SPs. These findings suggest that SP ratings of students' communication skill cannot be used to infer students' use of shared decision-making. Tools to determine students' skill in shared decision-making are needed

Validation of a self-efficacy instrument and its relationship to performance of crisis resource management skills

Self-efficacy is thought to be important for resuscitation proficiency in that it influences the development of and access to the associated medical knowledge, procedural skills and crisis resource management (CRM) skills. Since performance assessment of CRM skills is challenging, self-efficacy is often used as a measure of competence in this area. While self-efficacy may influence performance, the true relationship between self-efficacy and performance in this setting has not been delineated. We developed an instrument to measure pediatric residents’ self-efficacy in CRM skills and assessed its content validity, internal structure, and relationship to other variables. After administering the instrument to 125 pediatric residents, critical care fellows and faculty, we performed an exploratory factor analysis within a confirmatory factor analysis as well as a known group comparison. The analyses specified four factors that we defined as: situation awareness, team management, environment management, and decision making. Pediatric residents reported lower self-efficacy than fellows and faculty in each factor. We also examined the correlation between self-efficacy and performance scores for a subset of 30 residents who led video recorded simulated resuscitations and had their performances rated by three observers. We found a significant, positive correlation between residents’ self-efficacy in situation awareness and environment management and their overall performance of CRM skills. Our findings suggest that in a specific context, self-efficacy as a form of self-assessment may be informative with regards to performance

Does Student Confidence on Multiple-Choice Question Assessments Provide Useful Information?

Context: Feedback from multiple-choice question (MCQ) assessments is typically limited to a percentage correct score, from which estimates of student competence are inferred. The students\u27 confidence in their answers and the potential impact of incorrect answers on clinical care are seldom recorded. Our purpose was to evaluate student confidence in incorrect responses and to establish how confidence was influenced by the potential clinical impact of answers, question type and gender. Methods: This was an exploratory, cross-sectional study conducted using a convenience sample of 104 Year 3 dental students completing 20 MCQs on implant dentistry. Students were asked to select the most correct response and to indicate their confidence in it for each question. Identifying both correctness and confidence allowed the designation of uninformed (incorrect and not confident) or misinformed (incorrect but confident) responses. In addition to recording correct/incorrect responses and student confidence, faculty staff designated incorrect responses as benign, inappropriate or potentially harmful if applied to clinical care. Question type was identified as factual or complex. Logistic regression was used to evaluate relationships between student confidence, and question type and gender. Results: Students were misinformed more often than uninformed (22% versus 8%), and misinformed responses were more common with complex than factual questions (p \u3c 0.05). Students were significantly more likely to be confident of correct than incorrect benign, incorrect inappropriate or incorrect harmful answers (p \u3c 0.001), but, contrary to expectations, confidence did not decrease as answers became more harmful. Conclusions: Recording student confidence was helpful in identifying uninformed versus misinformed responses, which may allow for targeted remediation strategies. Making errors of calibration (confidence and accuracy) more visible may be relevant in feedback for professional development

Development of a Multi-Domain Assessment Tool for Quality Improvement Projects.

BackgroundImproving the quality of health care and education has become a mandate at all levels within the medical profession. While several published quality improvement (QI) assessment tools exist, all have limitations in addressing the range of QI projects undertaken by learners in undergraduate medical education, graduate medical education, and continuing medical education.ObjectiveWe developed and validated a tool to assess QI projects with learner engagement across the educational continuum.MethodsAfter reviewing existing tools, we interviewed local faculty who taught QI to understand how learners were engaged and what these faculty wanted in an ideal assessment tool. We then developed a list of competencies associated with QI, established items linked to these competencies, revised the items using an iterative process, and collected validity evidence for the tool.ResultsThe resulting Multi-Domain Assessment of Quality Improvement Projects (MAQIP) rating tool contains 9 items, with criteria that may be completely fulfilled, partially fulfilled, or not fulfilled. Interrater reliability was 0.77. Untrained local faculty were able to use the tool with minimal guidance.ConclusionsThe MAQIP is a 9-item, user-friendly tool that can be used to assess QI projects at various stages and to provide formative and summative feedback to learners at all levels

The efficacy of an anatomy and ultrasonography workshop on improving residents’ confidence and knowledge in regional anesthesia

Abstract Background Ultrasound Guided Regional Anesthesia (UGRA) has become the standard for regional anesthesia practice, but there is not a standardized educational approach for training residents. The objective of this study was to evaluate the efficacy of an UGRA workshop utilizing the theoretical framework of embodied cognition for anesthesiology residents. Methods A workshop was developed consisting of didactics, scanning training on standardized patients (SPs) and anatomy reviews on prosected cadavers that focused on the most common UGRA procedures for the upper and lower extremity. At the beginning of the workshop and at the end of the workshop residents completed pre-test and pre-confidence surveys, as well as post-test and post-confidence surveys, respectively to assess the impact of the workshop. Results 39 residents (100% of the possible residents) participated in the workshop in 2019. Residents’ confidence in identifying relevant anatomy for the most common UGRA procedures significantly increased in 13 of the 14 measurements. Residents’ knowledge gain was also statistically significant from the pre-test to post-test (20.13 ± 3.61 and 26.13 ± 2.34; p < .0001). The residents found the course overall to be very useful (4.90 ± 0.38) and in particular the cadaveric component was highly rated (4.74 ± 0.55). Conclusions In this study, we developed a workshop guided by the embodied cognition framework to aid in shortening the overall learning curve of UGRA for anesthesiology residents. Based on our results this workshop should be replicated by institutions that are hoping to decrease the learning curve associated with UGRA and increase residents’ confidence in identifying the relevant anatomy in UGRA nerve blocks

Optimizing referrals for GI specialty care

Rationale, aims and objectivesSpecialty care referrals have doubled in the last decade. Optimization of the pre-referral workup by a primary care doctor can lead to a more efficient first specialty visit with the patient. Guidance regarding pre-referral laboratory testing is a first step towards improving the specialty referral process. Our aim was to establish consensus regarding appropriate pre-referral workup for common gastrointestinal and liver conditions.MethodsThe Delphi method was used to establish local consensus for recommending certain laboratory tests prior to specialty referral for 13 clinical conditions. Seven conditions from The University of Michigan outpatient referral guidelines were used as a baseline. An expert panel of three PCPs and nine gastroenterologists from three academic hospitals participated in three iterative rounds of electronic surveys. Each panellist ranked each test using a 5-point Likert scale (strongly disagree to strongly agree). Local panellists could recommend additional tests for the initial diagnoses, and also recommended additional diagnoses needing guidelines: iron deficiency anaemia, abdominal pain, irritable bowel syndrome, fatty liver disease, liver mass and cirrhosis. Consensus was defined as ≥70% of experts scoring ≥4 (agree or strongly agree).ResultsApplying Delphi methodology to extrapolate externally developed referral guidelines for local implementation resulted in considerable modifications. For some conditions, many tests from the external group were eliminated by the local group (abdominal bloating; iron deficiency anaemia; irritable bowel syndrome). In contrast, for chronic diarrhoea, abnormal liver enzymes and viral hepatitis, all/most original tests were retained with additional tests added. For liver mass, fatty liver disease and cirrhosis, there was high concordance among the panel with few tests added or eliminated.ConclusionsConsideration of externally developed referral guidelines using a consensus-building process leads to significant local tailoring and adaption. Our next steps include implementation and dissemination of these guidelines and evaluating their impact on care efficiency in clinical practice