Assessment of the Human Factors Analysis and Classification System (HFACS): Intra-rater and Inter-rater Reliability

Human error has been identified as the primary contributing cause for up to 80% of the accidents in complex, high risk systems such as aviation, oil and gas, mining and healthcare. Many models have been proposed to analyze these incidents and identify their causes, focusing on the human factor. One such safety model is the Human Factors Analysis and Classification System (HFACS), a comprehensive accident investigation and analysis tool which focuses not only on the act of the individual preceding the accident but on other contributing factors in the system as well. Since its development, HFACS has received substantial research attention; however, the literature on its reliability is limited. This study adds to past research by investigating the overall intra-rater and inter-rater reliability of HFACS in addition to the intra-rater and inter-rater reliability for each tier and category. For this investigation, 125 coders with similar HFACS training coded 95 causal factors extracted from actual incident/accident reports from several sectors. The overall intra-rater reliability was evaluated using percent agreement, Krippendorff\u27s Alpha, and Cohen\u27s Kappa, while the inter-rater was analyzed using percent agreement, Krippendorff\u27s Alpha, and Fleiss\u27 Kappa. Because of analytical limitations, only percent agreement and Krippendorff\u27s Alpha were used for the intra-rater evaluation at the individual tier and category level and Fleiss\u27 Kappa and Krippendorff\u27s Alpha, for the corresponding inter-rater evaluation. The overall intra-rater and inter-rater results for the tier level and the individual HFACS tiers achieved acceptable reliability levels with respect to all agreement coefficients. Although the overall intra-rater and inter-rater reliability results at the category level were lower than the tier level, both types of reliabilities achieved acceptable levels with inter-rater reliability being lower than intra-rater. In addition, the intra-rater and inter-rater results for the individual HFACS categories varied from achieving low reliability levels to being acceptable. Both the inter-rater and intra-rater results found that the same 5 categories among the 19 - Skill Based Error, Decision Error, Inadequate Supervision, Planned Inappropriate Operations, and Supervisory Violation - were lower than the required minimum reliability threshold. While the overall findings suggest that HFACS is reasonably reliable, the fact that there were 5 categories with low reliability levels requires further research on ways and methods to improve its reliability. One such method could be to focus on training by designing and developing a standard HFACS training program that improves its reliability, which will have the potential to enhance both the confidence in using it as an accident analysis tool and the effectiveness of the safety plans and strategies based on it

Advancing Intercultural Communication Skills in Diverse Teams: An Intervention Program for Project-Based Engineering Courses

Studies show engineering graduates lack critical skills, including the ability to communicate effectively and appropriately in diverse teams. As institutions seek effective and affordable solutions to meet ABET accreditation standards for non-technical skills, we report on an intervention program designed to advance learners’ intercultural communication (IComm) skills. We adapted the UNESCO Story Circles methodology (2020) to undergraduate engineering contexts: discipline-specific prompts guide oral exchanges among students, tasking learners to practice IComm principles in teams. The 2021 mixedmethod study tracked 31 students’ attitudes toward teamwork and performance in project-based learning. Data sources included three survey instruments, individual project grades, and end-of-semester course evaluations. Results show strong student support for the intervention program across demographics and instructional modalities; improved learner attitudes toward teamwork in post-intervention surveys; improved 2021 end-of-semester course evaluations compared to 2019; and, higher individual grades on the team project. Findings support the use of our intervention program in project-based engineering courses and highlight the importance of guiding engineering students in the intentional practice of IComm principles

Perceptions of Risk, Work, and Lifestyle Changes on Mental Health of Healthcare Workers Amidst the Covid-19 Pandemic

The COVID-19 outbreak is significantly affecting the mental health of healthcare workers worldwide. This study aims to investigate mental health outcomes of healthcare workers in a health system located in the Southeastern US during the first peak of the pandemic and examine the association of specific factors on the mental well-being of healthcare workers. A cross-sectional survey of 388 healthcare workers was conducted. Data were collected using a 79-item questionnaire, which included the Patient Health Questionnaire (PHQ-9) instrument, the 7-item Generalized Anxiety Disorder (GAD-7) instrument, and the 22-item Impact of Event Scale-Revised (IES-R), to assess symptoms of depression, anxiety, and general distress, respectively. Data were analyzed using descriptive, bivariate, and multivariate statistics. 30.1%, 28.7%, and 39.4% of respondents reported depression, anxiety, and distress symptoms, respectively. Younger workers and females reported higher mental symptomologies. We identified significant, nontraditional factors associated with depression and anxiety symptoms among healthcare workers: healthcare procedure change, concern of exposing family to COVID-19, number of missed shifts, and access to psychological resources/services. These findings emphasize the importance of providing the proper training to reduce concerns of exposing family members and psychological interventions to promote mental health well-being for healthcare workers during the stressful COVID-19 pandemic

Rapid design and implementation of a UVC decontamination room

With the recent COVID-19 pandemic that has swept the world and the nation, hospitals around the country have experienced shortages in Personal Protective Equipment, specifically N95 filter face-mask respirators (FFRs). This has created the need for facilities to develop sterilization processes to enable reuse of face masks by the health care personnel. Among the various methods of sterilization, UVC light exposure is the easiest to implement given the factors of time, safety, and availability. Face masks and/or other PPE are exposed to UVC light for a specified time to kill any viruses or bacteria that may reside on the surfaces of the masks. A collaborative effort was formed in April of 2020 between Wellstar Health System and Kennesaw State University to (1) setup an appropriate sterilization room at a Wellstar hospital (2) develop the procedural guidelines necessary to ensure quality control and (3) assess employees’ perceptions of the N95 FFR decontamination process and efficacy. This paper will first describe the methodology used to validate the layout of the room, which consists of a rudimentary analytical analysis of the UVC photon intensity from bulb-to-mask, computer simulations to determine the lighting power density throughout the room, and experimental measurements to confirm the appropriate energy deposition. This paper will then document the procedures for handling and processing the pre- and post-sterilized masks followed by employee survey findings. It is the hope of the authors that this paper will serve to provide a generic blueprint for hospitals and other organizations to follow if a future need arises for rapid UVC decontamination

Evaluation of a Patient-Centered Fall-Prevention Tool Kit to Reduce Falls and Injuries: A Nonrandomized Controlled Trial

Importance: Falls represent a leading cause of preventable injury in hospitals and a frequently reported serious adverse event. Hospitalization is associated with an increased risk for falls and serious injuries including hip fractures, subdural hematomas, or even death. Multifactorial strategies have been shown to reduce falls in acute care hospitals, but evidence for fall-related injury prevention in hospitals is lacking. Objective: To assess whether a fall-prevention tool kit that engages patients and families in the fall-prevention process throughout hospitalization is associated with reduced falls and injurious falls. Design, Setting, and Participants: This nonrandomized controlled trial using stepped wedge design was conducted between November 1, 2015, and October 31, 2018, in 14 medical units within 3 academic medical centers in Boston and New York City. All adult inpatients hospitalized in participating units were included in the analysis. Interventions: A nurse-led fall-prevention tool kit linking evidence-based preventive interventions to patient-specific fall risk factors and designed to integrate continuous patient and family engagement in the fall-prevention process. Main Outcomes and Measures: The primary outcome was the rate of patient falls per 1000 patient-days in targeted units during the study period. The secondary outcome was the rate of falls with injury per 1000 patient-days. Results: During the interrupted time series, 37 231 patients were evaluated, including 17 948 before the intervention (mean [SD] age, 60.56 [18.30] years; 9723 [54.17%] women) and 19 283 after the intervention (mean [SD] age, 60.92 [18.10] years; 10 325 [53.54%] women). There was an overall adjusted 15% reduction in falls after implementation of the fall-prevention tool kit compared with before implementation (2.92 vs 2.49 falls per 1000 patient-days [95% CI, 2.06-3.00 falls per 1000 patient-days]; adjusted rate ratio 0.85; 95% CI, 0.75-0.96; P = .01) and an adjusted 34% reduction in injurious falls (0.73 vs 0.48 injurious falls per 1000 patient-days [95% CI, 0.34-0.70 injurious falls per 1000 patient-days]; adjusted rate ratio, 0.66; 95% CI, 0.53-0.88; P = .003). Conclusions and Relevance: In this nonrandomized controlled trial, implementation of a fall-prevention tool kit was associated with a significant reduction in falls and related injuries. A patient-care team partnership appears to be beneficial for prevention of falls and fall-related injuries. Trial Registration: ClinicalTrials.gov Identifier: NCT02969343