SUPPORTING DECISION MAKING FOR THE PREVENTION OF CHILD MALTREATMENT IN NORTH CAROLINA

Child maltreatment is a distressingly prevalent problem in the United States, with over 674,000 children estimated to be witness to domestic violence or otherwise affected by abuse or neglect in federal fiscal year 2017. While evidence-based programs exist to prevent child maltreatment, only a small proportion of families receive such services. Tools are needed to support decision makers when they are assessing their local context and selecting discrete evidence-based programs to reduce child maltreatment. This research addresses three aims in order to support such decision making in North Carolina (NC): 1) To understand how county-level indicators of child and family well-being co-vary using data from the U.S. Census and RWJF County Health Rankings; 2) To collaboratively develop a systems informed hypothesis of child maltreatment risk and protective factors using a Group Model Building (GMB) approach with NC stakeholders, and structure an early quantitative system dynamics simulation model to compare the potential effects of three evidence-based child maltreatment prevention programs, and 3) To develop and pilot test a multi-criteria decision analysis (MCDA) tool to assess whether interventions are differentially ranked with a manual ranking compared to ranks calculated with the tool. In Aim 1, we find that latent profiles of North Carolina counties can be characterized by low, moderate, and high risk, but the moderate risk profile is also associated with the highest level of predicted drug overdose deaths and with highest mean of predicted child maltreatment reports. In Aim 2, stakeholders emphasized the role of parental trauma and access to peer supports, and the simulation model offered preliminary insights into the importance of system shocks such as newborns. In Aim 3, over half of decision makers (55%) ranked the three interventions differently with their manual ranking compared to rankings calculated with the MCDA tool. The results of this research suggest that stakeholders conceptualize of child maltreatment risk factors in a multi-level, interconnected manner, and that decision support tools such as the ones presented here can aid with facilitating, not replacing, community conversations around how best to address child maltreatment within the local context.Doctor of Philosoph

Increasing the Delivery of Preventive Health Services in Public Education

The delivery of prevention services to children and adolescents through traditional healthcare settings is challenging for a variety of reasons. Parent- and community-focused services are typically not reimbursable in traditional medical settings, and personal healthcare services are often designed for acute and chronic medical treatment rather than prevention. To provide preventive services in a setting that reaches the widest population, those interested in public health and prevention often turn to school settings. This paper proposes that an equitable, efficient manner in which to promote health across the life course is to integrate efforts from public health, primary care, and public education through the delivery of preventive healthcare services, in particular, in the education system. Such an integration of systems will require a concerted effort on the part of various stakeholders, as well as a shared vision to promote child health via community and institutional stakeholder partnerships. This paper includes (1) examination of some key system features necessary for delivery of preventive services that improve child outcomes; (2) a review of the features of some common models of school health services for their relevance to prevention services; and (3) policy and implementation strategy recommendations to further the delivery of preventive services in schools. These recommendations include the development of common metrics for health outcomes reporting, facilitated data sharing of these metrics, shared organization incentives for integration, and improved reimbursement and funding opportunities

A scoping review of the use of ethnographic approaches in implementation research and recommendations for reporting

Background: Researchers have argued for the value of ethnographic approaches to implementation science (IS). The contested meanings of ethnography pose challenges and possibilities to its use in IS. The goal of this study was to identify sources of commonality and variation, and to distill a set of recommendations for reporting ethnographic approaches in IS. Methods: We included in our scoping review English-language academic journal articles meeting two criteria: (1) IS articles in the healthcare field and (2) articles that described their approach as ethnographic. In March 2019, we implemented our search criteria in four academic databases and one academic journal. Abstracts were screened for inclusion by at least two authors. We iteratively develop a codebook for full-text analysis and double-coded included articles. We summarized the findings and developed reporting recommendations through discussion. Results: Of the 210 articles whose abstracts were screened, 73 were included in full-text analysis. The number of articles increased in recent years. Ethnographic approaches were used within a wide variety of theoretical approaches and research designs. Articles primarily described using interviews and observational methods as part of their ethnographic approaches, though numerous other methods were also employed. The most cited rationales for using ethnographic approaches were to capture context-specific phenomena, understand insiders? perspective, and study complex interactions. In reporting on ethnographic approaches, we recommend that researchers provide information on researcher training and position, reflect on researchers? positionality, describe observational methods in detail, and report results from all the methods used. Conclusion: The number of IS studies using ethnography has increased in recent years. Ethnography holds great potential for contributing further to IS, particularly to studying implementation strategy mechanisms and understanding complex adaptive systems. Plain language summary: Researchers have proposed that ethnographic methods may be valuable to implementation research and practice. Ethnographic approaches have their roots in the field of anthropology, but they are now used in many fields. These approaches often involve a researcher spending time in 'real-world' settings, conducting interviews and observation to understand a group of people. That said, researchers disagree on the meaning of ethnography, which presents a challenge to its use in implementation science (IS). We searched for articles in the field of IS that described their methods as ethnographic. We then reviewed the articles, looking for similarities and differences in how and why ethnographic approaches were used. Many of these articles said they used ethnographic methods because they were interested in issues like context, research participants? views, and complex interactions. We found a large amount of variation in how ethnographic methods were used. We developed recommendations for describing ethnographic methods in a way that readers can clearly understand. We also made several observations of the value ethnographic approaches can bring to IS. Ethnographic methods may be especially useful to studying unplanned and unexpected changes that take place during implementation. These recommendations and observations could be helpful to implementation researchers wishing to use ethnographic methods

An Overview of Research and Evaluation Designs for Dissemination and Implementation

The wide variety of dissemination and implementation designs now being used to evaluate and improve health systems and outcomes warrants review of the scope, features, and limitations of these designs

Whos in the room where it happens? A taxonomy and five-step methodology for identifying and characterizing policy actors.

BACKGROUND: Engaging policy actors in research design and execution is critical to increasing the practical relevance and real-world impact of policy-focused dissemination and implementation science. Identifying and selecting which policy actors to engage, particularly actors involved in Big P public policies such as laws, is distinct from traditional engaged research methods. This current study aimed to develop a transparent, structured method for iteratively identifying policy actors involved in key policy decisions-such as adopting evidence-based interventions at systems-scale-and to guide implementation study sampling and engagement approaches. A flexible policy actor taxonomy was developed to supplement existing methods and help identify policy developers, disseminators, implementers, enforcers, and influencers. METHODS: A five-step methodology for identifying policy actors to potentially engage in policy dissemination and implementation research was developed. Leveraging a recent federal policy as a case study-The Family First Prevention Services Act (FFPSA)-publicly available documentation (e.g., websites, reports) were searched, retrieved, and coded using content analysis to characterize the organizations and individual policy actors in the room during policy decisions. RESULTS: The five steps are as follows: (1) clarify the policy implementation phase(s) of interest, (2) identify relevant proverbial or actual policymaking rooms, (3) identify and characterize organizations in the room, (4) identify and characterize policy actors in the room, and (5) quantify (e.g., count actors across groups), summarize, and compare rooms to develop or select engagement approaches aligned with the room and actors. The use and outcomes of each step are exemplified through the FFPSA case study. CONCLUSIONS: The pragmatic and transparent policy actor identification steps presented here can guide researchers methods for continuous sampling and successful policy actor engagement. Future work should explore the utility of the proposed methods for guiding selection and tailoring of engagement and implementation strategies (e.g., research-policy actor partnerships) to improve both Big P and little p (administrative guidelines, procedures) policymaking and implementation in global contexts

Who’s “in the room where it happens”? A taxonomy and five-step methodology for identifying and characterizing policy actors

Abstract Background Engaging policy actors in research design and execution is critical to increasing the practical relevance and real-world impact of policy-focused dissemination and implementation science. Identifying and selecting which policy actors to engage, particularly actors involved in “Big P” public policies such as laws, is distinct from traditional engaged research methods. This current study aimed to develop a transparent, structured method for iteratively identifying policy actors involved in key policy decisions—such as adopting evidence-based interventions at systems-scale—and to guide implementation study sampling and engagement approaches. A flexible policy actor taxonomy was developed to supplement existing methods and help identify policy developers, disseminators, implementers, enforcers, and influencers. Methods A five-step methodology for identifying policy actors to potentially engage in policy dissemination and implementation research was developed. Leveraging a recent federal policy as a case study—The Family First Prevention Services Act (FFPSA)—publicly available documentation (e.g., websites, reports) were searched, retrieved, and coded using content analysis to characterize the organizations and individual policy actors in the “room” during policy decisions. Results The five steps are as follows: (1) clarify the policy implementation phase(s) of interest, (2) identify relevant proverbial or actual policymaking “rooms,” (3) identify and characterize organizations in the room, (4) identify and characterize policy actors in the “room,” and (5) quantify (e.g., count actors across groups), summarize, and compare “rooms” to develop or select engagement approaches aligned with the “room” and actors. The use and outcomes of each step are exemplified through the FFPSA case study. Conclusions The pragmatic and transparent policy actor identification steps presented here can guide researchers’ methods for continuous sampling and successful policy actor engagement. Future work should explore the utility of the proposed methods for guiding selection and tailoring of engagement and implementation strategies (e.g., research-policy actor partnerships) to improve both “Big P” and “little p” (administrative guidelines, procedures) policymaking and implementation in global contexts

A structured approach to applying systems analysis methods for examining implementation mechanisms

Abstract Background It is challenging to identify and understand the specific mechanisms through which an implementation strategy affects implementation outcomes, as implementation happens in the context of complex, multi-level systems. These systems and the mechanisms within each level have their own dynamic environments that change frequently. For instance, sequencing may matter in that a mechanism may only be activated indirectly by a strategy through another mechanism. The dosage or strength of a mechanism may vary over time or across different health care system levels. To elucidate the mechanisms relevant to successful implementation amidst this complexity, systems analysis methods are needed to model and manage complexity. Methods The fields of systems engineering and systems science offer methods—which we refer to as systems analysis methods—to help explain the interdependent relationships between and within systems, as well as dynamic changes to systems over time. When applied to studying implementation mechanisms, systems analysis methods can help (i) better identify and manage unknown conditions that may or may not activate mechanisms (both expected mechanisms targeted by a strategy and unexpected mechanisms that the methods help detect) and (ii) flexibly guide strategy adaptations to address contextual influences that emerge after the strategy is selected and used. Results In this paper, we delineate a structured approach to applying systems analysis methods for examining implementation mechanisms. The approach includes explicit steps for selecting, tailoring, and evaluating an implementation strategy regarding the mechanisms that the strategy is initially hypothesized to activate, as well as additional mechanisms that are identified through the steps. We illustrate the approach using a case example. We then discuss the strengths and limitations of this approach, as well as when these steps might be most appropriate, and suggest work to further the contributions of systems analysis methods to implementation mechanisms research. Conclusions Our approach to applying systems analysis methods can encourage more mechanisms research efforts to consider these methods and in turn fuel both (i) rigorous comparisons of these methods to alternative mechanisms research approaches and (ii) an active discourse across the field to better delineate when these methods are appropriate for advancing mechanisms-related knowledge

Preventing Youth Internalizing Symptoms through the Familias Unidas Intervention: Examining Variation in Response

Prevention programs that strengthen parenting and family functioning have been found to reduce poor behavioral outcomes in adolescents, including substance use, HIV risk, externalizing and internalizing problems. However, there is evidence that not all youth benefit similarly from these programs. Familias Unidas is a family-focused intervention designed to prevent substance use and sexual risk among Hispanic youth, and has recently demonstrated unanticipated reductions in internalizing symptoms for some youth. This paper examines variation in intervention response for internalizing symptoms using individual-level data pooled across four distinct Familias Unidas trials: 1) 266 eighth grade students recruited from the general school population; 2) 160 ninth grade students from the general school population; 3) 213 adolescents with conduct, aggression and/or attention problems; and 4) 242 adolescents with a delinquency history. Causal inference growth mixture modeling suggests a three-class model. The two largest classes represent youth with low (60%) and medium (27%) internalizing symptoms at baseline and both intervention and control participants show reductions in internalizing symptoms. The third class (13%) represents youth with high levels of baseline internalizing symptoms who remain at steady levels of internalizing symptoms when exposed to the intervention, but who experience an increase in symptoms under the control condition. Female gender, low baseline levels of parent-adolescent communication, and older age were associated with membership in the high-risk class. These synthesis analyses involving a large sample of youth with varying initial risk levels represent a further step toward strengthening our knowledge of preventive intervention response, and improving preventive interventions

Blending Qualitative and Computational Linguistics Methods for Fidelity Assessment: Experience with the Familias Unidas Preventive Intervention

Careful fidelity monitoring and feedback are critical to implementing effective interventions. A wide range of procedures exist to assess fidelity; most are derived from observational assessments ( Schoenwald et al, 2013 ). However, these fidelity measures are resource intensive for research teams in efficacy/effectiveness trials, and are often unattainable or unmanageable for the host organization to rate when the program is implemented on a large scale. We present a first step towards automated processing of linguistic patterns in fidelity monitoring of a behavioral intervention using an innovative mixed methods approach to fidelity assessment that uses rule-based, computational linguistics to overcome major resource burdens. Data come from an effectiveness trial of the Familias Unidas intervention, an evidence-based, family-centered preventive intervention found to be efficacious in reducing conduct problems, substance use and HIV sexual risk behaviors among Hispanic youth. This computational approach focuses on “joining,” which measures the quality of the working alliance of the facilitator with the family. Quantitative assessments of reliability are provided. Kappa scores between a human rater and a machine rater for the new method for measuring joining reached .83. Early findings suggest that this approach can reduce the high cost of fidelity measurement and the time delay between fidelity assessment and feedback to facilitators; it also has the potential for improving the quality of intervention fidelity ratings

Automatic classification of communication logs into implementation stages via text analysis

To improve the quality, quantity, and speed of implementation, careful monitoring of the implementation process is required. However, some health organizations have such limited capacity to collect, organize, and synthesize information relevant to its decision to implement an evidence-based program, the preparation steps necessary for successful program adoption, the fidelity of program delivery, and the sustainment of this program over time. When a large health system implements an evidence-based program across multiple sites, a trained intermediary or broker may provide such monitoring and feedback, but this task is labor intensive and not easily scaled up for large numbers of sites. We present a novel approach to producing an automated system of monitoring implementation stage entrances and exits based on a computational analysis of communication log notes generated by implementation brokers. Potentially discriminating keywords are identified using the definitions of the stages and experts' coding of a portion of the log notes. A machine learning algorithm produces a decision rule to classify remaining, unclassified log notes. We applied this procedure to log notes in the implementation trial of multidimensional treatment foster care in the California 40-county implementation trial (CAL-40) project, using the stages of implementation completion (SIC) measure. We found that a semi-supervised non-negative matrix factorization method accurately identified most stage transitions. Another computational model was built for determining the start and the end of each stage. This automated system demonstrated feasibility in this proof of concept challenge. We provide suggestions on how such a system can be used to improve the speed, quality, quantity, and sustainment of implementation. The innovative methods presented here are not intended to replace the expertise and judgement of an expert rater already in place. Rather, these can be used when human monitoring and feedback is too expensive to use or maintain. These methods rely on digitized text that already exists or can be collected with minimal to no intrusiveness and can signal when additional attention or remediation is required during implementation. Thus, resources can be allocated according to need rather than universally applied, or worse, not applied at all due to their cost