Monitoring implementation in program evaluation with direct audio coding

This project explored the reliability and utility of transcription in coding qualitative data across two studies in a program evaluation context. The first study tested the method of direct audio coding, or coding audio files without transcripts, using qualitative data software. The presence and frequency of codes applied in direct audio coding and traditional transcription coding were compared and the two methods produced similar results. Direct audio coding was then employed in an evaluation study to monitor implementation and the method and to be reliable. Implications are discussed with considerations for both researchers and practitioners

Monitoring implementation in program evaluation with direct audio coding

This project explored the reliability and utility of transcription in coding qualitative data across two studies in a program evaluation context. The first study tested the method of direct audio coding, or coding audio files without transcripts, using qualitative data software. The presence and frequency of codes applied in direct audio coding and traditional transcription coding were compared and the two methods produced similar results. Direct audio coding was then employed in an evaluation study to monitor implementation and the method and to be reliable. Implications are discussed with considerations for both researchers and practitioners

Using Caregiver Strain to Predict Participation in a Peer-Support Intervention for Parents of Children With Emotional or Behavioral Needs

Children receiving services for severe emotional and behavioral difficulties are less likely to have parents who are involved in their education and support services. Peer-to-peer family support programs are one approach to increasing the self-efficacy and empowerment of parents’ engagement in the treatment of a child’s mental health conditions. Furthermore, programs providing parental support may reduce the strain and negative consequences caregivers may experience due to the stress of caring for a child with emotional and behavioral needs. Although much is known about the relation between caregivers’ strain and children’s use of mental health services, less is known about caregiver strain and parents’ participation in family support programs. This study evaluated whether caregiver strain predicted parents’ (N = 52) participation in a phone-based, peer-to-peer support intervention. Results of the regression analysis indicated that highly strained parents participated in four to seven more phone conversations over the course of intervention, which occurred across the academic year. Therefore, findings have implications for the school and mental health providers aiming to increase the involvement of parents of children with emotional and behavioral disorders

Convergent Validity of the Strength-Based Behavioral and Emotional Rating Scale with Youth in a Residential Setting

Strength-based assessment has been identified as an appropriate approach to use in planning treatment and evaluating outcomes of youth in residential settings. In previous research the Behavioral and Emotional Rating Scale-2, a standardized and norm-referenced strength-based measure, has demonstrated adequate reliability and validity with youth served in community and educational settings. The purpose of the present study was to examine the internal reliability and convergent validity of the BERS-2 by comparing the test to the Child Behavior Checklist (CBCL) and the Symptoms and Functioning Severity Scale (SFSS). The results indicate that the scores from the BERS-2 are internally consistent and converge with other behavioral and emotional measures which, taken together, suggest that the BERS-2 could be acceptable for assessing the emotional and behavioral strengths of youth in residential settings. Study limitations and future research directions are identified

Convergent Validity of the Strength-Based Behavioral and Emotional Rating Scale with Youth in a Residential Setting

Strength-based assessment has been identified as an appropriate approach to use in planning treatment and evaluating outcomes of youth in residential settings. In previous research the Behavioral and Emotional Rating Scale-2, a standardized and norm-referenced strength-based measure, has demonstrated adequate reliability and validity with youth served in community and educational settings. The purpose of the present study was to examine the internal reliability and convergent validity of the BERS-2 by comparing the test to the Child Behavior Checklist (CBCL) and the Symptoms and Functioning Severity Scale (SFSS). The results indicate that the scores from the BERS-2 are internally consistent and converge with other behavioral and emotional measures which, taken together, suggest that the BERS-2 could be acceptable for assessing the emotional and behavioral strengths of youth in residential settings. Study limitations and future research directions are identified

Examining Change in Therapeutic Alliance to Predict Youth Mental Health Outcomes

Objective—To examine the link between therapeutic alliance and youth outcomes. Method—The study was conducted at a group-home with 112 youth with a disruptive-behavior diagnosis. Therapeutic alliance was collected routinely via youth and staff report. Outcome data were collected using youth and staff reports of externalizing behavior as well as behavioral incidents occurring during care. Outcome data were collected following intake into services and at 6 and 12 months of care. Data were analyzed to examine (1) if youth behavior problems at intake were predictive of therapeutic alliance and (2) if changes in alliance were predictive of subsequent youth outcomes. These were conducted with a 6-month service-delivery model and replicated with a 12-month model. Results—There was some support for the first hypothesis, that initial levels of youth externalizing behavior would be related to alliance ratings; however, most of the effects were marginally significant. The second hypothesis, that changes in therapeutic alliance would be related to subsequent youth outcomes, was supported for the 6-month model, but not the 12-month model. Conclusions—Changes in therapeutic alliance may be predictive of youth outcomes during care. Additional research into examining therapeutic alliance trajectories is warranted to improve mental health services for youth

Therapeutic Alliance Between Youth and Staff in Residential Group Care: Psychometrics of the Therapeutic Alliance Quality Scale

Therapeutic alliance has been frequently studied in individual counseling sessions; however, research on therapeutic alliance in residential settings for youth with mental health diagnoses has been limited. This may be due, in part, to the presence of multiple service providers often in caregiving roles. The purpose of this study was to examine the psychometric quality of a widely utilized measure of therapeutic alliance used in psychotherapy with youth in residential care where the treatment is provided by a trained married couple. We also compared the relationship between youth ratings of their male and female service provider, as well as examined correlations in ratings between youth and staff on therapeutic alliance. Finally, we investigated the direction, magnitude, and trajectory of change in therapeutic alliance over a 12-month period following admission into residential care. The method was a longitudinal assessment of 135 youth and 124 staff regarding therapeutic alliance over the course of 12 months or discharge from services. Results indicated strong psychometric properties and high correlations for youth ratings of both their male and female service providers. However, the correlation was low between youth and service provider ratings of alliance. Longitudinal analyses indicated that rates of therapeutic alliance changed over time

Validation of the Symptoms and Functioning Severity Scale in Residential Group Care

Tests that measure the emotional and behavioral problems of children and youth are typically not normed and standardized on youth diagnosed with disruptive behavior, particularly those youth in residential care. Yet professional standards mandate that before instruments are used with a specific population the psychometric properties need to be studied and re-established: specifically, psychometric properties, including validity, need to be evaluated (AERA, APA, & NCME, 1999). The purpose of the present study was to assess the validity characteristics of the Symptoms and Functioning Severity Scale (SFSS; Bickman, et al., 2010), a widely used test developed for use in outpatient clinics, with youth in a residential care program. The convergent validity of the SFSS was established with the large correlations (.78-.86) with the CBCL. Several binary classification analyses including specificity, area under the receiver operating characteristic curve, positive and negative likelihood ratios, and the Youden Index supported the validity of the SFSS. However, the sensitivity index was somewhat low indicating the test may produce a high level of false negatives. Limitations, future research and implications are discussed

Measurement Properties of the Motivation for Youth Treatment Scale with a Residential Group Home Population

A client’s motivation to receive services has long been identified as a highly relevant component of mental health treatment. In fact, ample evidence demonstrates that client motivation is significantly related to seeking services, remaining in services, and improved client outcomes (e.g., Broome, Joe, & Simpson, 2001; Ryan, Plant, & O’Malley, 1995; Schroder, Sellman, Frampton, & Deering, 2009). Additionally, it has been recognized that motivation is a “dynamic” characteristic that changes throughout treatment (Melnick, De Leon, Hawke, Jainchill, & Kressel, 1997; Schroder et al., 2009). In this way, motivation is an important client factor to assess and monitor throughout the treatment process. The broad construct of motivation is comprised of two separate, but related components conceptualized as motivation to change and motivation for treatment. As defined by DiClamente, Schlundt, and Gemmell (2004), motivation to change refers to a willingness to recognize problematic behavior and take steps toward change, whereas motivation for treatment refers to a willingness to seek help and remain compliant with an intervention program. In other words, a motivated person not only perceives the importance of changing, but also has confidence that they are able to be successful at making the change (Burke, Arkowitz, & Menchola, 2003)

The Child Outcomes of a Behavior Model

Within 3-tier behavioral models, universal interventions are expected to prevent the onset of problem behavior in a majority of children altogether and to sustain improvements in child outcomes by the selected and indicated interventions. A cohort longitudinal design was used to assess the extent to which a 3-tier model achieves these expected outcomes. The respective universal, selected, and indicated interventions included Behavior and Academic Support and Enhancement. First Step to Success, and MultiSystemic Therapy. A total of 407 children in Grades K-3 from 1 of 4 longitudinal cohorts participated. The results of 2-level linear growth analyses indicate that the 3-tier behavior model achieved the anticipated outcomes with respect to social behavior. The results, limitations, and implications are discussed