Development and Psychometric Evaluation of the Assessment of Core CBT Skills (ACCS): An Observation Based Tool for Assessing Cognitive Behavioural Therapy Competence

This paper outlines the development and psychometric evaluation of the Assessment of Core CBT Skills (ACCS) rating scale. The ACCS aims to provide a novel assessment framework to deliver formative and summative feedback regarding therapists’ performance within observed cognitive-behavioural treatment sessions, and for therapists to rate and reflect on their own performance. Findings from three studies are outlined: 1) a feedback study (N = 66) examining content validity, face validity and usability, 2) a focus group (N = 9) evaluating usability and utility, and 3) an evaluation of the psychometric properties of the ACCS in ‘real world’ CBT training and routine clinical practice contexts. Results suggest that the ACCS has good face validity, content validity, and usability and provides a user-friendly tool that is useful for promoting self-reflection and providing formative feedback. Scores on both the self and assessor-rated versions of the ACCS demonstrate good internal consistency, inter-rater reliability, and discriminant validity. In addition, ACCS scores were found to be correlated with, but distinct from the Revised Cognitive Therapy Scale (CTS-R) and were comparable to CTS-R scores in terms of internal consistency and discriminant validity. Additionally, the ACCS may have advantages over the CTS-R in terms of inter-rater reliability of scores. The studies also provided insight into areas for refinement and a number of modifications were undertaken to improve the scale. In summary, the ACCS is an appropriate and useful measure of CBT competence that can be used to promote self-reflection and provide therapists with formative and summative feedback

Is Supervision Necessary? Examining the Effects of Internet-Based CBT Training With and Without Supervision

Objective: To investigate the effect of Internet-based training (IBT), with and without supervision, on therapists? (N = 61) cognitive behavioral therapy (CBT) skills in routine clinical practice. Method: Participants were randomized into three conditions: (1) Internet-based training with use of a consultation worksheet (IBT-CW), (2) Internet-based training with CBT supervision via Skype (IBT-S), and (3) ?delayed-training? control who did not receive the training until all data collection was completed (DT). The IBT conditions received access to training over a period of three months. CBT skills were evaluated at pre-, mid- and post- training / wait using assessor competence ratings of recorded therapy sessions. Results: Hierarchical linear analysis revealed that the IBT-S condition had significantly greater CBT competence at post training than IBT-CW and DT conditions at both the mid- and post-training/ wait assessment points. There were no significant differences between IBT-CW and the delayed (no)-training control (DT) conditions. Conclusions: IBT programs that include supervision may be a scalable and effective method of disseminating CBT into routine clinical practice, particularly for populations without ready access to more traditional ?live? methods of training. There was no evidence for a significant effect of IBT without supervision over a non-training control, suggesting that merely providing access to IBT programs may not be an effective method of disseminating CBT to routine clinical practice. Public Health Significance Statement: This study highlights the importance of providing supervision during an Internet-based CBT training program, particularly when therapists are applying newly acquired skills in routine clinical practice

Learning How to Ask - Does a one-day training increase trauma inquiry in routine substance use disorder practice? Results of a cluster-randomized controlled trial

Aims To examine the effectiveness of a one-day skills training program for increasing trauma inquiry in routine substance use disorder treatment. Design Cluster-randomized two-armed controlled trial, with 12 substance use disorder (SUD) organizations operating 25 counseling centers, randomly assigned to training in trauma inquiry (13 counseling centers of 8 SUD organizations) or no training (12 counseling centers of 4 SUD organizations). Setting SUD counseling centers in Northern Germany. Cases N = 5204 SUD counseling services. Intervention The professionals assigned to the intervention group received a one-day training in trauma inquiry plus a 1.5-hour refresher session 3 months later. Professionals in the control group received no training. Measures Over a 12-month period, professionals documented for each counseling service whether they asked the client about four traumatic events: physical abuse, emotional abuse, sexual abuse and neglect. Analysis Primary outcomes were rates of asking about physical abuse, sexual abuse, emotional abuse and neglect in the 6 months after training. These were compared across conditions, while adjusting for baseline probabilities in the 6 months before the intervention, using mixed-effects logistic regression. Findings In the 6 months after training, the rate of asking about physical abuse was 18% higher in the SUD counseling services of trained professionals, relative to services of untrained professionals (OR = 1.18, 95% CI = [1.01–1.37, p = .035]). No effect was found for asking about sexual abuse, emotional abuse and neglect. Conclusion A one-day training program in trauma inquiry, combined with a brief refresher session, was effective in increasing inquiries about physical abuse in routine counseling practice. The training was ineffective in increasing inquiries about sexual abuse, emotional abuse and neglect. The effectiveness of a one-day training of trauma inquiry might be increased by a longer training, or by combining it with additional elements, such as ongoing supervision

Psychological therapies for women who experience intimate partner violence

BACKGROUND: Intimate partner violence (IPV) against women is prevalent and strongly associated with mental health problems. Women experiencing IPV attend health services frequently for mental health problems. The World Health Organization recommends that women who have experienced IPV and have a mental health diagnosis should receive evidence-based mental health treatments. However, it is not known if psychological therapies work for women in the context of IPV and whether they cause harm. OBJECTIVES: To assess the effectiveness of psychological therapies for women who experience IPV on the primary outcomes of depression, self-efficacy and an indicator of harm (dropouts) at six- to 12-months' follow-up, and on secondary outcomes of other mental health symptoms, anxiety, quality of life, re-exposure to IPV, safety planning and behaviours, use of healthcare and IPV services, and social support. SEARCH METHODS: We searched the Cochrane Common Mental Disorders Controlled Trials Register (CCMDCTR), CENTRAL, MEDLINE, Embase, CINAHL, PsycINFO, and three other databases, to the end of October 2019. We also searched international trials registries to identify unpublished or ongoing trials and handsearched selected journals, reference lists of included trials and grey literature. SELECTION CRITERIA: We included randomised controlled trials (RCTs), quasi-RCTs, cluster-RCTs and cross-over trials of psychological therapies with women aged 16 years and older who self-reported recent or lifetime experience of IPV. We included trials if women also experienced co-existing mental health diagnoses or substance abuse issues, or both. Psychological therapies included a wide range of interventions that targeted cognition, motivation and behaviour compared with usual care, no treatment, delayed or minimal interventions. We classified psychological therapies according to Cochrane Common Mental Disorders's psychological therapies list. DATA COLLECTION AND ANALYSIS: Two review authors extracted data and undertook 'Risk of Bias' assessment. Treatment effects were compared between experimental and comparator interventions at short-term (up to six months post-baseline), medium-term (six to under 12 months, primary outcome time point), and long-term follow-up (12 months and above). We used standardised mean difference (SMD) for continuous and odds ratio (OR) for dichotomous outcomes, and used random-effects meta-analysis, due to high heterogeneity across trials. MAIN RESULTS: We included 33 psychological trials involving 5517 women randomly assigned to experimental (2798 women, 51%) and comparator interventions (2719 women, 49%). Psychological therapies included 11 integrative therapies, nine humanistic therapies, six cognitive behavioural therapy, four third-wave cognitive behavioural therapies and three other psychologically-orientated interventions. There were no trials classified as psychodynamic therapies. Most trials were from high-income countries (19 in USA, three in Iran, two each in Australia and Greece, and one trial each in China, India, Kenya, Nigeria, Pakistan, Spain and UK), among women recruited from healthcare, community, shelter or refuge settings, or a combination of any or all of these. Psychological therapies were mostly delivered face-to-face (28 trials), but varied by length of treatment (two to 50 sessions) and staff delivering therapies (social workers, nurses, psychologists, community health workers, family doctors, researchers). The average sample size was 82 women (14 to 479), aged 37 years on average, and 66% were unemployed. Half of the women were married or living with a partner and just over half of the participants had experienced IPV in the last 12 months (17 trials), 6% in the past two years (two trials) and 42% during their lifetime (14 trials). Whilst 20 trials (61%) described reliable low-risk random-sampling strategies, only 12 trials (36%) described reliable procedures to conceal the allocation of participant status. While 19 trials measured women's depression, only four trials measured depression as a continuous outcome at medium-term follow-up. These showed a probable beneficial effect of psychological therapies in reducing depression (SMD -0.24, 95% CI -0.47 to -0.01; four trials, 600 women; moderate-certainty evidence). However, for self-efficacy, there may be no evidence of a difference between groups (SMD -0.12, 95% CI -0.33 to 0.09; one trial with medium-term follow-up data, 346 women; low-certainty evidence). Further, there may be no difference between the number of women who dropped out from the experimental or comparator intervention groups, an indicator of no harm (OR 1.04, 95% CI 0.75 to 1.44; five trials with medium-term follow-up data, 840 women; low-certainty evidence). Although no trials reported adverse events from psychological therapies or participation in the trial, only one trial measured harm outcomes using a validated scale. For secondary outcomes, trials measured anxiety only at short-term follow-up, showing that psychological therapies may reduce anxiety symptoms (SMD -0.96, 95% CI -1.29 to -0.63; four trials, 158 women; low-certainty evidence). However, within medium-term follow-up, low-certainty evidence revealed that there may be no evidence between groups for the outcomes safety planning (SMD 0.04, 95% CI -0.18 to 0.25; one trial, 337 women), post-traumatic stress disorder (SMD -0.24, 95% CI -0.54 to 0.06; four trials, 484 women) or re-exposure to any form of IPV (SMD 0.03, 95% CI -0.14 to 0.2; two trials, 547 women). AUTHORS' CONCLUSIONS: There is evidence that for women who experience IPV, psychological therapies probably reduce depression and may reduce anxiety. However, we are uncertain whether psychological therapies improve other outcomes (self-efficacy, post-traumatic stress disorder, re-exposure to IPV, safety planning) and there are limited data on harm. Thus, while psychological therapies probably improve emotional health, it is unclear if women's ongoing needs for safety, support and holistic healing from complex trauma are addressed by this approach. There is a need for more interventions focused on trauma approaches and more rigorous trials (with consistent outcomes at similar follow-up time points), as we were unable to synthesise much of the research

Establishing evidence-based training of cognitive behavioural therapy treatments: empirical and theoretical guidance for dissemination

The need to provide effective Cognitive Behavioural Therapy (CBT) treatments in routine clinical practice necessitates effective methods of CBT training, which are scalable and can be disseminated in populations without ready access to specialist trainers or large scale funding. This thesis seeks to add theoretical and empirical guidance to the development of evidence-based CBT training. The first chapter reviews the current literature on effective CBT training, and suggests theoretical and methodological guidance for future research on evidence-based training practices. Cohort studies (Chapters 2 – 4) examine data from former trainees for guidance on accurate CBT competence measurement and effective CBT training components. The fifth and sixth chapters report randomised-controlled trials examining the effects of internet-based training (IBT) on therapists’ competence and patient outcomes (Chapter 6). Findings from the literature review (Chapter 1) and Chapter 2 suggested the need for training effectiveness trials using treatment trial methodology, which includes ‘blind’ observer ratings of therapist behaviour as the measure of competence. Chapter 3 found that extensive training led to significant improvement in competence for trainees with a range of individual baseline characteristics. Since such extensive training is not accessible to the majority of practicing therapists, Chapter 4 examined which elements of extensive training were perceived by trainees as having the greatest positive effect on competence; thus providing empirical guidance in planning more scalable forms of training, such as IBT. Findings from the first four chapters provided guidance in planning the two randomised controlled trials outlined in Chapters 5 and 6, resulting in rigorous empirical data showing significant positive effects of IBT on therapists’ competence and on their patients’ clinical outcomes. Finally, the results of this thesis are related to the framework for developing evidence-based training, which was suggested in Chapter 1 and suggestions for future research are made

Establishing evidence-based training in cognitive behavioral therapy: A review of current empirical findings and theoretical guidance

Cognitive behavior therapy's (CBT) demonstrated efficacy has prompted calls for its increased dissemination to routine clinical practice settings. For the widespread dissemination of CBT to be successful in achieving effects similar to the original efficacy trials, there must also be effective dissemination of CBT training practices. However, as yet, CBT training is not evidence-based. This review examines what can be learned from existing research into the efficacy and effectiveness of CBT training. Due to the paucity of research specifically investigating CBT training, CBT effectiveness and dissemination studies are also examined to glean information about potentially effective training practices. In order to draw conclusions about effective training practices, comparisons are drawn between studies according to the clinical outcomes that they achieved. Training approaches are compared according to dose and active training elements, and theoretical models of learning are applied to interpret the findings. The limitations of the existing literature are discussed, as well as recommendations for improving training research to meet the standards evident in treatment trials (e.g., random allocation, control conditions, self-report and blind assessment, and adherence monitoring). Finally, the process of developing efficacious CBT treatment protocols is offered as a template for developing evidence-based CBT training protocols

Establishing evidence-based training of cognitive behavioural therapy treatments : empirical and theoretical guidance for dissemination

The need to provide effective Cognitive Behavioural Therapy (CBT) treatments in routine clinical practice necessitates effective methods of CBT training, which are scalable and can be disseminated in populations without ready access to specialist trainers or large scale funding. This thesis seeks to add theoretical and empirical guidance to the development of evidence-based CBT training. The first chapter reviews the current literature on effective CBT training, and suggests theoretical and methodological guidance for future research on evidence-based training practices. Cohort studies (Chapters 2 – 4) examine data from former trainees for guidance on accurate CBT competence measurement and effective CBT training components. The fifth and sixth chapters report randomised-controlled trials examining the effects of internet-based training (IBT) on therapists’ competence and patient outcomes (Chapter 6). Findings from the literature review (Chapter 1) and Chapter 2 suggested the need for training effectiveness trials using treatment trial methodology, which includes ‘blind’ observer ratings of therapist behaviour as the measure of competence. Chapter 3 found that extensive training led to significant improvement in competence for trainees with a range of individual baseline characteristics. Since such extensive training is not accessible to the majority of practicing therapists, Chapter 4 examined which elements of extensive training were perceived by trainees as having the greatest positive effect on competence; thus providing empirical guidance in planning more scalable forms of training, such as IBT. Findings from the first four chapters provided guidance in planning the two randomised controlled trials outlined in Chapters 5 and 6, resulting in rigorous empirical data showing significant positive effects of IBT on therapists’ competence and on their patients’ clinical outcomes. Finally, the results of this thesis are related to the framework for developing evidence-based training, which was suggested in Chapter 1 and suggestions for future research are made.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Developing and evaluating valid, reliable and usable measures of assessing competence in Cognitive Behavioural Therapy

Tools for measuring competence in delivering Cognitive Behavioural Therapy (CBT) provide a means of assessing the training of new CBT therapists and ensuring the quality of treatment provision within routine practice, provide a framework for delivering formative feedback, promote ongoing self-reflection, and are essential to establishing treatment integrity in research trials. As such, identifying an optimal strategy for assessing the competence with which CBT is delivered is crucial to the continued progression of the field. However, research in this area has been somewhat limited to date. Thus, there are at present no evidence-based best practice guidelines outlining the way CBT competence should be assessed. Furthermore, many of the assessment measures currently available have been widely criticised, indicating a need for improved tools for assessing CBT competence. To begin addressing this issue, the first two chapters of this thesis focus on reviewing and evaluating current assessment methods. Chapter one provides a systematic review of current methods of assessing CBT competence and chapter two outlines a qualitative exploration of experts’ understandings and experiences of assessing CBT competence. Findings from these studies provide tentative recommendations for practitioners and researchers assessing CBT competence. These initial studies also highlight ways in which the assessment of CBT competence could be improved and therefore provide a platform for guiding subsequent thesis chapters which focus on further developing existing assessment measures. Specifically, chapters three to six focus on the development and evaluation of a novel CBT competence rating scale: the Assessment of Core CBT Skills (ACCS). The ACCS builds upon currently available scales (especially the Cognitive Therapy Scale- Revised: CTS-R) to provide an assessment framework for assessors to deliver formative and summative feedback regarding therapists’ performance within observed CBT treatment sessions and for therapists to rate and reflect on their own performance. Development of the ACCS involved three key stages: 1- theory-driven scale development (chapter three), 2- an ‘expert’ review of the content validity, face validity, and usability of the scale (chapter four), and 3- an evaluation of the scale involving a pilot study examining its psychometric properties (chapter five) and a focus group examining its usability and utility (chapter six). Results from these studies indicate that the ACCS is a useful learning tool, is easy to use, has good psychometric properties, and offers an acceptable alternative to the CTS-R. Finally, chapter seven examines whether assessors require training in how to use the ACCS, concluding that simply reading the ACCS manual may be sufficient to achieve acceptable levels of reliability and usability. The results from the thesis are then drawn together in the final concluding comments in chapter eight, which discusses the findings within the broader context of the assessment of CBT competence.This thesis is not currently available on ORA