BB Brodey

F Azocar

Francisca Azocar

GS Brown

Improving the Quality of Health Care for Mental and Substance-Use Conditions

Jeanne Miranda

M. Audrey Burnam

MJ Lambert

English

PubMed

Miranda, Jeanne

Azocar, Francisca

Burnam, M. Audrey

eScholarship - University of California

UCLAUCLA Previously Published WorksTitleAssessment of Evidence-Based Psychotherapy Practices in Usual Care: Challenges, Promising Approaches, and Future DirectionsPermalinkhttps://escholarship.org/uc/item/2873b9w9JournalAdministration and Policy in Mental Health and Mental Health Services Research, 37(3)ISSN1573-3289AuthorsMiranda, JeanneAzocar, FranciscaBurnam, M. AudreyPublication Date2010-05-01DOI10.1007/s10488-009-0246-7 Peer reviewedeScholarship.org Powered by the California Digital LibraryUniversity of CaliforniaORIGINAL PAPERAssessment of Evidence-Based Psychotherapy Practices in UsualCare: Challenges, Promising Approaches, and Future DirectionsJeanne Miranda • Francisca Azocar •M. Audrey BurnamPublished online: 31 March 2010 The Author(s) 2010. This article is published with open access at Springerlink.comIn this special issue of Administration and Policy in MentalHealth and Mental Health Services Research, we focuson measuring evidence-based practices in psychotherapywithin usual care practice. Measuring evidence-basedpractices in usual care settings is important for at leastthree reasons. First, such measurement would help identifypractices within usual care settings that are promising. Forexample, patients in usual care psychotherapy settings tendto have co-morbid mental disorders; whereas, our evidencebase for psychotherapy is largely based on single disorders.Mixing strategies from more than one evidence-base maybe effective in practice. Second, measuring evidence-basedpractices is essential for quality improvement interven-tions. Such measures could provide a baseline, as wellas a comparative measure of change following qualityimprovement interventions that attempt to introduce moreevidence-based care to patients. Third, understanding whatclinical practices in usual care works best individually or incombination can help tailor graduate training programs toimprove the pool of new professionals currently in training.Overall, developing measures of and understanding psy-chotherapy as it happens in the community is important forimproving our mental health service delivery system.National policy forums focused on improving the mentalhealth delivery system are all noting the need for improvedmeasurement of quality care. The Institute of MedicineReport, Improving The Quality of Health Care for Mentaland Substance-Use Conditions (2006), notes, ‘‘Measuringthe quality of care provided by individuals, organizations,and health plans and reporting back the results is linkedboth conceptually and empirically to reductions in varia-tions in care and increases in delivery of effective care.’’Recommendation 4–3 of this Report focuses on developingbetter measures of quality. Similarly, The President’sAdvisory Commission on consumer Protection and Qualityin the Health Care Industry (1998) identified mental healthcare as an area where quality measures, such as measuresof evidence-based practices, are not well addressed.Managed behavioral health organizations have theclinical and fiduciary responsibility over the quality of careprovided to their members; therefore it is in their owninterest to have tools that measure quality of care forpsychotherapy. Most mental health practitioners belong tomultiple managed care organization provider panels wherecurrent measures of quality of care for behavioral healthcare are based on population-level metrics. The HealthPlan Employer Data and Information Set (HEDIS) devel-oped by National Committee on Quality Assurance(NCQA) is designed to measure access to care, enrolleesatisfaction, use of services and the effectiveness of care.Within behavioral health the metrics for effectiveness, forexample, include follow-up after hospitalization rates(FUH), anti-depressant medication management (AMM),and follow-up for children prescribed attention-deficit/hyperactivity medications (ADD). However, these metricsare poor measures of actual quality of care, and primarilyapply to health plans rather than managed behavioral healthorganization. Most individuals receiving outpatient careJ. Miranda (&)Department of Psychiatry and Biobehavioral Sciences,University of California, Los Angeles, 10920 Wilshire Blvd.,Suite 300, Los Angeles, CA 90024, USAe-mail: mirandaj@ucla.eduF. AzocarOptimum Health Behavioral Solutions, 425 Market Street,27th Floor, San Francisco, CA 94105, USAM. A. BurnamBehavioral and Social Sciences Group, RAND Corporation,1776 Main Street, Santa Monica, CA 90401, USA123Adm Policy Ment Health (2010) 37:205–207DOI 10.1007/s10488-009-0246-7through managed behavioral health organization providerpanels are receiving psychotherapy, yet HEDIS metrics arebased on medication management and inpatient care. Thereare no measures currently available that can provide indi-vidual-level metrics of the use of evidence–based psycho-therapy to measure overall quality of care. As a proxy,managed behavioral health organizations may measure thequality of care provided by their provider panels byassessing the use of clinical-practice guidelines (Azocaret al. 2001, 2003) or by monitoring outcomes and in somecircumstances, providing feedback reports (Lambert et al.2005; Brodey et al. 2005; Brown et al. 2005); neither ofwhich look at the actual use of evidence-based practicesand how that relates to quality of care. The use of out-comes-informed care is growing but still a rarity amongbehavioral health providers and difficult to implementwithin managed behavioral health organizations because ofits association to pay-for-performance issues. The devel-opment of measures of evidence-based practices in outpa-tient psychotherapy will help engage the provider and theconsumer at the individual level as a stakeholder inimproving quality of care.The need for measuring evidence-based psychotherapypractices have also been called for by those individualsdeveloping policies to improve our mental health work-force. The Annapolis Coalition on Workforce Develop-ment, in their report ‘‘An Action Plan for BehavioralHealth Workforce Development (DHHS 2007)’’ discussthe recurrent finding of lack of reliable and valid data onthe status of workforce development strategies, includingevaluation of the sustained adoption of newly learned skillsin real-world service settings. In this special issue, webegin to answer these important policy recommendationsby presenting new evidence in this seminal study of evi-dence-based practices in clinical settings.In this special issue, four papers help develop methodsfor moving ahead in this important field of measuringevidence-based practices in clinical settings. Garland et al.(2010) offer an important overview of the methodologicalissues to be considered when measuring usual care aspracticed in clinical settings. Hepner et al. (2010a) providedata on a short, efficient measure of evidene-based prac-tices for depression care completed by providers of adultservices and Kelley et al. (2010) provide data on a similarmeasure for youth services. Hurlburt et al. (2010) thenpresent compelling data regarding therapists self-rating ofpsychotherapy techniques and goals as compared withratings by observers. Finally, Miranda et al. (2010) presentinitial data on a self-report measure of psychotherapytechniques completed by patients following usual caretherapy in managed health care settings.Following these initial papers addressing importantmethods issues in measuring psychotherapy practice, fourpapers present state-of-the-art data on psychotherapy inclinical settings. Brookman-Frazee et al. (2010) discusscharacteristics that predict the likelihood of evidence-basedpractices occurring in child therapy sessions. Hepner et al.(2010b) present data on evidence-based practices in psy-chotherapy among practitioners in a large, cross-nationalmanaged behavioral healthcare organization. Finally,Landry et al. (2010) present data on evidence-based prac-tices in a nationally representative sample of patientsreceiving mental health care services.Together, these articles define the state of the art formeasuring psychotherapy practices in clinical settings. Thisseries provides a clear overview of this seminal work andhelps shape new efforts to carefully understand the care weprovide to our nations vulnerable populations with mentalhealth care needs.Open Access This article is distributed under the terms of theCreative Commons Attribution Noncommercial License which per-mits any noncommercial use, distribution, and reproduction in anymedium, provided the original author(s) and source are credited.ReferencesAzocar, F., Cuffel, B., Goldman, W., & McCarter, L. (2003). Theimpact of evidence-based guideline dissemination for theassessment and treatment of major depression in a managedbehavioral health care organization. Journal of BehavioralHealth Services & Research, 30, 09–18.Azocar, F., Cuffel, B. D., Goldman, W., & McCulloch, J. (2001). Bestpractices: Dissemination of guidelines for the treatment of majordepression in a managed behavioral health care network.Psychiatic Services, 52, 1014–1016.Brodey, B. B., Cuffel, B. J., McCulloch, J., Tani, S., Maruish, M.,Brodey, I., et al. (2005). Acceptability and effectiveness ofpatient-reported assessments and feedback in a managed behav-ioral healthcare setting. American Journal of Managed Care, 11,774–780.Brookman-Frazee, L., Haine, R. A., Baker-Ericzen, M., Zoffness, R.,& Garland, A. F. (2010). Factors associated with use ofevidence-based practice strategies in usual care youth psycho-therapy. Administration and Policy in Mental Health and MentalHealth Services Research. doi:10.1007/s10488-009-0244-9.Brown, G. S., Jones, E., Lambert, M. J., & Minami, T. (2005).Evaluating the effectiveness of psychotherapist in a managedcare environment. American Journal of Managed Care, 2(8),513–520.DHHS. Annapolis Coalition (2007). An action plan for behavioralhealth workforce development: A framework for discussion bythe Annapolis Coalition on the Behavioral Health Workforce.(SAMHSA/DHHS Publication No. 280-02-0302). Rockville,MD: Department of Health and Human Services.Garland, A. F., Hurlburt, M., Brookman-Frazee, L., Taylor, R. M., &Accurso, E. C. (2010). Methodological challenges of character-izing usual care psychotherapeutic practice. Administration andPolicy in Mental Health and Mental Health Services Research.doi:10.1007/s10488-009-0237-8.Hepner, K. A., Azocar, F., Greenwood, G. L., Miranda, J., & Burnam,M. A. (2010a). Development of a clinician report measure toassess psychotherapy for depression in usual care settings.206 Adm Policy Ment Health (2010) 37:205–207123Administration and Policy in Mental Health and Mental HealthServices Research. doi:10.1007/s10488-009-0249-4.Hepner, K. A., Greenwood, G. L., Azocar, F., Miranda, J., & Burnam,M. A. (2010b). Usual care psychotherapy for depression in alarge managed behavioral health organization. Administrationand Policy in Mental Health and Mental Health ServicesResearch. doi:10.1007/s10488-009-0247-6.Hurlburt, M. S., Garland, A. F., Nguyen, K., & Brookman-Frazee, L.(2010). Child and family therapy process: Concordance oftherapist and observational perspectives. Administration andPolicy in Mental Health and Mental Health Services Research.doi:10.1007/s10488-009-0251-x.Improving the Quality of Health Care for Mental and Substance-UseConditions. (2006). Quality Chasm Series. Committee oncrossing the quality chasm: Adaptation to mental health andaddictive disorders. Washington, DC: The Institute of Medicine,The National Academies Press.Kelley, S. D., de Andrade, A. R. V., Sheffer, E., & Bickman, L.(2010). Exploring the black box: Measuring youth treatmentprocess and progress in usual care. Administration and Policy inMental Health and Mental Health Services Research, 37(3).Lambert, M. J., Harmon, C., Slade, K., Whipple, J. L., & Hawkins, E.J. (2005). Providing feedback to psychotherapists on theirpatients’ progress: Clinical results and practice suggestions.Journal of Clinical Psychology, 61, 165–174.Landry, C., Klap, R., Tang, L., Liao, D., Miranda, J., & Wells, K. B.(2010). The content of substance abuse and mental healthcounseling reported by patients in a national survey. Adminis-tration and Policy in Mental Health and Mental Health ServicesResearch, 37(3). doi:10.1007/s10488-009-0250-y.Miranda, J., Hepner, K. A., Azocar, F., Greenwood G., Ngo V., &Burnam, M. A. (2010). Development of a patient-report measureof psychotherapy for depression. Administration and Policy inMental Health and Mental Health Services Research. doi:10.1007/s10488-009-0238-7.The President’s Advisory Commission on Consumer Protection andQuality in the Health Care Industry. (1998). Quality First: BetterHealth Care for All Americans. http://www.hequalitycommission.gov/.Adm Policy Ment Health (2010) 37:205–207 207123

Assessment of Evidence-Based Psychotherapy Practices in Usual Care: Challenges, Promising Approaches, and Future Directions

Crossref

Springer - Publisher Connector

ORIGINAL PAPER
Assessment of Evidence-Based Psychotherapy Practices in Usual
Care: Challenges, Promising Approaches, and Future Directions
Jeanne Miranda • Francisca Azocar •
M. Audrey Burnam
Published online: 31 March 2010
 The Author(s) 2010. This article is published with open access at Springerlink.com
In this special issue of Administration and Policy in Mental
Health and Mental Health Services Research, we focus
on measuring evidence-based practices in psychotherapy
within usual care practice. Measuring evidence-based
practices in usual care settings is important for at least
three reasons. First, such measurement would help identify
practices within usual care settings that are promising. For
example, patients in usual care psychotherapy settings tend
to have co-morbid mental disorders; whereas, our evidence
base for psychotherapy is largely based on single disorders.
Mixing strategies from more than one evidence-base may
be effective in practice. Second, measuring evidence-based
practices is essential for quality improvement interven-
tions. Such measures could provide a baseline, as well
as a comparative measure of change following quality
improvement interventions that attempt to introduce more
evidence-based care to patients. Third, understanding what
clinical practices in usual care works best individually or in
combination can help tailor graduate training programs to
improve the pool of new professionals currently in training.
Overall, developing measures of and understanding psy-
chotherapy as it happens in the community is important for
improving our mental health service delivery system.
National policy forums focused on improving the mental
health delivery system are all noting the need for improved
measurement of quality care. The Institute of Medicine
Report, Improving The Quality of Health Care for Mental
and Substance-Use Conditions (2006), notes, ‘‘Measuring
the quality of care provided by individuals, organizations,
and health plans and reporting back the results is linked
both conceptually and empirically to reductions in varia-
tions in care and increases in delivery of effective care.’’
Recommendation 4–3 of this Report focuses on developing
better measures of quality. Similarly, The President’s
Advisory Commission on consumer Protection and Quality
in the Health Care Industry (1998) identified mental health
care as an area where quality measures, such as measures
of evidence-based practices, are not well addressed.
Managed behavioral health organizations have the
clinical and fiduciary responsibility over the quality of care
provided to their members; therefore it is in their own
interest to have tools that measure quality of care for
psychotherapy. Most mental health practitioners belong to
multiple managed care organization provider panels where
current measures of quality of care for behavioral health
care are based on population-level metrics. The Health
Plan Employer Data and Information Set (HEDIS) devel-
oped by National Committee on Quality Assurance
(NCQA) is designed to measure access to care, enrollee
satisfaction, use of services and the effectiveness of care.
Within behavioral health the metrics for effectiveness, for
example, include follow-up after hospitalization rates
(FUH), anti-depressant medication management (AMM),
and follow-up for children prescribed attention-deficit/
hyperactivity medications (ADD). However, these metrics
are poor measures of actual quality of care, and primarily
apply to health plans rather than managed behavioral health
organization. Most individuals receiving outpatient care
J. Miranda (&)
Department of Psychiatry and Biobehavioral Sciences,
University of California, Los Angeles, 10920 Wilshire Blvd.,
Suite 300, Los Angeles, CA 90024, USA
e-mail: mirandaj@ucla.edu
F. Azocar
Optimum Health Behavioral Solutions, 425 Market Street,
27th Floor, San Francisco, CA 94105, USA
M. A. Burnam
Behavioral and Social Sciences Group, RAND Corporation,
1776 Main Street, Santa Monica, CA 90401, USA
123
Adm Policy Ment Health (2010) 37:205–207
DOI 10.1007/s10488-009-0246-7
through managed behavioral health organization provider
panels are receiving psychotherapy, yet HEDIS metrics are
based on medication management and inpatient care. There
are no measures currently available that can provide indi-
vidual-level metrics of the use of evidence–based psycho-
therapy to measure overall quality of care. As a proxy,
managed behavioral health organizations may measure the
quality of care provided by their provider panels by
assessing the use of clinical-practice guidelines (Azocar
et al. 2001, 2003) or by monitoring outcomes and in some
circumstances, providing feedback reports (Lambert et al.
2005; Brodey et al. 2005; Brown et al. 2005); neither of
which look at the actual use of evidence-based practices
and how that relates to quality of care. The use of out-
comes-informed care is growing but still a rarity among
behavioral health providers and difficult to implement
within managed behavioral health organizations because of
its association to pay-for-performance issues. The devel-
opment of measures of evidence-based practices in outpa-
tient psychotherapy will help engage the provider and the
consumer at the individual level as a stakeholder in
improving quality of care.
The need for measuring evidence-based psychotherapy
practices have also been called for by those individuals
developing policies to improve our mental health work-
force. The Annapolis Coalition on Workforce Develop-
ment, in their report ‘‘An Action Plan for Behavioral
Health Workforce Development (DHHS 2007)’’ discuss
the recurrent finding of lack of reliable and valid data on
the status of workforce development strategies, including
evaluation of the sustained adoption of newly learned skills
in real-world service settings. In this special issue, we
begin to answer these important policy recommendations
by presenting new evidence in this seminal study of evi-
dence-based practices in clinical settings.
In this special issue, four papers help develop methods
for moving ahead in this important field of measuring
evidence-based practices in clinical settings. Garland et al.
(2010) offer an important overview of the methodological
issues to be considered when measuring usual care as
practiced in clinical settings. Hepner et al. (2010a) provide
data on a short, efficient measure of evidene-based prac-
tices for depression care completed by providers of adult
services and Kelley et al. (2010) provide data on a similar
measure for youth services. Hurlburt et al. (2010) then
present compelling data regarding therapists self-rating of
psychotherapy techniques and goals as compared with
ratings by observers. Finally, Miranda et al. (2010) present
initial data on a self-report measure of psychotherapy
techniques completed by patients following usual care
therapy in managed health care settings.
Following these initial papers addressing important
methods issues in measuring psychotherapy practice, four
papers present state-of-the-art data on psychotherapy in
clinical settings. Brookman-Frazee et al. (2010) discuss
characteristics that predict the likelihood of evidence-based
practices occurring in child therapy sessions. Hepner et al.
(2010b) present data on evidence-based practices in psy-
chotherapy among practitioners in a large, cross-national
managed behavioral healthcare organization. Finally,
Landry et al. (2010) present data on evidence-based prac-
tices in a nationally representative sample of patients
receiving mental health care services.
Together, these articles define the state of the art for
measuring psychotherapy practices in clinical settings. This
series provides a clear overview of this seminal work and
helps shape new efforts to carefully understand the care we
provide to our nations vulnerable populations with mental
health care needs.
Open Access This article is distributed under the terms of the
Creative Commons Attribution Noncommercial License which per-
mits any noncommercial use, distribution, and reproduction in any
medium, provided the original author(s) and source are credited.
References
Azocar, F., Cuffel, B., Goldman, W., & McCarter, L. (2003). The
impact of evidence-based guideline dissemination for the
assessment and treatment of major depression in a managed
behavioral health care organization. Journal of Behavioral
Health Services & Research, 30, 09–18.
Azocar, F., Cuffel, B. D., Goldman, W., & McCulloch, J. (2001). Best
practices: Dissemination of guidelines for the treatment of major
depression in a managed behavioral health care network.
Psychiatic Services, 52, 1014–1016.
Brodey, B. B., Cuffel, B. J., McCulloch, J., Tani, S., Maruish, M.,
Brodey, I., et al. (2005). Acceptability and effectiveness of
patient-reported assessments and feedback in a managed behav-
ioral healthcare setting. American Journal of Managed Care, 11,
774–780.
Brookman-Frazee, L., Haine, R. A., Baker-Ericzen, M., Zoffness, R.,
& Garland, A. F. (2010). Factors associated with use of
evidence-based practice strategies in usual care youth psycho-
therapy. Administration and Policy in Mental Health and Mental
Health Services Research. doi:10.1007/s10488-009-0244-9.
Brown, G. S., Jones, E., Lambert, M. J., & Minami, T. (2005).
Evaluating the effectiveness of psychotherapist in a managed
care environment. American Journal of Managed Care, 2(8),
513–520.
DHHS. Annapolis Coalition (2007). An action plan for behavioral
health workforce development: A framework for discussion by
the Annapolis Coalition on the Behavioral Health Workforce.
(SAMHSA/DHHS Publication No. 280-02-0302). Rockville,
MD: Department of Health and Human Services.
Garland, A. F., Hurlburt, M., Brookman-Frazee, L., Taylor, R. M., &
Accurso, E. C. (2010). Methodological challenges of character-
izing usual care psychotherapeutic practice. Administration and
Policy in Mental Health and Mental Health Services Research.
doi:10.1007/s10488-009-0237-8.
Hepner, K. A., Azocar, F., Greenwood, G. L., Miranda, J., & Burnam,
M. A. (2010a). Development of a clinician report measure to
assess psychotherapy for depression in usual care settings.
206 Adm Policy Ment Health (2010) 37:205–207
123
Administration and Policy in Mental Health and Mental Health
Services Research. doi:10.1007/s10488-009-0249-4.
Hepner, K. A., Greenwood, G. L., Azocar, F., Miranda, J., & Burnam,
M. A. (2010b). Usual care psychotherapy for depression in a
large managed behavioral health organization. Administration
and Policy in Mental Health and Mental Health Services
Research. doi:10.1007/s10488-009-0247-6.
Hurlburt, M. S., Garland, A. F., Nguyen, K., & Brookman-Frazee, L.
(2010). Child and family therapy process: Concordance of
therapist and observational perspectives. Administration and
Policy in Mental Health and Mental Health Services Research.
doi:10.1007/s10488-009-0251-x.
Improving the Quality of Health Care for Mental and Substance-Use
Conditions. (2006). Quality Chasm Series. Committee on
crossing the quality chasm: Adaptation to mental health and
addictive disorders. Washington, DC: The Institute of Medicine,
The National Academies Press.
Kelley, S. D., de Andrade, A. R. V., Sheffer, E., & Bickman, L.
(2010). Exploring the black box: Measuring youth treatment
process and progress in usual care. Administration and Policy in
Mental Health and Mental Health Services Research, 37(3).
Lambert, M. J., Harmon, C., Slade, K., Whipple, J. L., & Hawkins, E.
J. (2005). Providing feedback to psychotherapists on their
patients’ progress: Clinical results and practice suggestions.
Journal of Clinical Psychology, 61, 165–174.
Landry, C., Klap, R., Tang, L., Liao, D., Miranda, J., & Wells, K. B.
(2010). The content of substance abuse and mental health
counseling reported by patients in a national survey. Adminis-
tration and Policy in Mental Health and Mental Health Services
Research, 37(3). doi:10.1007/s10488-009-0250-y.
Miranda, J., Hepner, K. A., Azocar, F., Greenwood G., Ngo V., &
Burnam, M. A. (2010). Development of a patient-report measure
of psychotherapy for depression. Administration and Policy in
Mental Health and Mental Health Services Research. doi:
10.1007/s10488-009-0238-7.
The President’s Advisory Commission on Consumer Protection and
Quality in the Health Care Industry. (1998). Quality First: Better
Health Care for All Americans. http://www.hequalitycommi
ssion.gov/.
Adm Policy Ment Health (2010) 37:205–207 207
123


Acceptability and effectiveness of patient-reported assessments and feedback in a managed behavioral healthcare setting.

Adm Policy Ment Health

An action plan for behavioral health workforce development: A framework for discussion by the Annapolis Coalition on the Behavioral Health Workforce.

Best practices: Dissemination of guidelines for the treatment of major depression in a managed behavioral health care network.

Child and family therapy process: Concordance of therapist and observational perspectives.

Development of a clinician report measure to assess psychotherapy for depression in usual care settings.

Development of a patient-report measure of psychotherapy for depression.

Evaluating the effectiveness of psychotherapist in a managed care environment.

Exploring the black box: Measuring youth treatment process and progress in usual care.

Factors associated with use of evidence-based practice strategies in usual care youth psychotherapy.

Improving the Quality of Health Care for Mental and Substance-Use Conditions.

Methodological challenges of characterizing usual care psychotherapeutic practice.

Providing feedback to psychotherapists on their patients’ progress: Clinical results and practice suggestions.

The content of substance abuse and mental health counseling reported by patients in a national survey.

The impact of evidence-based guideline dissemination for the assessment and treatment of major depression in a managed behavioral health care organization.

The President’s Advisory Commission on Consumer Protection and Quality in the Health Care Industry.

Usual care psychotherapy for depression in a large managed behavioral health organization.

http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2877333

Assessment of Evidence-Based Psychotherapy Practices in Usual Care: Challenges, Promising Approaches, and Future Directions

Abstract

Similar works

Full text

Available Versions

eScholarship - University of California

Crossref

Springer - Publisher Connector

Springer - Publisher Connector