Using flawed, uncertain, proximate and sparse (FUPS) data in the context of complexity: learning from the case of child mental health.

The use of routinely collected data that are flawed and limited to inform service development in healthcare systems needs to be considered, both theoretically and practically, given the reality in many areas of healthcare that only poor-quality data are available for use in complex adaptive systems. Data may be compromised in a range of ways. They may be flawed, due to missing or erroneously recorded entries; uncertain, due to differences in how data items are rated or conceptualised; proximate, in that data items are a proxy for key issues of concern; and sparse, in that a low volume of cases within key subgroups may limit the possibility of statistical inference. The term 'FUPS' is proposed to describe these flawed, uncertain, proximate and sparse datasets. Many of the systems that seek to use FUPS data may be characterised as dynamic and complex, involving a wide range of agents whose actions impact on each other in reverberating ways, leading to feedback and adaptation. The literature on the use of routinely collected data in healthcare is often implicitly premised on the availability of high-quality data to be used in complicated but not necessarily complex systems. This paper presents an example of the use of a FUPS dataset in the complex system of child mental healthcare. The dataset comprised routinely collected data from services that were part of a national service transformation initiative in child mental health from 2011 to 2015. The paper explores the use of this FUPS dataset to support meaningful dialogue between key stakeholders, including service providers, funders and users, in relation to outcomes of services. There is a particular focus on the potential for service improvement and learning. The issues raised and principles for practice suggested have relevance for other health communities that similarly face the dilemma of how to address the gap between the ideal of comprehensive clear data used in complicated, but not complex, contexts, and the reality of FUPS data in the context of complexity

Measuring mental health and wellbeing outcomes for children and adolescents to inform practice and policy:a review of child self-report measures

There is a growing appetite for mental health and wellbeing outcome measures that can inform clinical practice at individual and service levels, including use for local and national benchmarking. Despite a varied literature on child mental health and wellbeing outcome measures that focus on psychometric properties alone, no reviews exist that appraise the availability of psychometric evidence and suitability for use in routine practice in child and adolescent mental health services (CAMHS) including key implementation issues. This paper aimed to present the findings of the first review that evaluates existing broadband measures of mental health and wellbeing outcomes in terms of these criteria. The following steps were implemented in order to select measures suitable for use in routine practice: literature database searches, consultation with stakeholders, application of inclusion and exclusion criteria, secondary searches and filtering. Subsequently, detailed reviews of the retained measures’ psychometric properties and implementation features were carried out. 11 measures were identified as having potential for use in routine practice and meeting most of the key criteria: 1) Achenbach System of Empirically Based Assessment, 2) Beck Youth Inventories, 3) Behavior Assessment System for Children, 4) Behavioral and Emotional Rating Scale, 5) Child Health Questionnaire, 6) Child Symptom Inventories, 7) Health of the National Outcome Scale for Children and Adolescents, 8) Kidscreen, 9) Pediatric Symptom Checklist, 10) Strengths and Difficulties Questionnaire, 11) Youth Outcome Questionnaire. However, all existing measures identified had limitations as well as strengths. Furthermore, none had sufficient psychometric evidence available to demonstrate that they could reliably measure both severity and change over time in key groups. The review suggests a way of rigorously evaluating the growing number of broadband self-report mental health outcome measures against standards of feasibility and psychometric credibility in relation to use for practice and policy

Feedback from Outcome Measures and Treatment Effectiveness, Treatment Efficiency, and Collaborative Practice: A Systematic Review.

Due to recent increases in the use of feedback from outcome measures in mental health settings, we systematically reviewed evidence regarding the impact of feedback from outcome measures on treatment effectiveness, treatment efficiency, and collaborative practice. In over half of 32 studies reviewed, the feedback condition had significantly higher levels of treatment effectiveness on at least one treatment outcome variable. Feedback was particularly effective for not-on-track patients or when it was provided to both clinicians and patients. The findings for treatment efficiency and collaborative practice were less consistent. Given the heterogeneity of studies, more research is needed to determine when and for whom feedback is most effective

Assessing the readability of the self-reported Strengths and Difficulties Questionnaire

The Strengths and Difficulties Questionnaire (SDQ) is one of the most widely used measures in child and adolescent mental health in clinical practice, community-based screening and research. Assessing the readability of such questionnaires is important as young people may not comprehend items above their reading ability when self-reporting. Analyses of readability in the present study indicate that the self-report SDQ might not be suitable for young people with a reading age below 13–14 years and highlight differences in readability between subscales. The findings suggest a need for caution in using the SDQ as a self-report measure for children below the age of 13, and highlight considerations of readability in measure development, selection and interpretation

Development and validation of the illness perceptions questionnaire for youth anxiety and depression (IPQ-Anxiety and IPQ-Depression)

Background: The Revised Illness Perceptions Questionnaire (IPQ-R) is a well-established measure for measuring illness representations with sound psychometric properties. However, one limitation is that it provides a generic measure of illness representations and lacks specificity to individual health conditions, making it difficult to capture the nuances of illness beliefs for different populations. Objective: The aim of this study was to develop reliable and valid versions of the IPQ-R for young people with anxiety and depression to better understand how they perceive and cognitively represent the course, severity, impact, and treatability of their anxiety and depression. Methods: This mixed-methods study consisted of a qualitative study, involving semi-structured interviews (n = 26) followed by think-aloud interviews (n = 13), and a quantitative study (n = 349), resulting in the development of the IPQ-Anxiety (IPQ-A) and IPQDepression (IPQ-D). Item development is reported, along with the psychometric properties of the measures. Concurrent validity was assessed by correlating the IPQ-A and IPQ-D with the Brief Illness Perceptions Questionnaire (B-IPQ) across equivalent dimensions. Results: Results suggest that the IPQ-A, IPQ-D, B-IPQ-A and B-IPQ-D are valid and reliable tools for measuring mental illness representations. The measures show acceptable model fit, high factor loadings, and good to excellent internal consistency, test – retest reliability across subscales and concurrent validity with mental health measures. Conclusions: The development of these measures represents an important step in the field of youth mental health by providing the opportunity for reliable assessment of young people’s conceptualisations of their anxiety and depression. Better understanding of young people’s illness beliefs has the potential to open a range of intervention possibilities by prioritising illness perceptions over the supposed objective condition severity and trajectory

Development and validation of the illness perceptions questionnaire for youth anxiety and depression (IPQ-Anxiety and IPQ-Depression)

Background: The Revised Illness Perceptions Questionnaire (IPQ-R) is a well-established measure for measuring illness representations with sound psychometric properties. However, one limitation is that it provides a generic measure of illness representations and lacks specificity to individual health conditions, making it difficult to capture the nuances of illness beliefs for different populations. Objective: The aim of this study was to develop reliable and valid versions of the IPQ-R for young people with anxiety and depression to better understand how they perceive and cognitively represent the course, severity, impact, and treatability of their anxiety and depression. Methods: This mixed-methods study consisted of a qualitative study, involving semi-structured interviews (n = 26) followed by think-aloud interviews (n = 13), and a quantitative study (n = 349), resulting in the development of the IPQ-Anxiety (IPQ-A) and IPQ-Depression (IPQ-D). Item development is reported, along with the psychometric properties of the measures. Concurrent validity was assessed by correlating the IPQ-A and IPQ-D with the Brief Illness Perceptions Questionnaire (B-IPQ) across equivalent dimensions. Results: Results suggest that the IPQ-A, IPQ-D, B-IPQ-A and B-IPQ-D are valid and reliable tools for measuring mental illness representations. The measures show acceptable model fit, high factor loadings, and good to excellent internal consistency, test – retest reliability across subscales and concurrent validity with mental health measures. Conclusions: The development of these measures represents an important step in the field of youth mental health by providing the opportunity for reliable assessment of young people’s conceptualisations of their anxiety and depression. Better understanding of young people’s illness beliefs has the potential to open a range of intervention possibilities by prioritising illness perceptions over the supposed objective condition severity and trajectory