Examining concurrent validity and item selection of the Session Wants and Needs Outcome Measure (SWAN-OM) in a children and young people web-based therapy service

Background: Single-session mental health interventions are frequently attended by children and young people (CYP) in both web-based and face-to-face therapy settings. The Session “Wants” and “Needs” Outcome Measure (SWAN-OM) is an instrument developed in a web-based therapy service to overcome the challenges of collecting outcomes and experiences of single-session therapies (SSTs). It provides pre-defined goals for the session, selected by the young person prior to the intervention, on which progress toward achievement is scored at the end of the session. Objective: The objective of this study was to evaluate the instrument's psychometric properties, including concurrent validity against three other frequently used outcome and experience measures, at a web-based and text-based mental health service. Methods: The SWAN-OM was administered for a period of 6 months to 1,401 CYP (aged 10–32 years; 79.3% white; 77.59% female) accessing SST on a web-based service. Item correlations with comparator measures and hierarchical logistic regressions to predict item selection were calculated for concurrent validity and psychometric exploration. Results: The most frequently selected items were “Feel better” (N = 431; 11.61%) and “Find ways I can help myself” (N = 411; 11.07%); unpopular items were “Feel safe in my relationships” (N = 53; 1.43%) and “Learn the steps to achieve something I want” (N = 58; 1.56%). The SWAN-OM was significantly correlated with the Experience of Service Questionnaire, particularly the item “Feel better” [rs(109) = 0.48, p < 0.001], the Youth Counseling Impact Scale, particularly the item “Learn the steps to achieve something I want” [rs(22) = 0.76, p < 0.001], and the Positive and Negative Affect Schedule, particularly the items “Learn how to feel better” [rs(22) = 0.72, p < 0.001] and “Explore how I feel” [rs(70) = −0.44, p < 0.001]. Conclusion: The SWAN-OM demonstrates good concurrent validity with common measures of outcome and experience. Analysis suggests that lesser-endorsed items may be removed in future iterations of the measure to improve functionality. Future research is required to explore SWAN-OM's potential to measure meaningful change in a range of therapeutic settings

The design and development of an experience measure for a peer community moderated forum in a digital mental health service.

Online digital mental health communities can contribute to users' mental health positively and negatively. Yet the measurement of experience, outcomes and impact mechanisms relating to digital mental health communities is difficult to capture. In this paper we demonstrate the development of an online experience measure for a specific children and young people's community forum inside a digital mental health service. The development of the Peer Online Community Experience Measure (POCEM) is informed by a multi-phased design: (i) item reduction through Estimate-Talk-Estimate modified Delphi methods, (ii) user testing with think-aloud protocols and (iii) a pilot study within the digital service community to explore observational data within the platform. Experts in the field were consulted to help reduce the items in the pool and to check their theoretical coherence. User testing workshops helped to inform the usability appearance, wording, and purpose of the measure. Finally, the pilot results highlight completion rates, differences in scores for age and roles and "relate to others", as the most frequent domain mechanism of support for this community. Outcomes frequently selected show the importance of certain aspects of the community, such as safety, connection, and non-judgment previously highlighted in the literature. Experience measures like this one could be used as indicators of active therapeutic engagement within the forum community and its content but further research is required to ascertain its acceptability and validity. Multi-phased approaches involving stakeholders and user-centred design activities enhances the development of digitally enabled measurement tools

Implementing responsible innovation: the role of the meso-level(s) between project and organisation

Much of academic discussion of responsible innovation (RI) has focused on RI integration into research projects. In addition, significant attention has also been paid to RI structures and policies at the research policy and institutional level. This article reports experiences of RI implementation with a focus on the intermediate i.e. meso-level. The research described here included a series of interviews that aimed to clarify researchers' perspectives on RI as well as barriers to and benefits of RI implementation. Two cases of engagement with research projects, with the aim of promoting RI, were undertaken. The analysis of the data demonstrates the crucial contribution that the meso-level of a research programme can make in interpreting, implementing and perpetuating RI across related activities. The article provides strong evidence that the scholarly debate surrounding RI should pay more explicit attention to this meso-level, ultimately strengthening RI theory and practice

The Helpfulness of Web-Based Mental Health and Well-being Forums for Providing Peer Support for Young People: Cross-sectional Exploration

BACKGROUND: Young people are increasingly seeking out web-based support for their mental health and well-being. Peer support forums are popular with this age group, with young individuals valuing the fact that the forums are available 24/7, providing a safe and anonymous space for exploration. Currently, little systematic evaluation of the helpfulness of such forums in providing support has been conducted. OBJECTIVE: This study examined the helpfulness of the support offered within web-based mental health and well-being peer support forums for young people. It specifically investigated the self-reported user ratings of helpfulness reported through the completion of a developing experience measure. The ratings will be used to consider further development of the measure and reflect upon the overall helpfulness of the forums as indicated by the reported scores. METHODS: The study used routinely collected practice-based outcome data from web-based mental health forums for young people. These forums are hosted by the UK-based web-based therapy and support service, Kooth. A cross-sectional design was used to explore—using a range of inferential statistical measures—the outcomes reported by those accessing the forums using a Peer Online Community Experience Measure (POCEM). To consider the helpfulness in general, 23,443 POCEMs completed in 2020 were used. A second data set of 17,137 completed POCEMs from the same year was used to consider whether various engagement indicators had an impact upon the helpfulness rating. RESULTS: Female users aged between 11 and 16 years predominantly completed the POCEM. This is in keeping with the majority of those using the service. In total, 74.6% (8240/11,045) of the scores on the POCEM indicated that the individuals found the posts helpful. An ANOVA indicated that male users were more likely to report obtaining intrapersonal support, whereas female users obtained interpersonal support. Furthermore, the POCEM scores reflected the internal consistency of the measure and provided an insight into the way that young people made use of the peer support resource; for instance, posts that were rated more helpful were correlated with spending longer time reading them, and the topics discussed varied throughout the day with more mental health issues being discussed later at night. CONCLUSIONS: The results seem to demonstrate that, overall, the young people involved in this study found web-based peer support helpful. They indicate that peer support can provide an important strand of care within a supportive mental health ecosystem, particularly during time periods when in-person support is typically closed. However, limitations were noted, suggesting that caution is needed when interpreting the results of this study. Although such services are incredibly well used, they have received little research attention to date. As such, further investigation into what constitutes helpful and unhelpful peer support is needed

The Session Wants and Need Outcome Measure: The Development of a Brief Outcome Measure for Single-Sessions of Web-Based Support

Single-session, brief interventions in therapy for young people make up a large proportion of service provision, including in digital mental health settings. Current nomothetic mental health measures are not specifically designed to capture the benefit or ‘change’ directly related to these brief interventions. As a consequence, we set out to design an outcome measure to concretely demonstrate the value of single-session interventions. The Session Wants and Needs Outcome Measure (SWAN-OM) aims to capture in-session goals and focuses on being user-centric, elements critical to the success of single-session and brief interventions which typically are asset-based and solution-focused. We describe the 4-stage process that was followed to develop this measure: (I) classical item generation and development, (II) content and (III) face validity pilot testing, and (IV) a user-experience approach with young people using framework analysis. This final stage was critical to ensure the integration of this outcome tool into a web-based digital therapy setting, a context which adds another layer of design complexity to item and measure development. This iterative methodology was used to overcome the challenges encountered and to place the needs of the young people and service practitioners at the centre of the design process, thus ensuring measure usability. To end, we highlight the main lessons learnt from engaging in this design process. Specifically, the needs of a measure for single- session interventions are considered, before outlining the learning associated with integrating the measure into a digital mental health platform. Both of these areas are emerging fields and, as such, this study contributes to our understanding of how an idiographic patient outcome theory driven measure can be created for use in a web-based digital mental health therapy service

ConversationMoC: encoding conversational dynamics using multiplex network for identifying moment of change in mood and mental health classification

Understanding mental health conversation dynamics is crucial,yet prior studies often overlooked the intricate interplay of social interactions. This paper introduces a unique conversationlevel dataset and investigates the impact of conversational context in detecting Moments of Change (MoC) in individual emotions and classifying Mental Health (MH) topics in discourse. In this study, we differentiate between analyzing individual posts and studying entire conversations, using sequential and graph-based models to encode the complex conversation dynamics. Further, we incorporate emotion and sentiment dynamics with social interactions using a graph multiplex model driven by Graph Convolution Networks (GCN). Comparative evaluations consistently highlight the enhanced performanceof the multiplex network, especially when combining reply, emotion, and sentiment network layers. This underscores the importance of understanding the intricate interplay between social interactions, emotional expressions, and sentiment patterns in conversations, especially within online mental health discussions. We are sharing our new dataset (ConversationMoC) and models with the broader research community to facilitate further research

An early economic evaluation of Kooth, a web-based mental health platform for children and young people with emerging mental health needs.

"Kooth" is a web-based mental health platform commissioned by the National Health Service (NHS), local authorities, charities, and businesses in the UK. The platform gives children and young people (CYP) access to an online community of peers and a team of counsellors. This study reports an early economic evaluation of the potential benefits of Kooth in the UK. An early evidence cost calculator was built to estimate the potential costs and savings of implementing Kooth from a UK NHS and crime sector perspective. A decision tree structure was used to track the progress of CYP with emerging mental health needs (EMHN), comparing CYP with access to Kooth to CYP without access to Kooth. The model implemented a 12-month time horizon and followed a typical Kooth contract in relation to costing, engagement, and CYP demographics. The base case results followed a cohort of 2160 CYP. The results of the cost calculator estimated that engagement with Kooth is associated with a cost saving of £469,237 to the NHS across a 12-month time horizon, or £236.15 per CYP with an EMHN. From a combined NHS and UK crime sector perspective, the cost savings increased to £489,897, or £246.54 per CYP with an EMHN. The largest cost savings were provided by an estimated reduction of 5346 GP appointments and 298 antidepressant prescriptions. For this cohort, the model predicted that engagement with Kooth averted 6 hospitalisations due to suicidal ideation and 13 hospitalisations due to self-harm. Furthermore, the number of smokers and binge drinkers was reduced by 20 and 24, respectively. When a crime sector perspective was taken, 3 crimes were averted. This early model demonstrates that Kooth has the potential to be a cost-saving intervention from both an NHS and a combined NHS and UK crime sector perspective. Cost savings were provided through aversion in clinical and social outcomes. The model used a conservative approach to balance the uncertainty around assumptions of the intermediate outcomes (GP and medication use). However, it is limited by a paucity of costing data and published evidence relating to the impact of digital mental health platforms

Table 1_v1_The design and development of an experience measure for a peer community moderated forum in a digital mental health service.docx

Online digital mental health communities can contribute to users' mental health positively and negatively. Yet the measurement of experience, outcomes and impact mechanisms relating to digital mental health communities is difficult to capture. In this paper we demonstrate the development of an online experience measure for a specific children and young people's community forum inside a digital mental health service. The development of the Peer Online Community Experience Measure (POCEM) is informed by a multi-phased design: (i) item reduction through Estimate-Talk-Estimate modified Delphi methods, (ii) user testing with think-aloud protocols and (iii) a pilot study within the digital service community to explore observational data within the platform. Experts in the field were consulted to help reduce the items in the pool and to check their theoretical coherence. User testing workshops helped to inform the usability appearance, wording, and purpose of the measure. Finally, the pilot results highlight completion rates, differences in scores for age and roles and “relate to others”, as the most frequent domain mechanism of support for this community. Outcomes frequently selected show the importance of certain aspects of the community, such as safety, connection, and non-judgment previously highlighted in the literature. Experience measures like this one could be used as indicators of active therapeutic engagement within the forum community and its content but further research is required to ascertain its acceptability and validity. Multi-phased approaches involving stakeholders and user-centred design activities enhances the development of digitally enabled measurement tools.</p