Comparison of control charts for monitoring clinical performance using binary data.

BACKGROUND: Time series charts are increasingly used by clinical teams to monitor their performance, but statistical control charts are not widely used, partly due to uncertainty about which chart to use. Although there is a large literature on methods, there are few systematic comparisons of charts for detecting changes in rates of binary clinical performance data. METHODS: We compared four control charts for binary data: the Shewhart p-chart; the exponentially weighted moving average (EWMA) chart; the cumulative sum (CUSUM) chart; and the g-chart. Charts were set up to have the same long-term false signal rate. Chart performance was then judged according to the expected number of patients treated until a change in rate was detected. RESULTS: For large absolute increases in rates (>10%), the Shewhart p-chart and EWMA both had good performance, although not quite as good as the CUSUM. For small absolute increases (<10%), the CUSUM detected changes more rapidly. The g-chart is designed to efficiently detect decreases in low event rates, but it again had less good performance than the CUSUM. IMPLICATIONS: The Shewhart p-chart is the simplest chart to implement and interpret, and performs well for detecting large changes, which may be useful for monitoring processes of care. The g-chart is a useful complement for determining the success of initiatives to reduce low-event rates (eg, adverse events). The CUSUM may be particularly useful for faster detection of problems with patient safety leading to increases in adverse event rates.  

How is Social Care Provided in Adult Prisons in England and Wales?

There is variation in provision of social care in prisons. Our research aimed to understand variation across adult prisons in England and Wales, including: (1) what social care is provided? (2) who delivers social care? (3) what peer support initiatives are used? (4) what social care indicators are relevant? and (5) are there differences between prison type and social care provision? We analysed Her Majesty’s Inspectorate of Prisons (HMIP) reports (published 2017–2020) from 102 prisons. From these reports we extracted and analysed data on social care provision. Elements of social care are not consistently delivered; need assessments (81.4 per cent) and referrals (75.5 per cent) were most frequently reported. Different providers (health care/social care/prison) deliver social care. Forty-one prisons (40.2 per cent) included peer support (formal to informal). We found no notable differences between prison categories and social care delivery, although, within category D prisons, a significantly larger proportion of those with a disability reported receiving support they needed. Inspection reports highlighted that prison social care should mirror community social care, but we could not fully evaluate this due to reporting issues. Social care provision varies; effectiveness of different models is not yet known. We provide recommendations to improve social care reporting within HMIP reports

Youth violence intervention programme for vulnerable young people attending emergency departments in London: a rapid evaluation

BACKGROUND: Youth violence intervention programmes involving the embedding of youth workers in NHS emergency departments to help young people (broadly aged between 11 and 24 years) improve the quality of their lives following their attendance at an emergency department as a result of violent assault or associated trauma are increasing across the NHS. This study evaluates one such initiative run by the charity Redthread in partnership with a NHS trust. OBJECTIVES: To evaluate the implementation and impact of a new youth violence intervention programme at University College London Hospital NHS Trust and delivered by the charity Redthread: (1) literature review of studies of hospital-based violent crime interventions; (2) evaluation of local implementation and of University College London Hospital staff and relevant local stakeholders concerning the intervention and its impact; (3) assessment of the feasibility of using routine secondary care data to evaluate the impact of the Redthread intervention; and (4) cost-effectiveness analysis of the Redthread intervention from the perspective of the NHS. METHODS: The evaluation was designed as a mixed-methods multiphased study, including an in-depth process evaluation case study and quantitative and economic analyses. The project was undertaken in different stages over two years, starting with desk-based research and an exploratory phase suitable for remote working while COVID-19 was affecting NHS services. A total of 22 semistructured interviews were conducted with staff at Redthread and University College London Hospital and others (e.g. a senior stakeholder involved in NHS youth violence prevention policy). We analysed Redthread documents, engaged with experts and conducted observations of staff meetings to gather more in-depth insights about the effectiveness of the intervention, the processes of implementation, staff perceptions and cost. We also undertook quantitative analyses to ascertain suitable measures of impact to inform stakeholders and future evaluations. RESULTS: Redthread's service was viewed as a necessary intervention, which complemented clinical and other statutory services. It was well embedded in the paediatric emergency department and adolescent services but less so in the adult emergency department. The diverse reasons for individual referrals, the various routes by which young people were identified, and the mix of specific support interventions provided, together emphasised the complexity of this intervention, with consequent challenges in implementation and evaluation. Given the relative unit costs of Redthread and University College London Hospital's inpatient services, it is estimated that the service would break even if around one-third of Redthread interventions resulted in at least one avoided emergency inpatient admission. This evaluation was unable to determine a feasible approach to measuring the quantitative impact of Redthread's youth violence intervention programme but has reflected on data describing the service, including costs, and make recommendations to support future evaluation. LIMITATIONS: The COVID-19 pandemic severely hampered the implementation of the Redthread service and the ability to evaluate it. The strongest options for analysis of effects and costs were not possible due to constraints of the consent process, problems in linking Redthread and University College London Hospital patient data and the relatively small numbers of young people having been engaged for longer-term support over the evaluation period. CONCLUSIONS: We have been able to contribute to the qualitative evidence on the implementation of the youth violence intervention programme at University College London Hospital, showing, for example, that NHS staff viewed the service as an important and needed intervention. In the light of problems with routine patient data systems and linkages, we have also been able to reflect on data describing the service, including costs, and made recommendations to support future evaluation. FUTURE WORK: No future work is planned. FUNDING: National Institute for Health and Care Research Health Services and Delivery Research programme (RSET: 16/138/17)

Remote home monitoring (virtual wards) for confirmed or suspected COVID-19 patients:a rapid systematic review

Background: the aim of this review was to analyze the implementation and impact of remote home monitoring models (virtual wards) for confirmed or suspected COVID-19 patients, identifying their main components, processes of implementation, target patient populations, impact on outcomes, costs and lessons learnt. Methods: we carried out a rapid systematic review on models led by primary and secondary care across seven countries (US, Australia, Canada, The Netherlands, Ireland, China, UK). The main outcomes included in the review were: impact of remote home monitoring on virtual length of stay, escalation, emergency department attendance/reattendance, admission/readmission and mortality. The search was updated on February 2021. We used the PRISMA statement and the review was registered on PROSPERO (CRD: 42020202888). Findings: the review included 27 articles. The aim of the models was to maintain patients safe in the appropriate setting. Most models were led by secondary care and confirmation of COVID-19 was not required (in most cases). Monitoring was carried via online platforms, paper-based systems with telephone calls or (less frequently) through wearable sensors. Models based on phone calls were considered more inclusive. Patient/career training was identified as a determining factor of success. We could not reach substantive conclusions regarding patient safety and the identification of early deterioration due to lack of standardized reporting and missing data. Economic analysis was not reported for most of the models and did not go beyond reporting resources used and the amount spent per patient monitored. Interpretation: future research should focus on staff and patient experiences of care and inequalities in patients' access to care. Attention needs to be paid to the cost-effectiveness of the models and their sustainability, evaluation of their impact on patient outcomes by using comparators, and the use of risk-stratification tools

A Cost Evaluation of COVID-19 Remote Home Monitoring Services in England

Background: Remote home monitoring services emerged as critical components of health care delivery from NHS England during the COVID-19 pandemic, aiming to provide timely interventions and reduce health care system burden. Two types of service were offered: referral by community health services to home-based care to ensure the right people were admitted to the hospital at the right time (called COVID Oximetry@home, CO@h); and referral by hospital to support patients’ transition from hospital to home (called COVID-19 Virtual Ward, CVW). The information collected for the oxygen levels and other symptoms was provided via digital means (technology-enabled) or over the phone (analogue-only submission mode). This study aimed to evaluate the costs of implementing remote home monitoring for COVID-19 patients across 26 sites in England during wave 2 of the pandemic. Understanding the operational and financial implications of these services from the NHS perspective is essential for effective resource allocation and service planning. Methods: We used a bottom-up costing approach at the intervention level to describe the costs of setting up and running the services. Twenty-six implementation sites reported the numbers of patients and staff involved in the service and other resources used. Descriptive statistics and multivariable regression analysis were used to assess cost variations and quantify the relationship between the number of users and costs while adjusting for other service characteristics. Results: The mean cost per patient monitored was lower in the CO@h service compared with the CVW service (£527 vs £599). The mean cost per patient was lower for implementation sites using technology-enabled and analogue data submission modes compared with implementation sites using analogue-only modes for both CO@h (£515 vs £561) and CVW (£584 vs £612) services. The number of patients enrolled in the services and the service type significantly affected the mean cost per patient. Conclusions: Our analysis provides a framework for evaluating the costs of similar services in the future and shows that the implementation of these services benefit from the employment of tech-enabled data submission modes

Patient and staff experiences of using technology-enabled and analogue models of remote home monitoring for COVID-19 in England: A mixed-method evaluation

Objective: To evaluate patient and staff experiences of using technology-enabled (‘tech-enabled’) and analogue remote home monitoring models for COVID-19, implemented in England during the pandemic. // Methods: Twenty-eight sites were selected for diversity in a range of criteria (e.g. pre-hospital or early discharge service, mode of patient data submission). Between February and May 2021, we conducted quantitative surveys with patients, carers and staff delivering the service, and interviewed patients, carers, and staff from 17 of the 28 services. Quantitative data were analysed using descriptive statistics and both univariate and multivariate analyses. Qualitative data were interpreted using thematic analysis. // Results: Twenty-one sites adopted mixed models whereby patients could submit their symptoms using either tech-enabled (app, weblink, or automated phone calls) or analogue (phone calls with a health professional) options; seven sites offered analogue-only data submission (phone calls or face-to-face visits with a health professional). Sixty-two patients and carers were interviewed, and 1069 survey responses were received (18 % response rate). Fifty-eight staff were interviewed, and 292 survey responses were received (39 % response rate). Patients who used tech-enabled modes tended to be younger (p = 0.005), have a higher level of education (p = 0.011), and more likely to identify as White British (p = 0.043). Most patients found relaying symptoms easy, regardless of modality, though many received assistance from family or friends. Staff considered the adoption of mixed delivery models beneficial, enabling them to manage large patient numbers and contact patients for further assessment as needed; however, they suggested improvements to the functionality of systems to better fit clinical and operational needs. Human contact was important in all remote home monitoring options. // Conclusions: Organisations implementing tech-enabled remote home monitoring at scale should consider adopting mixed models which can accommodate patients with different needs; focus on the usability and interoperability of tech-enabled platforms; and encourage digital inclusivity for patients

A rapid mixed-methods evaluation of remote home monitoring models during the COVID-19 pandemic in England

BACKGROUND: Remote home monitoring services were developed and implemented for patients with COVID-19 during the pandemic. Patients monitored blood oxygen saturation and other readings (e.g. temperature) at home and were escalated as necessary. OBJECTIVE: To evaluate effectiveness, costs, implementation, and staff and patient experiences (including disparities and mode) of COVID-19 remote home monitoring services in England during the COVID-19 pandemic (waves 1 and 2). METHODS: A rapid mixed-methods evaluation, conducted in two phases. Phase 1 (July-August 2020) comprised a rapid systematic review, implementation and economic analysis study (in eight sites). Phase 2 (January-June 2021) comprised a large-scale, multisite, mixed-methods study of effectiveness, costs, implementation and patient/staff experience, using national data sets, surveys (28 sites) and interviews (17 sites). RESULTS: Phase 1 Findings from the review and empirical study indicated that these services have been implemented worldwide and vary substantially. Empirical findings highlighted that communication, appropriate information and multiple modes of monitoring facilitated implementation; barriers included unclear referral processes, workforce availability and lack of administrative support. Phase 2 We received surveys from 292 staff (39% response rate) and 1069 patients/carers (18% response rate). We conducted interviews with 58 staff, 62 patients/carers and 5 national leads. Despite national roll-out, enrolment to services was lower than expected (average enrolment across 37 clinical commissioning groups judged to have completed data was 8.7%). There was large variability in implementation of services, influenced by patient (e.g. local population needs), workforce (e.g. workload), organisational (e.g. collaboration) and resource (e.g. software) factors. We found that for every 10% increase in enrolment to the programme, mortality was reduced by 2% (95% confidence interval: 4% reduction to 1% increase), admissions increased by 3% (-1% to 7%), in-hospital mortality fell by 3% (-8% to 3%) and lengths of stay increased by 1.8% (-1.2% to 4.9%). None of these results are statistically significant. We found slightly longer hospital lengths of stay associated with virtual ward services (adjusted incidence rate ratio 1.05, 95% confidence interval 1.01 to 1.09), and no statistically significant impact on subsequent COVID-19 readmissions (adjusted odds ratio 0.95, 95% confidence interval 0.89 to 1.02). Low patient enrolment rates and incomplete data may have affected chances of detecting possible impact. The mean running cost per patient varied for different types of service and mode; and was driven by the number and grade of staff. Staff, patients and carers generally reported positive experiences of services. Services were easy to deliver but staff needed additional training. Staff knowledge/confidence, NHS resources/workload, dynamics between multidisciplinary team members and patients' engagement with the service (e.g. using the oximeter to record and submit readings) influenced delivery. Patients and carers felt services and human contact received reassured them and were easy to engage with. Engagement was conditional on patient, support, resource and service factors. Many sites designed services to suit the needs of their local population. Despite adaptations, disparities were reported across some patient groups. For example, older adults and patients from ethnic minorities reported more difficulties engaging with the service. Tech-enabled models helped to manage large patient groups but did not completely replace phone calls. LIMITATIONS: Limitations included data completeness, inability to link data on service use to outcomes at a patient level, low survey response rates and under-representation of some patient groups. FUTURE WORK: Further research should consider the long-term impact and cost-effectiveness of these services and the appropriateness of different models for different groups of patients. CONCLUSIONS: We were not able to find quantitative evidence that COVID-19 remote home monitoring services have been effective. However, low enrolment rates, incomplete data and varied implementation reduced our chances of detecting any impact that may have existed. While services were viewed positively by staff and patients, barriers to implementation, delivery and engagement should be considered. STUDY REGISTRATION: This study is registered with the ISRCTN (14962466). FUNDING: This project was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (RSET: 16/138/17; BRACE: 16/138/31) and NHSEI and will be published in full in Health and Social Care Delivery Research; Vol. 11, No. 13. See the NIHR Journals Library website for further project information. The views expressed in this publication are those of the authors and not necessarily those of the National Institute for Health and Care Research or the Department of Health and Social Care