Advancing current approaches to disease management evaluation:Capitalizing on heterogeneity to understand what works and for whom

BACKGROUND: Evaluating large-scale disease management interventions implemented in actual health care settings is a complex undertaking for which universally accepted methods do not exist. Fundamental issues, such as a lack of control patients and limited generalizability, hamper the use of the ‘gold-standard’ randomized controlled trial, while methodological shortcomings restrict the value of observational designs. Advancing methods for disease management evaluation in practice is pivotal to learn more about the impact of population-wide approaches. Methods must account for the presence of heterogeneity in effects, which necessitates a more granular assessment of outcomes. METHODS: This paper introduces multilevel regression methods as valuable techniques to evaluate ‘real-world’ disease management approaches in a manner that produces meaningful findings for everyday practice. In a worked example, these methods are applied to retrospectively gathered routine health care data covering a cohort of 105,056 diabetes patients who receive disease management for type 2 diabetes mellitus in the Netherlands. Multivariable, multilevel regression models are fitted to identify trends in clinical outcomes and correct for differences in characteristics of patients (age, disease duration, health status, diabetes complications, smoking status) and the intervention (measurement frequency and range, length of follow-up). RESULTS: After a median one year follow-up, the Dutch disease management approach was associated with small average improvements in systolic blood pressure and low-density lipoprotein, while a slight deterioration occurred in glycated hemoglobin. Differential findings suggest that patients with poorly controlled diabetes tend to benefit most from disease management in terms of improved clinical measures. Additionally, a greater measurement frequency was associated with better outcomes, while longer length of follow-up was accompanied by less positive results. CONCLUSIONS: Despite concerted efforts to adjust for potential sources of confounding and bias, there ultimately are limits to the validity and reliability of findings from uncontrolled research based on routine intervention data. While our findings are supported by previous randomized research in other settings, the trends in outcome measures presented here may have alternative explanations. Further practice-based research, perhaps using historical data to retrospectively construct a control group, is necessary to confirm results and learn more about the impact of population-wide disease management

Usability testing of a monitoring and feedback tool to stimulate physical activity

Introduction: A monitoring and feedback tool to stimulate physical activity, consisting of an activity sensor, smartphone application (app), and website for patients and their practice nurses, has been developed: the 'It's LiFe!' tool. In this study the usability of the tool was evaluated by technology experts and end users (people with chronic obstructive pulmonary disease or type 2 diabetes, with ages from 40-70 years), to improve the user interfaces and content of the tool. Patients and methods: The study had four phases: 1) a heuristic evaluation with six technology experts; 2) a usability test in a laboratory by five patients; 3) a pilot in real life wherein 20 patients used the tool for 3 months; and 4) a final lab test by five patients. In both lab tests (phases 2 and 4) qualitative data were collected through a thinking-aloud procedure and video recordings, and quantitative data through questions about task complexity, text comprehensiveness, and readability. In addition, the post-study system usability questionnaire (PSSUQ) was completed for the app and the website. In the pilot test (phase 3), all patients were interviewed three times and the Software Usability Measurement Inventory (SUMI) was completed. Results: After each phase, improvements were made, mainly to the layout and text. The main improvement was a refresh button for active data synchronization between activity sensor, app, and server, implemented after connectivity problems in the pilot test. The mean score on the PSSUQ for the website improved from 5.6 (standard deviation [SD] 1.3) to 6.5 (SD 0.5), and for the app from 5.4 (SD 1.5) to 6.2 (SD 1.1). Satisfaction in the pilot was not very high according to the SUMI. Discussion: The use of laboratory versus real-life tests and expert-based versus user-based tests revealed a wide range of usability issues. The usability of the It's LiFe! tool improved considerably during the study

User-centered development and testing of a monitoring system that provides feedback regarding physical functioning to elderly people

Purpose: To involve elderly people during the development of a mobile interface of a monitoring system that provides feedback to them regarding changes in physical functioning and to test the system in a pilot study. Methods and participants: The iterative user-centered development process consisted of the following phases: (1) selection of user representatives; (2) analysis of users and their context; (3) identification of user requirements; (4) development of the interface; and (5) evaluation of the interface in the lab. Subsequently, the monitoring and feedback system was tested in a pilot study by five patients who were recruited via a geriatric outpatient clinic. Participants used a bathroom scale to monitor weight and balance, and a mobile phone to monitor physical activity on a daily basis for six weeks. Personalized feedback was provided via the interface of the mobile phone. Usability was evaluated on a scale from 1 to 7 using a modified version of the Post-Study System Usability Questionnaire (PSSUQ); higher scores indicated better usability. Interviews were conducted to gain insight into the experiences of the participants with the system. Results: The developed interface uses colors, emoticons, and written and/or spoken text messages to provide daily feedback regarding (changes in) weight, balance, and physical activity. The participants rated the usability of the monitoring and feedback system with a mean score of 5.2 (standard deviation 0.90) on the modified PSSUQ. The interviews revealed that most participants liked using the system and appreciated that it signaled changes in their physical functioning. However, usability was negatively influenced by a few technical errors. Conclusion: Involvement of elderly users during the development process resulted in an interface with good usability. However, the technical functioning of the monitoring system needs to be optimized before it can be used to support elderly people in their self-management

Patients with Cluster A Personality Disorders in Psychotherapy: An Effectiveness Study

Abstract BACKGROUND: While psychopharmacological studies are common in patients with cluster A personality disorders, the effects of psychotherapy have received little attention. The aim of this study is to explore whether psychotherapeutic treatment yields health gains for these patients. METHODS: The study was conducted between March 2003 and June 2008 in 6 mental health care centres in the Netherlands, with a sample of 57 patients with a DSM-IV-TR axis II cluster A diagnosis. Patients were assigned to 3 settings of psychotherapeutic treatment (outpatient, day hospital, inpatient), and effectiveness was assessed at 18 months after baseline. An intention-to-treat analysis was conducted for psychiatric symptoms (Brief Symptom Inventory), psychosocial functioning (Outcome Questionnaire-45) and quality of life (EQ-5D), using multilevel statistical modelling. As the study was non-randomised, the propensity score method was used to control for initial differences. RESULTS: Patients in the day hospital and inpatient group improved substantially in terms of psychiatric symptoms, social and interpersonal functioning, and quality of life. Patients in the outpatient group showed less improvement. Direct comparison of the improvement of psychiatric symptoms showed significant results in favour of day hospital (p = 0.046) and inpatient (p = 0.01) treatment, as compared to outpatient treatment. However, due to substantial baseline differences, this direct comparison should be judged carefully. CONCLUSIONS: Cluster A psychopathology is not a contraindication to benefit from psychotherapy. This is especially true for more intensive forms like inpatient and day hospital treatment. Future research should focus more on psychotherapeutic treatment to gain further insight into effective treatment options for this patient grou

Effectiveness of outpatient, day hospital, and inpatient psychotherapeutic treatment for patients with cluster B personality disorders

Abstract Background: For patients with cluster B personality disor- ders there is no consensus regarding the optimal treatment setting. The aim of this study was to compare the effective- ness of different psychotherapeutic settings for patient

Effectiveness of different modalities of psychotherapeutic treatment for patients with cluster C personality disorders: results of a large prospective multicentre study.

Abstract BACKGROUND: No previous studies have compared the effectiveness of different modalities of psychotherapeutic treatment, as defined by different settings and durations, for patients with cluster C personality disorders. The aim of this multicentre study wa