Assessing the equivalence of web-based and paper-and-pencil questionnaires using differential item and test functioning (DIF and DTF) analysis:A case of the Four-Dimensional Symptom Questionnaire (4DSQ)

Purpose:  Many paper-and-pencil (P&P) questionnaires have been migrated to electronic platforms. Differential item and test functioning (DIF and DTF) analysis constitutes a superior research design to assess measurement equivalence across modes of administration. The purpose of this study was to demonstrate an item response theory (IRT)-based DIF and DTF analysis to assess the measurement equivalence of a Web-based version and the original P&P format of the Four-Dimensional Symptom Questionnaire (4DSQ), measuring distress, depression, anxiety, and somatization. Methods:  The P&P group (n=2031) and the Web group (n=958) consisted of primary care psychology clients. Unidimensionality and local independence of the 4DSQ scales were examined using IRT and Yen’s Q3. Bifactor modeling was used to assess the scales’ essential unidimensionality. Measurement equivalence was assessed using IRT-based DIF analysis using a 3-stage approach: linking on the latent mean and variance, selection of anchor items, and DIF testing using the Wald test. DTF was evaluated by comparing expected scale scores as a function of the latent trait. Results:  The 4DSQ scales proved to be essentially unidimensional in both modalities. Five items, belonging to the distress and somatization scales, displayed small amounts of DIF. DTF analysis revealed that the impact of DIF on the scale levelwas negligible. Conclusions:  IRT-based DIF and DTF analysis is demonstrated as a way to assess the equivalence of Web-based and P&P questionnaire modalities. Data obtained with the Web-based 4DSQ are equivalent to data obtained with the P&P version

Detecting depressive and anxiety disorders in distressed patients in primary care; comparative diagnostic accuracy of the Four-Dimensional Symptom Questionnaire (4DSQ) and the Hospital Anxiety and Depression Scale (HADS)

BACKGROUND: Depressive and anxiety disorders often go unrecognized in distressed primary care patients, despite the overtly psychosocial nature of their demand for help. This is especially problematic in more severe disorders needing specific treatment (e.g. antidepressant pharmacotherapy or specialized cognitive behavioural therapy). The use of a screening tool to detect (more severe) depressive and anxiety disorders may be useful not to overlook such disorders. We examined the accuracy with which the Four-Dimensional Symptom Questionnaire (4DSQ) and the Hospital Anxiety and Depression Scale (HADS) are able to detect (more severe) depressive and anxiety disorders in distressed patients, and which cut-off points should be used. METHODS: Seventy general practitioners (GPs) included 295 patients on sick leave due to psychological problems. They excluded patients with recognized depressive or anxiety disorders. Patients completed the 4DSQ and HADS. Standardized diagnoses of DSM-IV defined depressive and anxiety disorders were established with the Composite International Diagnostic Interview (CIDI). Receiver Operating Characteristic (ROC) analyses were performed to obtain sensitivity and specificity values for a range of scores, and area under the curve (AUC) values as a measure of diagnostic accuracy. RESULTS: With respect to the detection of any depressive or anxiety disorder (180 patients, 61%), the 4DSQ and HADS scales yielded comparable results with AUC values between 0.745 and 0.815. Also with respect to the detection of moderate or severe depressive disorder, the 4DSQ and HADS depression scales performed comparably (AUC 0.780 and 0.739, p 0.165). With respect to the detection of panic disorder, agoraphobia and social phobia, the 4DSQ anxiety scale performed significantly better than the HADS anxiety scale (AUC 0.852 versus 0.757, p 0.001). The recommended cut-off points of both HADS scales appeared to be too low while those of the 4DSQ anxiety scale appeared to be too high. CONCLUSION: In general practice patients on sick leave because of psychological problems, the 4DSQ and the HADS are equally able to detect depressive and anxiety disorders. However, for the detection of cases severe enough to warrant specific treatment, the 4DSQ may have some advantages over the HADS, specifically for the detection of panic disorder, agoraphobia and social phobi

Understanding Ferguson's delta: time to say good-bye?

A critique of Hankins, M: 'How discriminating are discriminative instruments?' Health and Quality of Life Outcomes 2008, 6:3

Predicting Return to Work in Employees Sick-Listed Due to Minor Mental Disorders

Objective To investigate which factors predict return to work (RTW) after 3 and 6 months in employees sick-listed due to minor mental disorders. Methods Seventy GPs recruited 194 subjects at the start of sick leave due to minor mental disorders. At baseline (T0), 3 and 6 months later (T1 and T2, respectively), subjects received a questionnaire and were interviewed by telephone. Using multivariate logistic regression analyses, we developed three prediction models to predict RTW at T1 and T2. Results The RTW rates were 38% after 3 months (T1) and 61% after 6 months (T2). The main negative predictors of RTW at T1 were: (a) a duration of the problems of more than 3 months before sick leave; and (b) somatisation. The main negative predictors of RTW at T2 were: (a) a duration of the problems of more than 3 months before sick leave; (b) more than 3 weeks of sick leave before inclusion in the study; and (c) anxiety. The main negative predictors of RTW at T2 for those who had not resumed work at T1 were: (a) more than 3 weeks of sick leave before inclusion in the study; and (b) depression at T1. The predictive power of the models was moderate with AUC-values between 0.695 and 0.763. Conclusions The main predictors of RTW were associated with the severity of the problems. A long duration of the problems before the occurrence of sick leave and a long duration of sick leave before seeking help predict a relatively small probability to RTW within 3–6 months. High baseline somatisation and anxiety, and high depression after 3 months make the prospect even worse. Since these predictors are readily assessable with just a few questions and a symptom questionnaire, this opens the opportunity to select high-risk employees for a targeted intervention to prevent long-term absenteeism

A Cluster-Randomised Trial Evaluating an Intervention for Patients with Stress-Related Mental Disorders and Sick Leave in Primary Care

Objective: Mental health problems often affect functioning to such an extent that they result in sick leave. The worldwide reported prevalence of mental health problems in the working population is 10%–18%. In developed countries, mental health problems are one of the main grounds for receiving disability benefits. In up to 90% of cases the cause is stress-related, and health-care utilisation is mainly restricted to primary care. The aim of this study was to assess the effectiveness of our Minimal Intervention for Stress-related mental disorders with Sick leave (MISS) in primary care, which is intended to reduce sick leave and prevent chronicity of symptoms. Design: Cluster-randomised controlled educational trial. Setting: Primary health-care practices in the Amsterdam area, The Netherlands. Participants: A total of 433 patients (MISS n ¼ 227, usual care [UC] n ¼ 206) with sick leave and self-reported elevated level of distress. Interventions: Forty-six primary care physicians were randomised to either receive training in the MISS or to provide UC. Eligible patients were screened by mail. Outcome Measures: The primary outcome measure was duration of sick leave until lasting full return to work. The secondary outcomes were levels of self-reported distress, depression, anxiety, and somatisation. Results: No superior effect of the MISS was found on duration of sick leave (hazard ratio 1.06, 95% confidence interval 0.87–1.29) nor on severity of self-reported symptoms. Conclusions: We found no evidence that the MISS is more effective than UC in our study sample of distressed patients. Continuing research should focus on the potential beneficial effects of the MISS; we need to investigate which elements of the intervention might be useful and which elements should be adjusted to make the MISS effective

Systematic review of measurement properties of questionnaires measuring somatization in primary care patients

Objective The aim of this review is to critically appraise the evidence on measurement properties of self-report questionnaires measuring somatization in adult primary care patients and to provide recommendations about which questionnaires are most useful for this purpose. Methods We assessed the methodological quality of included studies using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. To draw overall conclusions about the quality of the questionnaires, we conducted an evidence synthesis using predefined criteria for judging the measurement properties. Results We found 24 articles on 9 questionnaires. Studies on the Patient Health Questionnaire-15 (PHQ-15) and the Four-Dimensional Symptom Questionnaire (4DSQ) somatization subscale prevailed and covered the broadest range of measurement properties. These questionnaires had the best internal consistency, test-retest reliability, structural validity, and construct validity. The PHQ-15 also had good criterion validity, whereas the 4DSQ somatization subscale was validated in several languages. The Bodily Distress Syndrome (BDS) checklist had good internal consistency and structural validity. Some evidence was found for good construct validity and criterion validity of the Physical Symptom Checklist (PSC-51) and good construct validity of the Symptom Check-List (SCL-90-R) somatization subscale. However, these three questionnaires were only studied in a small number of primary care studies. Conclusion Based on our findings, we recommend the use of either the PHQ-15 or 4DSQ somatization subscale for somatization in primary care. Other questionnaires, such as the BDS checklist, PSC-51 and the SCL-90-R somatization subscale show promising results but have not been studied extensively in primary care. © 2017 Elsevier Inc

Occupational physicians' perceived barriers and suggested solutions to improve adherence to a guideline on mental health problems:Analysis of a peer group training

Background Despite the impact of mental health problems on sickness absence, only few occupational health guidelines addressing these problems are available. Moreover, adherence has found to be suboptimal. To improve adherence to the Dutch guideline on mental health problems a training was developed for Dutch occupational physicians (OPs) focusing on identifying barriers and addressing them. The aim of this study was to provide an overview of the barriers that OPs perceived in adhering to the Dutch guideline on mental health problems as well as their solutions to overcome them. Methods A qualitative study was conducted using data from the peer group training. Thirty-two (6 groups of 4 to 6) OPs received a multiple-session interactive training over the course of a year, focusing on identifying and addressing barriers, using a Plan-Do-Check-Act approach. Sessions were audio-taped and transcribed verbatim. Thematic content analysis was performed by two researchers with a selection of 50 % (21 out of 42) of the transcripts to identify the perceived barriers and the suggested solutions, using AtlasTi 7.0. Results Knowledge-related barriers were perceived regarding the content of all parts of the guideline. Commonly perceived attitude-related barriers were a lack of self-efficacy to perform certain guideline recommendations and difficulties with changing habits and routines. External barriers that were commonly perceived were work-contextual barriers, such as a lack of time/work pressure, tight contracts between occupational health services (OHSs) and employers, and conflicting policy of and a lack of collaboration with other parties (e.g. employer, other healthcare providers). The most often tested solutions by OPs during the training were sharing information, experiences, tips and tricks and referring to existing tools, or developing new tools to facilitate guideline usage. Conclusions Dutch OPs perceive a range of knowledge-related, attitude-related and external barriers in adhering to the guideline on mental health problems. The tested solutions during the training particularly seemed to focus on knowledge and attitude-related barriers. To optimally implement this or similar mental health guidelines, it may be important to complement guideline training and education of individual or groups of OPs, with interventions that address external barriers such as changing tight contracts, or improving communication and collaboration with other parties. Keywords Mental health, Practice guideline, Occupational medicine, Barriers, Solutions, Implementatio

Effectiveness of an intervention to enhance occupational physicians’ guideline adherence on sickness absence duration in workers with common mental disorders:A cluster-randomized controlled trial

Purpose Evidence-based guidelines in occupational health care improve the quality of care and may reduce sickness absence duration. Notwithstanding that, guideline adherence of occupational physicians (OPs) is limited. Based on the literature on guideline implementation, an intervention was developed that was shown to effectively improve self-reported adherence in OPs. The aim of present study was to evaluate whether this intervention leads to earlier return to work (RTW) in workers with common mental disorders (CMD). Methods In a two-armed cluster randomized controlled trial, 66 OPs were randomized. The trial included 3379 workers, with 1493 in the intervention group and 1886 in the control group. The outcome measures were: time to full RTW, time to first RTW, and total hours of sickness absence. Cox regression analyses and generalized linear mixed model analyses were used for the evaluations. Results The median time to RTW was 154 days among the 3228 workers with CMD. No significant differences occurred in (time to) full RTW between intervention and control group HR 0.96 (95% CI 0.81–1.15) nor for first RTW HR 0.96 (95% CI 0.80–1.15). The mean total hours of sickness absence was 478 h in the intervention group and 483 h in the control group. Conclusions The intervention to enhance OPs’ guideline adherence did not lead to earlier RTW in workers with CMD guided by the OPs. Possible explanations are the remaining external barriers for guideline use, and that perceived guideline adherence might not represent actual guideline adherence and improved care