Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist

Background: The COSMIN checklist is a standardized tool for assessing the methodological quality of studies on measurement properties. It contains 9 boxes, each dealing with one measurement property, with 5-18 items per box about design aspects and statistical methods. Our aim was to develop a scoring system for the COSMIN checklist to calculate quality scores per measurement property when using the checklist in systematic reviews of measurement properties. Methods: The scoring system was developed based on discussions among experts and testing of the scoring system on 46 articles from a systematic review. Four response options were defined for each COSMIN item (excellent, good, fair, and poor). A quality score per measurement property is obtained by taking the lowest rating of any item in a box ("worst score counts"). Results: Specific criteria for excellent, good, fair, and poor quality for each COSMIN item are described. In defining the criteria, the "worst score counts" algorithm was taken into consideration. This means that only fatal flaws were defined as poor quality. The scores of the 46 articles show how the scoring system can be used to provide an overview of the methodological quality of studies included in a systematic review of measurement properties. Conclusions: Based on experience in testing this scoring system on 46 articles, the COSMIN checklist with the proposed scoring system seems to be a useful tool for assessing the methodological quality of studies included in systematic reviews of measurement properties. © The Author(s) 2011

The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: A clarification of its content

<p>Abstract</p> <p>Background</p> <p>The COSMIN checklist (COnsensus-based Standards for the selection of health status Measurement INstruments) was developed in an international Delphi study to evaluate the methodological quality of studies on measurement properties of health-related patient reported outcomes (HR-PROs). In this paper, we explain our choices for the design requirements and preferred statistical methods for which no evidence is available in the literature or on which the Delphi panel members had substantial discussion.</p> <p>Methods</p> <p>The issues described in this paper are a reflection of the Delphi process in which 43 panel members participated.</p> <p>Results</p> <p>The topics discussed are internal consistency (relevance for reflective and formative models, and distinction with unidimensionality), content validity (judging relevance and comprehensiveness), hypotheses testing as an aspect of construct validity (specificity of hypotheses), criterion validity (relevance for PROs), and responsiveness (concept and relation to validity, and (in) appropriate measures).</p> <p>Conclusions</p> <p>We expect that this paper will contribute to a better understanding of the rationale behind the items, thereby enhancing the acceptance and use of the COSMIN checklist.</p

Measurement Properties of Questionnaires Assessing Complementary and Alternative Medicine Use in Pediatrics: A Systematic Review

Complementary and alternative medicine (CAM) is commonly used by children, but estimates of that use vary widely partly due to the range of questionnaires used to assess CAM use. However, no studies have attempted to appraise measurement properties of these questionnaires. The aim of this systematic review was to critically appraise and summarize measurement properties of questionnaires of CAM use in pediatrics.A search strategy was implemented in major electronic databases in March 2011 and conference websites, scientific journals and experts were consulted. Studies were included if they mentioned a questionnaire assessing the prevalence of CAM use in pediatrics. Members of the team independently rated the methodological quality of the studies (using the COSMIN checklist) and measurement properties of the questionnaires (using the Terwee and Cohen criteria).A total of 96 CAM questionnaires were found in 104 publications. The COSMIN checklist showed that no studies reported adequate methodological quality. The Terwee criteria showed that all included CAM questionnaires had indeterminate measurement properties. According to the Cohen score, none were considered to be a well-established assessment, two approached the level of a well-established assessment, seven were promising assessments and the remainder (n = 87) did not reach the score's minimum standards.None of the identified CAM questionnaires have been thoroughly validated. This systematic review highlights the need for proper validation of CAM questionnaires in pediatrics, which may in turn lead to improved research and knowledge translation about CAM in clinical practice

Cross-cultural adaptation and construct validity of the German version of the Adult Social Care Outcomes Toolkit for service users (German ASCOT)

Background: There has been considerable interest in using the Adult Social Care Outcomes Toolkit (ASCOT), developed in England, to measure quality-of-life outcomes of long-term care (LTC-QoL) service provision in national and cross-national studies. Objectives: The aim of this study was to translate and culturally adapt the original ASCOT service user measure into German and to evaluate its content and construct validity in Austrian home care service users. Methods: The translation and cultural adaptation process followed the ISPOR TCA guidelines. We used qualitative data from six cognitive debriefing interviews with Austrian recipients of home care services to assess linguistic and content validity. In addition, cross-sectional survey data (n = 633) were used to evaluate construct validity by testing hypothesized associations established in a previous study for the original English ASCOT service user instrument. Results: Cognitive debriefing interviews confirmed that the German adaptation of the ASCOT service user instrument was understood as intended, although two domains (‘Control over daily life’ and ‘Dignity’) and selected phrases of the response options were challenging to translate into German. All ASCOT domains were statistically significantly associated with related constructs and sensitive to service user sub-group differences. Conclusions: We found good evidence for a valid cross-cultural adaptation of the German version of ASCOT for service users. The analysis also supports the construct validity of the translated instrument and its use in evaluations of QoL-effects of LTC service provision in German-speaking countries. Further research on the reliability and feasibility in different care settings is encouraged

The translation, validity and reliability of the German version of the Fremantle Back Awareness Questionnaire

Background: The Fremantle Back Awareness Questionnaire (FreBAQ) claims to assess disrupted self-perception of the back. The aim of this study was to develop a German version of the Fre-BAQ (FreBAQ-G) and assess its test-retest reliability, its known-groups validity and its convergent validity with another purported measure of back perception. Methods: The FreBaQ-G was translated following international guidelines for the transcultural adaptation of questionnaires. Thirty-five patients with non-specific CLBP and 48 healthy participants were recruited. Assessor one administered the FreBAQ-G to each patient with CLBP on two separate days to quantify intra-observer reliability. Assessor two administered the FreBaQ-G to each patient on day 1. The scores were compared to those obtained by assessor one on day 1 to assess inter-observer reliability. Known-groups validity was quantified by comparing the FreBAQ-G score between patients and healthy controls. To assess convergent validity, patient\u27s FreBAQ-G scores were correlated to their two-point discrimination (TPD) scores. Results: Intra- and Inter-observer reliability were both moderate with ICC3.1 = 0.88 (95%CI: 0.77 to 0.94) and 0.89 (95%CI: 0.79 to 0.94), respectively. Intra- and inter-observer limits of agreement (LoA) were 6.2 (95%CI: 5.0±8.1) and 6.0 (4.8±7.8), respectively. The adjusted mean difference between patients and controls was 5.4 (95%CI: 3.0 to 7.8, p\u3c0.01). Patient\u27s FreBAQ-G scores were not associated with TPD thresholds (Pearson\u27s r = -0.05, p = 0.79). Conclusions: The FreBAQ-G demonstrated a degree of reliability and known-groups validity. Interpretation of patient level data should be performed with caution because the LoA were substantial. It did not demonstrate convergent validity against TPD. Floor effects of some items of the FreBAQ-G may have influenced the validity and reliability results. The clinimetric properties of the FreBAQ-G require further investigation as a simple measure of disrupted self-perception of the back before firm recommendations on its use can be made

The assessment of depression in people with multiple sclerosis : a systematic review of psychometric validation studies

Background: The prevalence of depression in people with multiple sclerosis (PwMS) is high; however, symptoms common to both conditions makes measurement difficult. There is no high quality overview of validation studies to guide the choice of depression inventory for this population. Methods: A systematic review of studies validating the use of generic depression inventories in people with MS was conducted using MEDLINE and PsycINFO. Studies validating the use of depression inventories in PwMS and published in English were included; validation studies of tests for cognitive function and general mental health were excluded. Eligible studies were then quality assessed using the COSMIN checklist and findings synthesised narratively by instrument and validity domain. Results: Twenty-one studies (N=5,991 PwMS) evaluating 12 instruments were included in the review. Risk of bias varied greatly between instrument and validity domain. Conclusions: The review of validation studies was constrained by poor quality reporting and outcome reporting bias. Well-conducted evaluations of some instruments are unavailable for some validity domains. This systematic review provides an evidence base for trade-offs in the selection of an instrument for assessing self-reported symptoms of depression in research or clinical practice involving people with MS. We make detailed and specific recommendations for where further research is needed. Registration: PROSPERO CRD42014010597 Keywords Depression; Multiple Sclerosis; Reproducibility of Results; Psychometrics; Chronic Diseas

Semantics in active surveillance for men with localized prostate cancer - results of a modified Delphi consensus procedure

Active surveillance (AS) is broadly described as a management option for men with low-risk prostate cancer, but semantic heterogeneity exists in both the literature and in guidelines. To address this issue, a panel of leading prostate cancer specialists in the field of AS participated in a consensus-forming project using a modified Delphi method to reach international consensus on definitions of terms related to this management option. An iterative three-round sequence of online questionnaires designed to address 61 individual items was completed by each panel member. Consensus was considered to be reached if >= 70% of the experts agreed on a definition. To facilitate a common understanding among all experts involved and resolve potential ambiguities, a face-to-face consensus meeting was held between Delphi survey rounds two and three. Convenience sampling was used to construct the panel of experts. In total, 12 experts from Australia, France, Finland, Italy, the Netherlands, Japan, the UK, Canada and the USA participated. By the end of the Delphi process, formal consensus was achieved for 100% (n = 61) of the terms and a glossary was then developed. Agreement between international experts has been reached on relevant terms and subsequent definitions regarding AS for patients with localized prostate cancer. This standard terminology could support multidisciplinary communication, reduce the extent of variations in clinical practice and optimize clinical decision making.Peer reviewe

Clinimetric properties of the Turkish translation of a modified neck disability index

<p>Abstract</p> <p>Background</p> <p>Neck pain is a common problem that can greatly affect a person's activities of daily living. Functional status questionnaires are important in assessing this effect, and are used to follow up neck pain management programs. The Neck Disability Index (NDI) is the first-created scale for neck pain-related disability and is widely translated and in common used in many countries. Our aim is investigate to clinometric properties of a Turkish version of modified NDI and to give a choice in daily practise of versions to be used.</p> <p>Methods</p> <p>The modified NDI was applied to 30 patients for reliability. 185 patients participated in the validity study. All patients were recruited from the outpatient clinic of our department. The scale was translated by the forward and backward translation procedure according to the COSMIN criteria. The test was repeated at 48 hours interval for reliability study. SPSS-10.0, software was used for statistical analyses. The Intraclass correlation coefficient was used for the test- retest reliability of the modified NDI. Cronbach α was used for internal consistency. Factor analysis was used for construct validity. The validity of the modified NDI with respect to the SF-36, HAD, VAS pain, VAS disability was assessed using Spearman correlations.</p> <p>Results</p> <p>The Intraclass correlation coefficient between first and second (within 48 hours) evaluation of test (rs) was 0.92. Questions 1,4,6,8,10 were shown to have excellent reliability. (rs > 0.9). Question 10 was the most frequently challenged question because "recreational and social activities" do not have not the same meanings in Turkey than in western countries. This required that detailed explanations be provided by the investigators. Cronbach's alpha for the total index was 0.88. A single factor accounting for 80.2% of the variance was obtained. Validity studies demonstrated good and moderate correlations (rs) among NDI, HAD, VAS, physical function subtitle of SF 36 (0.62, 0.76, 0.68).</p> <p>Conclusions</p> <p>The modified NDI-Turkish version is a reliable and valid test and is suitable for daily practise.</p