Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist

Background: The COSMIN checklist is a standardized tool for assessing the methodological quality of studies on measurement properties. It contains 9 boxes, each dealing with one measurement property, with 5-18 items per box about design aspects and statistical methods. Our aim was to develop a scoring system for the COSMIN checklist to calculate quality scores per measurement property when using the checklist in systematic reviews of measurement properties. Methods: The scoring system was developed based on discussions among experts and testing of the scoring system on 46 articles from a systematic review. Four response options were defined for each COSMIN item (excellent, good, fair, and poor). A quality score per measurement property is obtained by taking the lowest rating of any item in a box ("worst score counts"). Results: Specific criteria for excellent, good, fair, and poor quality for each COSMIN item are described. In defining the criteria, the "worst score counts" algorithm was taken into consideration. This means that only fatal flaws were defined as poor quality. The scores of the 46 articles show how the scoring system can be used to provide an overview of the methodological quality of studies included in a systematic review of measurement properties. Conclusions: Based on experience in testing this scoring system on 46 articles, the COSMIN checklist with the proposed scoring system seems to be a useful tool for assessing the methodological quality of studies included in systematic reviews of measurement properties. © The Author(s) 2011

Measurement Properties of Questionnaires Assessing Complementary and Alternative Medicine Use in Pediatrics: A Systematic Review

Complementary and alternative medicine (CAM) is commonly used by children, but estimates of that use vary widely partly due to the range of questionnaires used to assess CAM use. However, no studies have attempted to appraise measurement properties of these questionnaires. The aim of this systematic review was to critically appraise and summarize measurement properties of questionnaires of CAM use in pediatrics.A search strategy was implemented in major electronic databases in March 2011 and conference websites, scientific journals and experts were consulted. Studies were included if they mentioned a questionnaire assessing the prevalence of CAM use in pediatrics. Members of the team independently rated the methodological quality of the studies (using the COSMIN checklist) and measurement properties of the questionnaires (using the Terwee and Cohen criteria).A total of 96 CAM questionnaires were found in 104 publications. The COSMIN checklist showed that no studies reported adequate methodological quality. The Terwee criteria showed that all included CAM questionnaires had indeterminate measurement properties. According to the Cohen score, none were considered to be a well-established assessment, two approached the level of a well-established assessment, seven were promising assessments and the remainder (n = 87) did not reach the score's minimum standards.None of the identified CAM questionnaires have been thoroughly validated. This systematic review highlights the need for proper validation of CAM questionnaires in pediatrics, which may in turn lead to improved research and knowledge translation about CAM in clinical practice

The translation, validity and reliability of the German version of the Fremantle Back Awareness Questionnaire

Background: The Fremantle Back Awareness Questionnaire (FreBAQ) claims to assess disrupted self-perception of the back. The aim of this study was to develop a German version of the Fre-BAQ (FreBAQ-G) and assess its test-retest reliability, its known-groups validity and its convergent validity with another purported measure of back perception. Methods: The FreBaQ-G was translated following international guidelines for the transcultural adaptation of questionnaires. Thirty-five patients with non-specific CLBP and 48 healthy participants were recruited. Assessor one administered the FreBAQ-G to each patient with CLBP on two separate days to quantify intra-observer reliability. Assessor two administered the FreBaQ-G to each patient on day 1. The scores were compared to those obtained by assessor one on day 1 to assess inter-observer reliability. Known-groups validity was quantified by comparing the FreBAQ-G score between patients and healthy controls. To assess convergent validity, patient\u27s FreBAQ-G scores were correlated to their two-point discrimination (TPD) scores. Results: Intra- and Inter-observer reliability were both moderate with ICC3.1 = 0.88 (95%CI: 0.77 to 0.94) and 0.89 (95%CI: 0.79 to 0.94), respectively. Intra- and inter-observer limits of agreement (LoA) were 6.2 (95%CI: 5.0±8.1) and 6.0 (4.8±7.8), respectively. The adjusted mean difference between patients and controls was 5.4 (95%CI: 3.0 to 7.8, p\u3c0.01). Patient\u27s FreBAQ-G scores were not associated with TPD thresholds (Pearson\u27s r = -0.05, p = 0.79). Conclusions: The FreBAQ-G demonstrated a degree of reliability and known-groups validity. Interpretation of patient level data should be performed with caution because the LoA were substantial. It did not demonstrate convergent validity against TPD. Floor effects of some items of the FreBAQ-G may have influenced the validity and reliability results. The clinimetric properties of the FreBAQ-G require further investigation as a simple measure of disrupted self-perception of the back before firm recommendations on its use can be made

Clinimetric properties of the Turkish translation of a modified neck disability index

<p>Abstract</p> <p>Background</p> <p>Neck pain is a common problem that can greatly affect a person's activities of daily living. Functional status questionnaires are important in assessing this effect, and are used to follow up neck pain management programs. The Neck Disability Index (NDI) is the first-created scale for neck pain-related disability and is widely translated and in common used in many countries. Our aim is investigate to clinometric properties of a Turkish version of modified NDI and to give a choice in daily practise of versions to be used.</p> <p>Methods</p> <p>The modified NDI was applied to 30 patients for reliability. 185 patients participated in the validity study. All patients were recruited from the outpatient clinic of our department. The scale was translated by the forward and backward translation procedure according to the COSMIN criteria. The test was repeated at 48 hours interval for reliability study. SPSS-10.0, software was used for statistical analyses. The Intraclass correlation coefficient was used for the test- retest reliability of the modified NDI. Cronbach α was used for internal consistency. Factor analysis was used for construct validity. The validity of the modified NDI with respect to the SF-36, HAD, VAS pain, VAS disability was assessed using Spearman correlations.</p> <p>Results</p> <p>The Intraclass correlation coefficient between first and second (within 48 hours) evaluation of test (rs) was 0.92. Questions 1,4,6,8,10 were shown to have excellent reliability. (rs > 0.9). Question 10 was the most frequently challenged question because "recreational and social activities" do not have not the same meanings in Turkey than in western countries. This required that detailed explanations be provided by the investigators. Cronbach's alpha for the total index was 0.88. A single factor accounting for 80.2% of the variance was obtained. Validity studies demonstrated good and moderate correlations (rs) among NDI, HAD, VAS, physical function subtitle of SF 36 (0.62, 0.76, 0.68).</p> <p>Conclusions</p> <p>The modified NDI-Turkish version is a reliable and valid test and is suitable for daily practise.</p

Instruments to measure patient experience of healthcare quality in hospitals: a systematic review

Improving and sustaining the quality of hospital care is an international challenge. Patient experience data can be used to target improvement and research. However, the use of patient experience data has been hindered by confusion over multiple instruments (questionnaires) with unknown psychometric testing and utility.MethodsWe conducted a systematic review and utility critique of questionnaires to measure patient experience of healthcare quality in hospitals. Databases (Medical Literature Analysis and Retrieval System (MEDLINE), Cumulative Index to Nursing and Allied Health Literature (CINAHL), Psychological Information (PsychINFO) and Web of Knowledge until end of November 2013) and grey literature were scrutinised. Inclusion criteria were applied to all records with a 10 % sample independently checked. Critique included (1) application of COSMIN checklists to assess the quality of each psychometric study, (2) critique of psychometric results of each study using Terwee et al. criteria and (3) development and critique of additional aspects of utility for each instrument. Two independent reviewers completed each critique. Synthesis included combining findings in a utility matrix.We obtained 1157 records. Of these, 26 papers measuring patient experience of hospital quality of care were identified examining 11 international instruments. We found evidence of extensive theoretical/development work. The quality of methods and results was variable but mostly of a high standard. Additional aspects of utility found that (1) cost efficiency was mostly poor, due to the resource necessary to obtain reliable samples; (2) acceptability of most instruments was good and (3) educational impact was variable, with evidence on the ease of use, for approximately half of the questionnaires.ConclusionsSelecting the right patient experience instrument depends on a balanced consideration of aspects of utility, aided by the matrix. Data required for high stakes purposes requires a high degree of reliability and validity, while those used for quality improvement may tolerate lower levels of reliability in favour of other aspects of utility (educational impact, cost and acceptability)

Measurement of Upper Limb Range of Motion Using Wearable Sensors: A Systematic Review.

Background: Wearable sensors are portable measurement tools that are becoming increasingly popular for the measurement of joint angle in the upper limb. With many brands emerging on the market, each with variations in hardware and protocols, evidence to inform selection and application is needed. Therefore, the objectives of this review were related to the use of wearable sensors to calculate upper limb joint angle. We aimed to describe (i) the characteristics of commercial and custom wearable sensors, (ii) the populations for whom researchers have adopted wearable sensors, and (iii) their established psychometric properties. Methods: A systematic review of literature was undertaken using the following data bases: MEDLINE, EMBASE, CINAHL, Web of Science, SPORTDiscus, IEEE, and Scopus. Studies were eligible if they met the following criteria: (i) involved humans and/or robotic devices, (ii) involved the application or simulation of wearable sensors on the upper limb, and (iii) calculated a joint angle. Results: Of 2191 records identified, 66 met the inclusion criteria. Eight studies compared wearable sensors to a robotic device and 22 studies compared to a motion analysis system. Commercial (n = 13) and custom (n = 7) wearable sensors were identified, each with variations in placement, calibration methods, and fusion algorithms, which were demonstrated to influence accuracy. Conclusion: Wearable sensors have potential as viable instruments for measurement of joint angle in the upper limb during active movement. Currently, customised application (i.e. calibration and angle calculation methods) is required to achieve sufficient accuracy (error < 5°). Additional research and standardisation is required to guide clinical application

Quality-of-life assessment in dementia: the use of DEMQOL and DEMQOL-Proxy total scores

Purpose There is a need to determine whether health-related quality-of-life (HRQL) assessments in dementia capture what is important, to form a coherent basis for guiding research and clinical and policy decisions. This study investigated structural validity of HRQL assessments made using the DEMQOL system, with particular interest in studying domains that might be central to HRQL, and the external validity of these HRQL measurements. Methods HRQL of people with dementia was evaluated by 868 self-reports (DEMQOL) and 909 proxy reports (DEMQOL-Proxy) at a community memory service. Exploratory and confirmatory factor analyses (EFA and CFA) were conducted using bifactor models to investigate domains that might be central to general HRQL. Reliability of the general and specific factors measured by the bifactor models was examined using omega (?) and omega hierarchical (? h) coefficients. Multiple-indicators multiple-causes models were used to explore the external validity of these HRQL measurements in terms of their associations with other clinical assessments. Results Bifactor models showed adequate goodness of fit, supporting HRQL in dementia as a general construct that underlies a diverse range of health indicators. At the same time, additional factors were necessary to explain residual covariation of items within specific health domains identified from the literature. Based on these models, DEMQOL and DEMQOL-Proxy overall total scores showed excellent reliability (? h > 0.8). After accounting for common variance due to a general factor, subscale scores were less reliable (? h < 0.7) for informing on individual differences in specific HRQL domains. Depression was more strongly associated with general HRQL based on DEMQOL than on DEMQOL-Proxy (?0.55 vs ?0.22). Cognitive impairment had no reliable association with general HRQL based on DEMQOL or DEMQOL-Proxy. Conclusions The tenability of a bifactor model of HRQL in dementia suggests that it is possible to retain theoretical focus on the assessment of a general phenomenon, while exploring variation in specific HRQL domains for insights on what may lie at the ‘heart’ of HRQL for people with dementia. These data suggest that DEMQOL and DEMQOL-Proxy total scores are likely to be accurate measures of individual differences in HRQL, but that subscale scores should not be used. No specific domain was solely responsible for general HRQL at dementia diagnosis. Better HRQL was moderately associated with less depressive symptoms, but this was less apparent based on informant reports. HRQL was not associated with severity of cognitive impairment