Response to Comment on “Estimating the reproducibility of psychological science”

Gilbert et al. conclude that evidence from the Open Science Collaboration's Reproducibility Project: Psychology indicates high reproducibility, given the study methodology. Their very optimistic assessment is limited by statistical misconceptions and by causal inferences from selectively interpreted, correlational data. Using the Reproducibility Project: Psychology data, both optimistic and pessimistic conclusions about reproducibility are possible, and neither are yet warranted.status: publishe

The International Index of Erectile Function (IIEF)-A Systematic Review of Measurement Properties

Introduction: The International Index of Erectile Function (IIEF) is a patient-reported outcome measure to evaluate erectile dysfunction and other sexual problems in men. Aim: To perform a systematic review of the measurement properties of the 15-item patient-reported outcome measure (IIEF-15) and the shortened 5-item version (IIEF-5). Methods: A systematic search of scientific literature up to April 2018 was performed. Data were extracted and analyzed according to COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) guidelines for structural validity, internal consistency, reliability, measurement error, hypothesis testing for construct validity, and responsiveness. Evidence of measurement properties was categorized into sufficient, insufficient, inconsistent, or indeterminate, and quality of evidence as very high, high, moderate, or low. Results: 40 studies were included. The evidence for criterion validity (of the Erectile Function subscale), and responsiveness of the IIEF-15 was sufficient (high quality), but inconsistent (moderate quality) for structural validity, internal consistency, construct validity, and test-retest reliability. Evidence for structural validity, test-retest reliability, construct validity, and criterion validity of the IIEF-5 was sufficient (moderate quality) but indeterminate for internal consistency, measurement error, and responsiveness. Clinical Implications: Lack of evidence for and evidence not supporting some of the measurement properties of the IIEF-15 and IIEF-5 shows the importance of further research on the validity of these questionnaires in clinical research and clinical practice. Strengths & Limitations: A strength of the current review is the use of predefined guidelines (COSMIN). A limitation of this review is the use of a precise rather than a sensitive search filter regarding measurement properties to identify studies to be included. Conclusion: The IIEF requires more research on structural validity (IIEF-15), internal consistency (IIEF-15 and IIEF-5), construct validity (IIEF-15), measurement error (IIEF-15 and IIEF-5), and responsiveness (IIEF-5). The most pressing matter for future research is determining the unidimensionality of the IIEF-5 and the exact factor structure of the IIEF-15. Neijenhuijs KI, Holtmaat K, Aaronson NK, et al. The International Index of Erectile Function (IIEF)—A Systematic Review of Measurement Properties. J Sex Med 2019;16:1078–1091

The Female Sexual Function Index (FSFI)—A Systematic Review of Measurement Properties

Introduction: The Female Sexual Function Index (FSFI) is a patient-reported outcome measure measuring female sexual dysfunction. The FSFI-19 was developed with 6 theoretical subscales in 2000. In 2010, a shortened version became available (FSFI-6). Aim: To investigate the measurement properties of the FSFI-19 and FSFI-6. Methods: A systematic search was performed of Embase, Medline, and Web of Science for studies that investigated measurement properties of the FSFI-19 or FSFI-6 up to April 2018. Data were extracted and analyzed according to COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) guidelines. Evidence was categorized into sufficient, insufficient, inconsistent, or indeterminate, and quality of evidence as very high, high, moderate, or low. Main Outcome Measures: The Main Outcome Measure is the evidence of a measurement property, and the quality of evidence based on the COSMIN guidelines. Results: 83 studies were included. Concerning the FSFI-19, the evidence for internal consistency was sufficient and of moderate quality. The evidence for reliability was sufficient but of low quality. The evidence for criterion validity was sufficient and of high quality. The evidence for structural validity was inconsistent of low quality. The evidence for construct validity was inconsistent of moderate quality. Concerning the FSFI-6, the evidence for criterion validity was sufficient of moderate quality. The evidence for internal consistency was rated as indeterminate. The evidence for reliability was inconsistent of low quality. The evidence for construct validity was inconsistent of very low quality. No information was available on structural validity of the FSFI-6, and measurement error, responsiveness, and cross-cultural validity of both FSFI-6 and FSFI-19. Clinical Implications: Conflicting and lack of evidence for some of the measurement properties of the FSFI-19 and FSFI-6 indicates the importance of further research on the validity of these patient-reported outcome measures. We advise researchers who use the FSFI-19 to perform confirmatory factor analyses and report the factor structure found in their sample. Regardless of these concerns, the FSFI-19 and FSFI-6 have strong criterion validity. Pragmatically, they are good screening tools for the current definition of female sexual dysfunction. Strength & Limitation: A strong point of the review is the use of predefined guidelines. A limitation is the use of a precise rather than a sensitive search filter. Conclusions: The FSFI requires more research on structural validity (FSFI-19 and FSFI-6), reliability (FSFI-6), construct validity (FSFI-19), measurement error (FSFI-19 and FSFI-6), and responsiveness (FSFI-19 and FSFI-6). Further corroboration of measurement invariance (both across cultures and across subpopulations) in the factor structure of the FSFI-19 is necessary, as well as tests for the unidimensionality of the FSFI-6. Neijenhuijs KI, Hooghiemstra N, Holtmaat K, et al. The Female Sexual Function Index (FSFI)—A Systematic Review of Measurement Properties. J Sex Med 2019;16:640–660

A systematic review of the measurement properties of the European Organisation for Research and Treatment of Cancer In-patient Satisfaction with Care Questionnaire, the EORTC IN-PATSAT32

PURPOSE: The EORTC IN-PATSAT32 is a patient-reported outcome measure (PROM) to assess cancer patients' satisfaction with in-patient health care. The aim of this study was to investigate whether the initial good measurement properties of the IN-PATSAT32 are confirmed in new studies. METHODS: Within the scope of a larger systematic review study (Prospero ID 42017057237), a systematic search was performed of Embase, Medline, PsycINFO, and Web of Science for studies that investigated measurement properties of the IN-PATSAT32 up to July 2017. Study quality was assessed, data were extracted, and synthesized according to the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology. RESULTS: Nine studies were included in this review. The evidence on reliability and construct validity were rated as sufficient and of the quality of the evidence as moderate. The evidence on structural validity was rated as insufficient and of low quality. The evidence on internal consistency was indeterminate. Measurement error, responsiveness, criterion validity, and cross-cultural validity were not reported in the included studies. Measurement error could be calculated for two studies and was judged indeterminate. CONCLUSION: In summary, the IN-PATSAT32 performs as expected with respect to reliability and construct validity. No firm conclusions can be made yet whether the IN-PATSAT32 also performs as well with respect to structural validity and internal consistency. Further research on these measurement properties of the PROM is therefore needed as well as on measurement error, responsiveness, criterion validity, and cross-cultural validity. For future studies, it is recommended to take the COSMIN methodology into account

A systematic review of the measurement properties of the Body Image Scale (BIS) in cancer patients

Introduction: Body image is acknowledged as an important aspect of health-related quality of life in cancer patients. The Body Image Scale (BIS) is a patient-reported outcome measure (PROM) to evaluate body image in cancer patients. The aim of this study was to systematically review measurement properties of the BIS among cancer patients. Methods: A search in Embase, MEDLINE, PsycINFO, and Web of Science was performed to identify studies that investigated measurement properties of the BIS (Prospero ID 42017057237). Study quality was assessed (excellent, good, fair, poor), and data were extracted and analyzed according to the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology on structural validity, internal consistency, reliability, measurement error, hypothesis testing for construct validity, and responsiveness. Evidence was categorized into sufficient, insufficient, inconsistent, or indeterminate. Results: Nine studies were included. Evidence was sufficient for structural validity (one factor solution), internal consistency (α = 0.86–0.96), and reliability (r > 0.70); indeterminate for measurement error (information on minimal important change lacked) and responsiveness (increasing body image disturbance in only one study); and inconsistent for hypothesis testing (conflicting results). Quality of the evidence was moderate to low. No studies reported on cross-cultural validity. Conclusion: The BIS is a PROM with good structural validity, internal consistency, and test-retest reliability, but good quality studies on the other measurement properties are needed to optimize evidence. It is recommended to include a wider variety of cancer diagnoses and treatment modalities in these future studies

Measuring health-related quality of life in colorectal cancer patients: systematic review of measurement properties of the EORTC QLQ-CR29

Introduction The EORTC QLQ-CR29 is a patient-reported outcome measure to evaluate health-related quality of life among colorectal cancer patients in research and clinical practice. The aim of this systematic review was to investigate whether the initial positive results regarding the measurement properties of the QLQ-CR29 are confirmed in subsequent studies. Methods A systematic search of Embase, Medline, PsycINFO, and Web of Science was conducted to identify studies investigating the measurement properties of the QLQ-CR29 published up to January 2019. For the 11 included studies, data were extracted, methodological quality was assessed, results were synthesized, and evidence was graded according to the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology on the measurement properties: structural validity, internal consistency, reliability, measurement error, construct validity (hypothesis testing, including known-group comparison, convergent and divergent validity), cross-cultural validity, and responsiveness. Results Internal consistency was rated as “sufficient,” with low evidence. Reliability was rated as “insufficient,” with moderate evidence. Construct validity (hypothesis testing; known-group comparison, convergent and divergent validity) was rated as “inconsistent,” with moderate evidence. Structural validity, measurement error, and responsiveness were rated as “indeterminate” and could therefore not be graded. Conclusion This review indicates that current evidence supporting the measurement properties of the QLQ-CR29 is limited. Additionally, better quality research is needed, taking into account the COSMIN methodology

Measuring health-related quality of life in colorectal cancer patients: systematic review of measurement properties of the EORTC QLQ-CR29

INTRODUCTION: The EORTC QLQ-CR29 is a patient-reported outcome measure to evaluate health-related quality of life among colorectal cancer patients in research and clinical practice. The aim of this systematic review was to investigate whether the initial positive results regarding the measurement properties of the QLQ-CR29 are confirmed in subsequent studies. METHODS: A systematic search of Embase, Medline, PsycINFO, and Web of Science was conducted to identify studies investigating the measurement properties of the QLQ-CR29 published up to January 2019. For the 11 included studies, data were extracted, methodological quality was assessed, results were synthesized, and evidence was graded according to the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology on the measurement properties: structural validity, internal consistency, reliability, measurement error, construct validity (hypothesis testing, including known-group comparison, convergent and divergent validity), cross-cultural validity, and responsiveness. RESULTS: Internal consistency was rated as "sufficient," with low evidence. Reliability was rated as "insufficient," with moderate evidence. Construct validity (hypothesis testing; known-group comparison, convergent and divergent validity) was rated as "inconsistent," with moderate evidence. Structural validity, measurement error, and responsiveness were rated as "indeterminate" and could therefore not be graded. CONCLUSION: This review indicates that current evidence supporting the measurement properties of the QLQ-CR29 is limited. Additionally, better quality research is needed, taking into account the COSMIN methodology