Establishing the Reliability and Accuracy of Data Collection for the FIFA 11+ Pre-Participation Study

The FIFA 11+ Pre-Participation study at the University of Vermont is investigating the impact that FIFA 11+ has on injuries to high school athletes. The FIFA 11+ Pre-Participation program has been shown to reduce the incidence of lower extremity injury in elite soccer athletes; however, it is unclear if the program has a similar effect on developing high school athletes or if it can reduce injury in other sports. Recognizing that an athletes’ compliance with the FIFA 11+ program may be directly linked to the effectiveness of the program, an outcome measure that documented compliance was developed. The outcome measure that was developed was a form designed to record pre-participation components of a warm-up by observers for the FIFA 11+ study. The objective of this investigation was to establish the intrarater and interrater reliability and accuracy of this compliance outcome measure. A repeated-measures study design was used to determine the reliability and accuracy of the outcome measure. The examiners who collected data for the FIFA 11+ study were asked to volunteer for this investigation, which involved attending two observation sessions that were two weeks apart. The observation sessions involved watching five warm-up videos, each one about ten minutes long, and then recording what occurred in the warm-up. They used the outcome measure to record their observations of the same five pre-participation warm-ups during each session. The outcome measure had 66 warm-up exercises, or components, that could be recorded. Intraclass Correlation Coefficients (ICCs) were used to determine the intrarater and interrater reliability for each component of the outcome measure. A component with an ICC above 0.60 was considered reliable for this study. The sensitivity and specificity of each component, as well as percent agreement of the examiners with the gold standard examiner for each component were used to determine the accuracy. A component was accurate if above 60% of the observations were in agreement with the gold standard examiner that the component either was or was not present in the warm-up. If any components were proven unreliable or inaccurate the outcome measure was simplified by reducing the number of components. The new components, which were each a result of combining two unreliable components, had ICCs, sensitivity and specificity recalculated as if all observations of either of the original components counted toward the new component. The outcome measure was established to be partially reliable and partially accurate. Out of the 34 components observed there were five components that were intrarater unreliable and 18 components that were interrater unreliable. All of the components that were intrarater unreliable were also interrater unreliable. Only one component was inaccurate with 58% of the observations of that component in agreement with the gold standard examiner’s observations. Of the total 18 unreliable components, seven were combined with another component to simplify the outcome measure

Physical outcome measure for critical care patients following intensive care discharge

Introduction: The aim of this study was to evaluate the most suitable physical outcome measures to be used with critical care patients following discharge. ICU survivors experience physical problems such as reduced exercise capacity and intensive care acquired weakness. NICE guideline ‘Rehabilitation after critical illness’ (1) recommends the use of outcome measures however does not provide any specific guidance. A recent Cochrane review noted wide variability in measures used following ICU discharge (2). Methods: Discharged ICU patients attended a five week multidisciplinary programme. Patients’ physical function was assessed during the programme, at 6 months and 12 months post discharge. Three outcome measures were included in the initial two cohorts. The Six Minute Walk Test (6MWT) and the Incremental Shuttle Walk test (ISWT) were chosen as they have been used within the critical care follow up setting (2). The Chester Step Test (CST) is widely thought to be a good indicator of ability to return to work (one of the programmes primary aims). Ethics approval was waived as the programme was part of a quality improvement initiative. Results: Data was collected for the initial patients attending the programme (n = 13), median age was 52 (IQR = 38-72), median ICU LOS was 19 days (IQR = 4-91), median APACHE II was 23 (IQR = 19-41) and 11 were men. One patient was so physically debilitated that the CST or ISWT could not be completed however a score was achieved using the 6MWT. Another patient almost failed to achieve level 1 of the ISWT. Subsequent patients for this project (total n = 47) have all therefore been tested using the 6MWT. Good inter-rater and intrarater reliability and validity have been reported for the 6MWT (3). Conclusions: Exercise capacity measurement is not achievable for some patients with either the ISWT or the CST due to the severity of their physical debilitation. Anxiety, post-traumatic stress disorder and depression are common psychological problems post discharge (4), therefore using a test with a bleep is not appropriate. Therefore, the 6MWT is the most appropriate physical outcome measure to be used with critical care patients post discharge

Contemporary outcome measures in acute stroke research: choice of primary outcome measure

BACKGROUND AND PURPOSE: The diversity of available outcome measures for acute stroke trials is challenging and implies that the scales may be imperfect. To assist researchers planning trials and to aid interpretation, this article reviews and makes recommendations on the available choices of scales. The aim is to identify an approach that will be universally accepted and that should be included in most acute trials, without seeking to restrict options for special circumstances. METHODS: The article considers outcome measures that have been widely used or are currently advised. It examines desirable properties for outcome measures such as validity, relevance, responsiveness, statistical properties, availability of training, cultural and language issues, resistance to comorbidity, as well as potential weaknesses. Tracking and agreement among outcomes are covered. RESULTS: Typical ranges of scores for the common scales are described, along with their statistical properties, which in turn influence optimal analytic techniques. The timing of recovery on scores and usual practice in trial design are considered. CONCLUSIONS: The preferred outcome measure for acute trials is the modified Rankin Scale, assessed at 3 months after stroke onset or later. The interview should be conducted by a certified rater and should involve both the patient and any relevant caregiver. Incremental benefits at any level of the modified Rankin Scale may be acceptable. The modified Rankin Scale is imperfect but should be retained in its present form for comparability with existing treatment comparisons. No second measure should be required, but correlations with supporting scales may be used to confirm consistency in direction of effects on other measures

A new outcome measure for cost-utility analyses of screening programs

We show that, under some plausible assumptions, the gain in QALYs a screening program offers is a positive linear transformation of the program's sensitivity level. This result simplifies considerably the cost-utility analysis of mutually exclusive screening programs.

A NEW OUTCOME MEASURE FOR COST-UTILITY ANALYSES OF SCREENING PROGRAMS

In this paper we provide a new outcome measure for the cost-utility analyses of alternative screening programs of a particular disease. We show that for non-invasive screening programs satisfying plausible assumptions, QALYs can be replaced by a simpler outcome: the sensitivity of the program. In other words, the cost-utility analysis can be made without computing the utility each program offers. Consequently, results would be immune to two of the most controversial issues in the cost-utility analysis approach: the elicitation method to obtain quality weights of health profiles, and the discount rate for future health benefits. The assumptions are particularly suitable in the case of selecting between the universal and the selective implementation of a non-invasive screening program. Therefore, we apply our result to provide an additional viewpoint in the current debate about the implementation of a universal or selective newborn screening program to detect congenital hearing impairment.Cost-utility analysis, cost-sensitivity ratios, screening programs,

The SF36 as an outcome measure of services for end stage renal failure

OBJECTIVE: —To evaluate the use of the short form 36 (SF36) as a measure of health related quality of life of patients with end stage renal failure, document the results, and investigate factors, including mode of treatment, which may influence it. DESIGN: Cross sectional survey of patients with end stage renal failure, with the standard United Kingdom version of the SF36 supplemented by specific questions for end stage renal failure. SETTING: A teaching hospital renal unit. Subjects and methods—660 patients treated at the Sheffield Kidney Institute by haemodialysis, peritoneal dialysis, and transplantation. Internal consistency, percentage of maximal or minimal responses, SF36 scores, effect sizes, correlations between independent predictor variables and individual dimension scores of the SF36. Multiple regression analysis of the SF36 scores for the physical functioning, vitality, and mental health dimensions against treatment, age, risk (comorbidity) score, and other independent variables. RESULTS: A high response rate was achieved. Internal consistency was good. There were no floor or ceiling effects other than for the two “role” dimensions. Overall health related quality of life was poor compared with the general population. Having a functioning transplant was a significant predictor of higher score in the three dimensions (physical functioning, vitality, and mental health) for which multiple regression models were constructed. Age, sex, comorbidity, duration of treatment, level of social and emotional support, household numbers, and hospital dialysis were also (variably) significant predictors. CONCLUSIONS: The SF36 is a practical and consistent questionnaire in this context, and there is evidence to support its construct validity. Overall the health related quality of life of these patients is poor, although transplantation is associated with higher scores independently of the effect of age and comorbidity. Age, comorbidity, and sex are also predictive of the scores attained in the three dimensions studied. Further studies are required to ascertain whether altering those predictor variables which are under the influence of professional carers is associated with changes in health related quality of life, and thus confirm the value of this outcome as a measure of quality of care

Using 'dead or dependent' as an outcome measure in clinical trials in Parkinson's disease

Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.Peer reviewedPublisher PD