Establishing the Reliability and Accuracy of Data Collection for the FIFA 11+ Pre-Participation Study

The FIFA 11+ Pre-Participation study at the University of Vermont is investigating the impact that FIFA 11+ has on injuries to high school athletes. The FIFA 11+ Pre-Participation program has been shown to reduce the incidence of lower extremity injury in elite soccer athletes; however, it is unclear if the program has a similar effect on developing high school athletes or if it can reduce injury in other sports. Recognizing that an athletes’ compliance with the FIFA 11+ program may be directly linked to the effectiveness of the program, an outcome measure that documented compliance was developed. The outcome measure that was developed was a form designed to record pre-participation components of a warm-up by observers for the FIFA 11+ study. The objective of this investigation was to establish the intrarater and interrater reliability and accuracy of this compliance outcome measure. A repeated-measures study design was used to determine the reliability and accuracy of the outcome measure. The examiners who collected data for the FIFA 11+ study were asked to volunteer for this investigation, which involved attending two observation sessions that were two weeks apart. The observation sessions involved watching five warm-up videos, each one about ten minutes long, and then recording what occurred in the warm-up. They used the outcome measure to record their observations of the same five pre-participation warm-ups during each session. The outcome measure had 66 warm-up exercises, or components, that could be recorded. Intraclass Correlation Coefficients (ICCs) were used to determine the intrarater and interrater reliability for each component of the outcome measure. A component with an ICC above 0.60 was considered reliable for this study. The sensitivity and specificity of each component, as well as percent agreement of the examiners with the gold standard examiner for each component were used to determine the accuracy. A component was accurate if above 60% of the observations were in agreement with the gold standard examiner that the component either was or was not present in the warm-up. If any components were proven unreliable or inaccurate the outcome measure was simplified by reducing the number of components. The new components, which were each a result of combining two unreliable components, had ICCs, sensitivity and specificity recalculated as if all observations of either of the original components counted toward the new component. The outcome measure was established to be partially reliable and partially accurate. Out of the 34 components observed there were five components that were intrarater unreliable and 18 components that were interrater unreliable. All of the components that were intrarater unreliable were also interrater unreliable. Only one component was inaccurate with 58% of the observations of that component in agreement with the gold standard examiner’s observations. Of the total 18 unreliable components, seven were combined with another component to simplify the outcome measure

Accuracy and Reliability of Examiners’ Observations of Pre-Practice Warm-Up and FIFA 11+ Injury Prevention Program Exercises

Background: The Fédération Internationale de Football Association (FIFA) 11+ is an injury prevention program that decreases the incidence of lower extremity injuries. The purpose of the current study was to understand what specific exercises prevented injury from occurring. We thus developed and tested a form to identify these exercises. We hypothesize that trained examiners could accurately and reliably use this form to identify and record individual exercises performed during preparticipation warm-up. Methods: A repeated-measures study design was used in this investigation. After observing five prepractice warm-up videos obtained from multiple high schools, 11 examiners observed and recorded performed exercises at two different times. The videos included four soccer teams and one American football team. Accuracy, interexaminer reliability, and intraexaminer reliability were assessed. Sensitivity, specificity, accuracy, and percent agreement with a FIFA 11+ expert were measured for each exercise component. Results: The intraclass correlation coefficients between examiners and individually ranged from 0.22 to 1.00 and 0.58 to 1.00, respectively. Reliability was lowest for exercises with similar movements. The percent agreement across all examiners for individual exercises ranged from 20% to 100%. Additionally, the percent agreement between each examiner and the “gold standard” examiner was high (range, 69.6% to 90.4%). For exercises with similar movements, accuracy and reliability were considerably improved (97%) when combined into one category. Conclusion: We determined that trained examiners with different backgrounds and experience can make accurate and reliable observations of most exercises observed in warm-up programs. Using the proposed form, researchers can accurately record exercises and perform quality and fidelity assessments of warm-up exercise routines