Purpose: Continuing education (CE) is intended to help clinicians maintain competence, develop and advance knowledge and skills, and enhance knowledge, skills, and abilities beyond the levels required for entry-level practice. Based on previous literature, the current mode of CE in athletic training does not appear to be helping clinicians maintain competence. The purpose of this research was to validate a comprehensive assessment based on the Role Delineation Study/Practice Analysis (6th ed.) through item analysis and estimates of reliability to be used to assess athletic trainers’ actual knowledge. Method: We conducted an instrumentation validation study using Qualtrics® web-based platform. Athletic trainers (n=191; age=31.5±8.1yrs; years of experience=8.9±11.1yrs) in good standing with the NATA and BOC completed both administrations of the assessment. Six experts developed 220 multiple-choice items for inclusion with broad application across the five domains of clinical practice (Injury/Illness and Wellness Protection [49 items], Clinical Evaluation and Diagnosis [63 items], Immediate and Emergency Care [29 items], Treatment and Rehabilitation [29 items], and Organizational and Professional Health and Wellbeing [50 items]). A random sample of NATA members were recruited via email, received weekly reminders, and then after four weeks, they completed a second administration of the assessment. We evaluated the assessment tool for item difficulty, item discrimination, internal consistency, item total statistics, and test-retest reliability. Results: We eliminated 42 items from the tool created by the experts that were too difficult (0.90). We eliminated 50 additional items due to point-biserial correlations between item performance and total domain score performance below 0.20. We identified additional weaknesses in 57 items through intraclass correlation coefficients (ICCConclusions: We developed a valid and reliable assessment tool to measure athletic trainers’ actual knowledge. Future research should utilize a validated assessment of actual knowledge to guide continuing education activities
