Rasch scaling procedures for informing development of a valid Fetal Surveillance Education Program multiple-choice assessment

<p>Abstract</p> <p>Background</p> <p>It is widely recognised that deficiencies in fetal surveillance practice continue to contribute significantly to the burden of adverse outcomes. This has prompted the development of evidence-based clinical practice guidelines by the Royal Australian and New Zealand College of Obstetricians and Gynaecologists and an associated Fetal Surveillance Education Program to deliver the associated learning. This article describes initial steps in the validation of a corresponding multiple-choice assessment of the relevant educational outcomes through a combination of item response modelling and expert judgement.</p> <p>Methods</p> <p>The Rasch item response model was employed for item and test analysis and to empirically derive the substantive interpretation of the assessment variable. This interpretation was then compared to the hierarchy of competencies specified a priori by a team of eight subject-matter experts. Classical Test Theory analyses were also conducted.</p> <p>Results</p> <p>A high level of agreement between the hypothesised and derived variable provided evidence of construct validity. Item and test indices from Rasch analysis and Classical Test Theory analysis suggested that the current test form was of moderate quality. However, the analyses made clear the required steps for establishing a valid assessment of sufficient psychometric quality. These steps included: increasing the number of items from 40 to 50 in the first instance, reviewing ineffective items, targeting new items to specific content and difficulty gaps, and formalising the assessment blueprint in light of empirical information relating item structure to item difficulty.</p> <p>Conclusion</p> <p>The application of the Rasch model for criterion-referenced assessment validation with an expert stakeholder group is herein described. Recommendations for subsequent item and test construction are also outlined in this article.</p

The Development of Core Competencies for the Practice of Marriage and Family Therapy

In response to a series of national policy reports regarding what has been termed the quality chasm in health and mental health care in the United States, in January 2003, the American Association for Marriage and Family Therapy convened a task force to develop core competencies (CC) for the practice of marriage and family therapy (MFT). The task force also was responding to a call for outcome-based education and for the need to answer questions about what marriage and family therapists do. Development of the CC moves the field of MFT into a leading-edge position in mental health. This article describes the development of the CC, outcomes of the development process for the competencies, and recommendations for their continued development and implementation