47 research outputs found
Recommended from our members
Recommending cut scores with a subset of items: An empirical illustration
Many testing programs face the practical challenge of having limited resources to conduct comprehensive standard setting studies. Some researchers have suggested that replicating a group’s recommended cut score on a full-length test may be possible by using a subset of the items. However, these studies were based on simulated data. This study describes a standard setting application using two independent panels providing judgments on a 300-item licensure test. Specifically, one panel provided judgments on all 300 items; whereas the second panel made judgments on a randomly-selected subset of 150 items. Both panels also participated in an alternate standard setting method to evaluate panel comparability. Results suggest caution for practitioners considering using subsets of items for standard setting studies. Accessed 7,224 times on https://pareonline.net from May 11, 2010 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right
Recommended from our members
Evaluating the appropriateness and use of domain critical errors
The consequences associated with the uses and interpretations of scores for many credentialing testing programs have important implications for a range of stakeholders. Within licensure settings specifically, results from examination programs are often one of the final steps in the process of assessing whether individuals will be allowed to enter practice. This article focuses on the concept of domain critical errors and suggests a framework for considering their use in practice. Domain critical errors are defined here as knowledge, skills, abilities, or judgments that are essential to the definition of minimum qualifications in a testing program\u27s pass-fail decision-making process. Using domain critical errors has psychometric and policy implications, particularly for licensure programs that are mandatory for entry-level practice. Because these errors greatly influence pass-fail decisions, the measurement community faces an ongoing challenge to promote defensible practices while concurrently providing assessment literacy development about the appropriate design and use of testing methods like domain critical errors. Accessed 4,769 times on https://pareonline.net from October 17, 2012 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right
Guest Editor's Introduction: Qualitative Inquiries of Participants' Experiences With Standard Setting
Recommending a nursing-specific passing standard for the IELTS examination
Licensure testing programs in the United States (e.g., nursing) face an increasing challenge of measuring the competency of internationally trained candidates, both in relation to their clinical competence and their English language competence. To assist with the latter, professional licensing bodies often adopt well-established and widely available international English language proficiency measures. In this context, the National Council of State Boards of Nursing (NCSBN) sought to develop a nursing-specific passing standard on the International English Language Testing System that U.S. jurisdictions could consider in their licensure decisions for internationally trained candidates. Findings from a standard setting exercise were considered by NCSBN's Examination Committee in conjunction with other relevant information to produce a legally defensible passing standard on the test. This article reports in detail on the standard setting exercise conducted as part of this policy-making process; it describes the techniques adopted, the procedures followed, and the outcomes obtained. The study is contextualized within the current literature on standard setting. The latter part of the article describes the nature of the policy-making process to which the study contributed and discusses some of the implications of including a language literacy test as part of a licensure testing program