Abstract Computer-based testing with many credentialing examination agencies has become a common occurence. At the same time, selecting a test design is difficult because several are available-parallel-forms, computer-adaptive (CAT), and multi-stage (MST), and the merits of these designs interact with exam conditions. These conditions include item quality, bank size, candidate score distribution, placement of the passing score, exam length, and more. In this study three popular computer-based test designs under some common examination conditions were investigated using computer simulation techniques. Item quality and bank size were varied. The results from the study were clear: both item bank size and item quality had a practically significant impact on decision consistency and accuracy. Interestingly, even in nearly ideal situations, the choice of test design was not a factor in the results. Two conclusions seem to follow from the findings: (1) more time and resources should be committed to expanding both the size and quality of item banks, and (2) designs that individualize an exam administration such as MST and CAT, may not be especially helpful when the primary purpose of an examination is to make pass-fail decisions, and conditions are present for using parallel-forms of examinations with a target information function that can be centered at the passing score. Obviously, the validity of these conclusions needs to be thoroughly checked with additional simulations and real data
