Font Size: 

One aspect of a test that is especially vulnerable to inquiry is the method by which passing standards are established. Accordingly, a key concern for credentialing organizations is how to implement and maintain processes that produce valid, legally defensible standards. This session compares two standard setting methods used for performance-based assessments (e.g., OSCEs). The Borderline Group Method (BGM) uses a cohort of ‘borderline’ examinees to derive the passing score. This method, however, can introduce unintentional bias in the passing score for small-scale examinations where the number of examinees is small, and in the worst case, the method may not be viable if there are no examinees in the borderline range. The Borderline Regression Method (BRM) compensates for this limitation by employing a linear regression that allows the passing score to be set using the scores from all examinees, not just a subset of examinees.

Objectives: The purpose of this study is to compare the accuracy and reliability of passing scores derived from the BGM and BRM when applied to a small-scale performance-based assessment.

Methods: Passing scores on a high-stakes assessment were derived using both methods. Confidence intervals were used to compare the precision of the passing scores derived from each method.

Results: For all stations, the passing score for the BRM was lower than the BGM and, overall, the pass rate for the BRM was higher than the BGM. More importantly, the 95% confidence intervals were smaller for the passing scores derived using the BRM compared to the BGM.

Conclusions: The passing score derived from the BRM was more accurate than that derived from the BGM. Moreover, this study supports the literature that simply averaging the borderline groups will lead to generally biased results. Overall, one can conclude that the regression approach is advantageous over the more traditional BGM.