The purpose of this study was to evaluate a method for calculating the change in item difficulty over time for continuous testing programs. Item difficulty was evaluated from a classical test theory framework in which several adjusted p-values were calculated for each item. The method is applicable to IRT difficulty estimates and thusly can be classified as an item parameter drift method.

The data were from 2 high volume examinations administered continuously. The change in difficulty was calculated for each item bi-weekly. In total, 26 bi-weekly item difficulty estimates were calculated, each of which used the data from at least 100 examinees and typically from over 1,000 examinees. From there, the correlation between the item’s difficulty and time was calculated. As a comparison point, a traditional way of evaluating item difficulty change was calculated. A before and after adjusted p-value were calculated for each item, each based on six months of data.

A contingency table will document the number of items that significantly change via each method. Attempts will be made to identify patterns in the item difficulty estimates for items within each cell.

The analyses for this study are scheduled to be complete in April 2016. We anticipate that items which gradually and consistently change will be identified by only the weekly estimates while those that have erratic but dramatic changes will be identified by only the traditional method.

The results of this study will be important because they provide an alternative to the traditional way of evaluating change. Although the traditional approach is legitimate, if fails to utilize time as a way of partitioning the variance inherent in the data obtained from continuous testing programs. Given the recent advances in test result management systems and psychometric software, it is very reasonable to implement this method today.