Open Conference Systems, ITC 2016 Conference

Font Size: 
PAPER: Challenges for Testing when Item Exposure is the Norm
Fang Chen

Building: Pinnacle
Room: Cordova-SalonD
Date: 2016-07-02 03:30 PM – 05:00 PM
Last modified: 2016-05-21

Abstract


Item exposure has been traditionally treated a topic for computer-based adaptive testing (CAT). Algorithms are written to control the frequency of exposure during test-assembly and the items are expected to be reused following some criterion. This type of "exposure in design" may survive in some culture. However, in a culture where test-preparation relies on practicing real items, the exposed items have short longevity. Similarly, many practices in the international community may not be appropriate. To explore alternatives, it is necessary to understand how items are exposed and how this affects the testing circle at each stage of testing. In this study, a large-scale high-stake test is traced during and after its recent real-item administration. the route of exposure and speed of spread is analyzed at three time points. Surveys are also given to examine test-preparation practices. Results confirmed the aforementioned concerns. The first item exposure was captured even before the test ended and information of various qualities was found on line as time moves on. In order to facilitate all valid decisions relevant to each test, it is recommended that the test practice adjusted. A framework of a testing system different from the trational design is proposed. Pros and cons are presented and challenges discussed. In all, educational testing is to serve the national goal for progress. It is never a value-free process. While the statistical models provide objective standards, subject-matter-expertise should be involved. This not only benefits the testing circle but also the end purpose of a motivated and healthy educational community at large in a unique culture.

Keywords: large-scale assessment, high-stake test, item exposure, validity


An account with this site is required in order to view papers. Click here to create an account.