Introduction

Item position effects have been an important concern in educational and psychological measurement.  This type of effect may occur in both paper-pencil tests and computer-based assessments when examinees receive the same test items at different positions.  If there is a significant item position effect for the items, this may lead to an unfair assessment.

Objectives

Previously, item position effects were often examined using Hierarchical Generalized Linear Mixed Model (HGLM), which is computationally burdensome for large data, and is mathematically limited to Rasch model.  In this study, we aim to introduce a Structural Equation Modeling (SEM) approach to overcome the disadvantages of HGLM, and to demonstrate the proposed method using data from an operational large-scale reading assessment.

Methodology

The data come from a statewide reading assessment in the US. The sample consisted of 11734 3^rd grade students who responded to 45 reading items related to 7 reading passages. Because the test was given in computers, item positions were scrambled across students, which resulted in four different patterns of item positions (referred to as test forms in this study). We examined the overall form effects, passage position effects, and item position effects using the SEM approach.

Results

The results showed that one of the 7 passages and 10 of the 45 items showed significant position effects in the test, although there was no overall form effect detected across the four forms.  All SEM models converged in less than 2 minutes, whereas their HGLM counterparts either took more than 25 minutes or failed to converge.

Conclusion

This study contributes to the literature by introducing a flexible SEM approach to estimate position effects.  Compared to HGLM, the SEM approach is computationally more efficient, easier to interpret, and allows the examination of position effects not only for Rasch model but also 2PL model.