Developing high-quality test items is a costly and time-consuming endeavor, yet the need for large item banks increases as testing becomes more frequent and as computer-adaptive tests call for many items of varying difficulty. To address this need, some test developers have begun to implement automatic item generation (AIG) to supplement item writing efforts. Research and practice on AIG continue to grow as technology and psychometric methods develop, but the field lacks content-specific frameworks for implementing AIG. A key step of AIG is developing item models which function as templates for item generation. Despite item model creation serving a critical methodological component of AIG, no published studies or articles exist, to my knowledge, that describe the principles or standards used to create item models. Instead, researchers merely present the cognitive model and item models they used without describing how those item models were created, raising a concern about the theory and research – or lack thereof – used to arrive at item models for AIG.

The purpose of this paper is to present a framework for creating AIG item models to assess middle school students’ algebraic reasoning. The framework, which is based on a comprehensive literature review of AIG methodology and the cognitive complexity of algebra tasks, includes five stages of item model development which guide an AIG user through a scientific process of writing item models for AIG. The goal of the framework is to produce item models that can be used to generate unique, heterogeneous items of predicted difficulties by systematically manipulating features of the items.  This paper describes the framework and concurrently demonstrates the framework’s application with an example of algebra items. The paper concludes by proposing methods for evaluating the framework and discussing its limitations.