Introduction

As demographic shifts in a country’s population occur due to immigration and/or student mobility, the proportion of minority language speakers may increase.  If adjustments are not made to an assessment and its items in recognition of these shifts, the potential may exist for increased numbers of items to function differentially between the majority and minority language test takers.  Typically, such items are discarded. The process of replacing them with new items could be costly, therefore, the question must be asked­–are these differences due to construct-irrelevant variance (e.g., due to the use of unfamiliar terms, idioms, etc.)?  If not, discarding them may be unnecessary.  As a mitigation strategy, we propose applying an evidence center design (ECD) approach (Author, 2006) to item revisions to reduce the cost of new item development and guide the revision of existing items to reduce their possible sources of construct irrelevance while preserving the item.

Objectives and Methodology

ECD extends beyond a domain analysis, which focuses on defining the construct and how knowledge of that construct is represented.  The goal of this presentation is to focus on the domain modeling layer of ECD and discuss how it can be implemented as part of the test/item revision process.  Specifically, we will discuss how to represent the construct by defining it in terms of an examinee’s knowledge/skills/abilities.   In the case of items with DIF, we will demonstrate by example how minor changes to the language of an item or the context/setting of an item can be used to remove possible sources of construct-irrelevant variance.

Conclusions

The use of an ECD approach may not only benefit the revision of items, but further strengthen test developers’ knowledge of an assessed domain or construct, by taking linguistic and cultural minorities into consideration–and be potentially useful throughout test development.

References

Author (2006). Implications of Evidenceâ€Centered Design for Educational Testing. Educational Measurement: Issues and Practice, 25(4), 6-20.