Determining if a test is doing what it was designed to do involves understanding consequences of test results used for decision-making. Each test score use (e.g., improving policy and practice or informing language-based selection practices) is associated with consequences for one or more stakeholders (e.g., test takers, institutions, schools, and the public). While some consequences are intended, some are unintended. And while some unintended consequences may be positive, it is more likely the case that unintended consequences negatively impact stakeholders (AERA, APA, & NCME, 2014).

Although test publishers may wish it untrue, it is likely that every test is subject to some unintended, negative consequences, including those occurring as part of test use. This includes not considering contextual factors underlying data from international assessments, which may lead to inaccurate data interpretation thereby limiting the usefulness and meaningfulness of the data from international assessments to guide policy and practice. In recognition of this issue, The Standards for Educational and Psychological Testing (p. 19) concluded that “unintended consequences merit close examination.â€

Few systematic mechanisms have been developed to help identify unintended consequences. Such approaches may require not only understanding how to detect unintended consequences but also how to create mitigation plans to alleviate unintended consequences. One emergent approach is the use of a Theory of Action (Bennett, 2010) to identify and mitigate unintended consequences.

In this workshop, participants will be presented with examples of unintended consequences, the process used to identify them, and possible solutions for addressing the consequences from Theories of Action developed across an array of assessments administered internationally. Participants will begin the development of a Theory of Action for their own assessment program. Finally, participants will be presented other examples of mitigation plans, such as building a validity rationale, to help support the validity of their own assessment program.