Font Size: 

The Smarter Balanced Assessment Consortium (Smarter) consists of 15 U.S. states and affiliates serving approximately 7 million students representing diverse cultures and languages. Translated tests are an important and common testing resource for English learners (ELs), students who speak English as a second language in addition to developing their first language. Until recently, no analytical tool systematically analyzed the quality of test translations. Few have focused on how test translation review processes can improve the quality of translated test items, particularly at a large scale. Here, we discuss the application of the theory of test translation error (TTTE) for examining translations in an online computer adaptive mathematics assessment.

By examining test translations with a disconfirming lens, expecting translation error rather than translation accuracy, the TTTE postulates that translation error is inevitable but can be minimized. Additionally, it considers the severity of an error, a probabilistic notion that the translation of an item is objectionable to the degree in which many small or very serious errors are identified.

We analyzed the Spanish translation of 216 mathematics items in grades 3, 7, and 11. Using the TTTE, we identified the number and severity of errors in each item. Very few severe errors were identified indicating high quality of the translations. This analysis provides insight into applying the TTTE in a new context: computer administered tests. We discuss adaptations of the existing TTTE framework in this new context, including the addition of a new dimension and expanding existing dimensions to capture error at the item and system levels. Results indicate that the TTTE is a useful tool for reviewing translations and identifying patterns in the types of errors observed.

Knowledge gained here informs application of the TTTE to ensure accuracy in translations of large-scale assessments serving diverse student populations.