Critical examination of the feasibility of comparative studies remains a desideratum in international research in higher education, especially in economics (Troitschanskaia, Shavelson & Kuhn, 2015). In our study, we investigate the comparability of economic knowledge (EK) test data in higher education in Germany, Japan, South Korea, and the United States, where the Test of Understanding College Economics (TUCE; 60 multiple-choice items, Walstad et al., 2007) was used to assess the EK of students nationwide (N_Ger=1,629; N_Jap=1,188; N_Kor=2,163; N_US=11,059).

Tests were adapted and validated according to the TRAPD translation process, including Translation, Review, Adjudication, Pretesting, and Documentation (Harkness, 2008), the Test Adaptation Guidelines (TAG) (Hambleton, 2005; International Test Commission, 2005) on the four categories context, test adaptation, administration, and score interpretation and the Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association & National Council on Measurement in Education 2004; 2014). Test adaptations were examined and compared by means of back translations and interviews with translators to identify deviations in terms of functional equivalence. Furthermore, interviews were conducted with experts and students, and textbooks and module handbooks were analyzed in the respective countries in order to determine the comparability of the content addressed in the adapted tests. Using a scoring guide to ensure data was coded the same way in all the countries, we conducted analyses of measurement invariance and differential item functioning as well as scaling of data at a national level and at an international level. Our findings indicate that these adaptations of the TUCE provide comparable data of EK across these countries.

In our presentation, we will share our qualitative and quantitative findings from the adaptation and validation process and will discuss conceptual challenges concerning score interpretations and measurement methods, which should perhaps be considered in the next generation of TAG.