Introduction
International Test Commission Guidelines (2010) provide standards to ensure successful translation and adaptation of tests in a multi-lingual multi-cultural context. This symposium intends to introduce the validation process used in both large scale education assessments as well as international credentialing testing. The objectives are to examine the challenges in implementing ITC guidelines in test translation and adaptation, and to offer best practice insight in the international context.

Contributions
Four papers are intended to illustrate the implementation of the ITC (2010) guidelines on translating and adapting tests: context, test development and adaptation, administration and score interpretation, and documentation. The first paper, "Transability assessment: Minimizing sources of language-driven and cultural-driven bias in multilingual surveys" describes the linguistic equivalence framework adopted by PISA, PIAAC, and TIMMS. The paper examines the method of translatability assessment prior to translating and adapting an international large-scale assessment. The second paper, "Challenges and best practice evidence for a multi-lingual and multi-cultural workplace credentialing test translation and adaptation" introduces the translation and adaptation process of a workplace-skills credentialing test, and findings of errors and corrections through small scale tryout studies. The third paper, "Psychometric evidence of the Career Readiness Certificate Tests equivalence in Spanish and Thai languages" examines the psychometric evidence of English tests translated and adapted to multi-language use. Measurement equivalence is examined from item difficulties, speededness, omit rates, DIF, and factor analysis. The fourth paper investigates the context effect in adopting a workplace skills credential test trans-adapted for use in Thailand. The paper introduces the motivation, rationale for the adoption, the test administration process, the examinees reaction to the trans-adapted tests, and the score use by the employers.

Conclusion
Overall, the symposium aims at sharing insights in challenges and processes in continued effort to improve translation and adaptation of tests in an international context.

******

Paper 1 – Musab Hayatli and Steve Dept, cApStAn Linguistic Quality Control, Belgium

Translatability assessment: Minimising sources of language-driven and culture-driven bias in multilingual surveys

Introduction
In the past two decades, new standards have been defined for the adaptation of assessment instruments into multiple languages. A greater sophistication in translation designs has become the norm in large-scale surveys (PISA, PIAAC, TIMSS, etc.). Analysis of translation verification feedback from these studies suggested that issues in the English master versions are often the main source of problems found in the target versions. This has led to developing a translatability assessment as part of an effort to optimize the master version before translation begins.

Objectives
Using empirical data, a typology of translatability issues was developed to capture issues that translators typically grapple with when adapting assessments; and a process was set up (i) to organize detection of potential language/item interactions; and (ii) to produce better, more translatable master versions.

Design/Methodology
A Translatability Assessment involves submitting items to a pool of experienced linguists covering target languages. They produce advance translations with a view to identifying/reporting translation, adaptation and cultural issues, using a framework of translatability categories. Whenever possible, alternative wording is proposed to circumvent the problem without loss of meaning. In other cases, translation notes are drafted to help translators address survey adaptation issues.

Results
The feedback from translatability assessments was deemed extremely useful by item writers, who are usually not in a position to foresee translation/adaptation difficulties that might arise from materials that function well in English.

Conclusions
In the Translatability Assessment, item developers and linguists succeed in combining psychometrics and linguistics to produce a master version that is fit for adaptation. Ultimately, measurement equivalence can only be demonstrated through analysis of results. However, integrating into the adaptation process as much information as possible about features known to affect psychometric properties of items will contribute to producing better instruments that will be more comparable across languages.

Paper 2 - Donna Matovinovic, ACT, Canada

Challenges and best practice evidence for a multi-lingual and multicultural workplace credentialing test translation and adaption

Introduction
ACT WorkKeys Career Readiness Certificate (CRC) test was published in 2006 initially for U.S. domestic users. Increasingly, Career Readiness Certificate test was adopted in multiple countries by global employers, government organizations, and education institutions.

Objectives
The WorkKeys translation and adaptation framework was developed based on Standards (AERA/NCME/APA, 2014) and International Test Commission Guidelines for Translating and Adapting Tests (2010). In this paper, we describe the principles that guide the CRC trans-adaptation process, and our experience and challenges in ACT WorkKeys Spanish and Thai language test translation-adaptation.

Methods
Based on Standards (2014) and ITC guidelines (2010), we define the WorkKeys translation and adaptation guiding principles to focus on cultural and language sensitivity to ensure validity and fairness. We use judgmental review process implemented by multiple linguistic and construct reviewers in the translation and adaptation process. Empirical data from usability study and small scale field test study were analyzed to understand psychometric evidence of item and test equivalence. Using the data, we evaluate the new language version of the test for quality and test equivalence.

Results
The Spanish and Thai language test items were adapted in part by using local measurement metrics, names, maps in Applied Mathematics, Reading for Information tests, and Locating Information tests. Ercikan and Lyons-Thomas (2013) check list of expert review for linguistic equivalence was introduced in the process. Certain items were flagged for multiple reviews using both judgmental review for errors, and pilot study item statistics.

Conclusion
Cultural relevance, the actual meaning of an item, item format, and omissions/additions that affect meaning, verb tense, vocabulary difficulties, key word translation, lack of appropriate word in translation were important areas for detecting errors and documenting test equivalence. We continue to improve the practice and learn from errors detected from the trans-adaptation and data collection.

Paper 3 – Xiaohong Gao, Haiyan Lin, and Changhua Rich, ACT, USA and China

Psychometric evidence of Trans-adapted Career Readiness Certificate Tests equivalence

Introduction
This paper examines the measurement equivalence from psychometric studies of English, Spanish and Thai language versions of ACT National Career Readiness Certificate tests. Intact forms were translated and adapted into Spanish and Thai language versions of the tests. Data on Spanish and Thai versions of the tests were collected in 2015.

Objectives
The Standards for Educational and Psychological Testing (2014), ITC Guidelines (2010), and U.S. Uniform Guidelines on Employee Selection Procedures (1978) were followed in psychometric analysis of the international version of the tests. We wish to shed light on the challenges and the on-going effort to improve practice in test transadaptation for the employability credentialing assessment.

Design/Methods
The studies of psychometric analysis of measurement equivalency follow common methods in the literature (Hambleton, Sireci, & Robin, 1999). First, descriptive item statistics for each version of the test are analyzed. Second, differential item functioning (DIF) analysis is used to examine the performance differences for a single item conditioned upon the different language groups with equivalent ability levels. Third, dimensionality analysis is conducted to provide information about how the items are related to each other and to the overall construct being measured. Also, reliability and standard error of measurement are estimated for different language versions.

Results
The initial results show differences on item difficulties and item discrimination indices of some items in Thai language tests in comparison to English language version. The speededness on some items was found relatively large. Other data analyses are still underway and results of the analyses will be presented.

Conclusions
Validation is an on-going process. The psychometric analysis of several language versions provided valuable information during initial launch of the transadapted tests. We continue collecting, analyzing, and documenting the translation and adaptation equivalence of the multiple-versions of the CRC tests.

Paper 4 - H.Thithaphol (Ole) Dow Chemical, Thailand, Hannibal Guerrero, Mike Bouldin, Helen Palmer, Changhua Rich, ACT, Thailand, USA, and China

Dow Thailand’s perspective on skills training and competency assessment in the workplace

Introduction
Dow Thailand is the largest Asia Pacific manufacturing base for Dow. Working with Dow U.S., we set up the employee training systems to enable people working with new technology as well as increasing productivity. ACT provided the Career Readiness Certificate credential testing in Thai language for our employees in Thailand.

Objectives
Our goal in this presentation is to introduce Dow Thailand’s experience in adopting ACT WorkKeys tests in the Thai language, and the context and the motivation for adopting WorkKeys. We describe the test administration conditions we used and employees’ feedback on the test. We will also describe how the test results were used.

Methods
ACT Career Readiness Certificate tests in Thai language were administered via WorkKeys online test platform for three subject areas: Reading for Information, Applied Mathematics, and Locating Information. These tests were first transadapted into the Thai language.  The tests were administered to 500 employees during April and August of 2015.

Results
Most of our examinees achieved a level 3, 4, or 5; some employees even scored level 6. To ensure the validity of score interpretation, we worked with internal subject matter experts (Dow senior employees) and ACT experts to review the job profiles of the examinees against the U.S. Occupational Information Network job skills descriptions and required WorkKeys skills levels. We found some differences in the job skills and the test levels used in the U.S. The test results were used for evaluating employees’ essential job skills status.

Conclusion
The test results provided Dow Thailand and its employees a useful tool to measure employee skills in problem solving, reading comprehension, and analytical skills that are important for the performance on the job. We see the benefit and intend to continue using WorkKeys for evaluation of the employees and to assist our decision making.

Discussant – Steve Sireci, University of Massachusetts, USA