A large number of psychological and educational tests are translated and adapted across languages and cultures. Unfortunately, the quality of these adaptations can be rather poor, with harmful consequences for individuals evaluated these tests. In order to support test adaptation quality, the International Test Commission developed Guidelines for Test Adaptation. The first edition was published by van de Vijver and Hambleton (1996), and by Hambleton (2002), and Hambleton, Merenda and Spielberger (2005).  But only minor editorial changes were seen in the publication of the guidelines between 1996 and 2005.  In the meantime, many advances have taken place since 1996.

Therefore, the ITC Council formed a six-person committee and assigned them the task of updating the ITC Guidelines for Test Adaptation to emphasize the new knowledge that was being advanced and the many experiences that were being gained by researchers in the field. The Guidelines for Test Adaptation, version 2.0 was released in July 2015.

The new guidelines fall into four main categories: Those concerned with the cultural context, those concerned with the technicalities of instrument development and adaptation, those concerned with test administration, and those concerned with documentation and interpretation. This symposium focuses on several innovative guidelines in the version 2.0. The rational underlying the guidelines is explained, and their use is illustrated.

*****

The Second Edition of the International Test Commission Guidelines for Test Adaptation
Ron Hambleton

The field of test translation and adaptation methodology has advanced rapidly in the last 25 years or so with the publication of many important books and technical papers.  The advances have facilitated the growth of cross-cultural psychology, large scale international assessments such as TIMSS and PISA, expansion of credentialing exams, and improvements in test fairness and validity.  The first edition of the ITC Guidelines for Test Adaptation has been cited nearly 1000 times in the psychological and educational literature, and so clearly, they have had a role in the rapid development of relevant methodology and the improvement of practices.  But the first edition of the ITC Guidelines appeared in the literature as early as 1994 and by 2005 they had become dated because of the many advances and needed to be updated.The ITC formed a committee to put forward a second edition of the Guidelines and they have been formally placed now on the ITC web-site for final comments before their release at the 2016 ITC Conference in Vancouver.  In this brief presentation, the 18 guidelines  around six broad topics will be discussed:  (1) Pre-condition, (2) Test Development, (3) Confirmation, (4) A, (5) Score Scales and Interpretation, and (6) Documentation.  This short presentation will set the framework for the remaining presentations in the symposium.

 

Operationalizing the ITC Guidelines for Test Adaptation
Ana Maria Hernandez, Maria Dolores Hidalgo, Juana Gomez-Benito & Ron Hambleton

The ITC Guidelines for Test Adaptation were proposed to improve the quality of adaptation and the comparability of scores across cultures. Rios and Sireci (2014) reviewed a number of papers to check whether the methodologies related to test adaptation had improved or not with the publication of the ITC Guidelines.  Although they only focused on papers where test adaptation and cross cultural comparisons were carried out in the same study, the majority of studies reviewed did not follow the recommendations put forth by the ITC. They concluded that better dissemination mechanisms of the guidelines are needed to improve their implementation. We agree dissemination is crucial but, in this study, we addressed a different point.  We posited that more implementation guidance was needed to reduce ambiguity and challenges in applying the guidelines.  The ITC had always hoped that researchers in the field would pick up this challenge. The purpose of this study was to offer a checklist to help test adapters implement the specific recommendations that can be derived from the ITC guidelines.

Based on the recommendations made in the ITC Guidelines on Test Adaptation (second edition), as well as the literature review on test adaptation, and test evaluation models and checklists (such as EFPA and COSMIN), each of the new (18) ITC guidelines was operationalized through a number of principles. For each principle, acceptability and excellence criterion were proposed. The original proposal was reviewed by a panel of experts by means of the Delphi method. A final checklist derived from the Delphi method that operationalized the guidelines for test adaptation in all the phases (planning, development, confirmation, administration, score interpretation, and documentation) was obtained. The proposed checklist is intended to help researchers from multidisciplinary fields to implement the ITC guidelines when adapting tests and evaluating the quality of test adaptation.

 

Test adaptation and the intellectual property rights
Jacques Gregoire

The ITC Guidelines for Test Adaptation Version 2 includes three new precondition guidelines that should be fulfilled before starting a test adaptation. One of them mentions the need to “obtain the necessary permission from the holder of the intellectual property rights relating to the test before carrying out any adaptationâ€. Intellectual property rights refer to a set of rights over creations of the human mind. They protect the interest of creators by giving them moral and economic rights over their creations. As tests are clearly creations of the human mind, they are covered by intellectual property rights. Most of the time the copyright does not refer to specific item contents, but to the original organization of the test (structure of the scales, scoring system, organization of the material…). Consequently, mimicking an existing test (i.e. keeping the structure of the original test and its scoring system, but creating new items), is a breach of the original  intellectual property rights. When authorized to carry out an adaptation, the test developer should respect the original characteristics of the test (structure, material, format, scoring…), unless an agreement from the holder of the intellectual property allows modifications of these characteristics. In this presentation, we discuss the issue of conducting a test adaptation (i.e. developing a version of the original test taking into account new linguistic and cultural constraints) respecting intellectual property rights.

 

Personality Constructs and Test Adaptation
Fanny Cheung

The ITC Guidelines for Test Adaptation Version 2 provides a detailed set of guidelines for adapting tests for use in various different linguistic and cultural contexts.  Test adaptation is required when an existing instrument is used in another cultural group.  These 18 ITC guidelines are organized into six categories to facilitate their use:  pre-condition, test development, confirmation, administration, score scales and interpretation, and documentation.  Personality assessment is particularly grounded in cultural context. Direct translation may not be able to capture the meaning and functions of the test items which are meant to represent behavioural manifestation of underlying personality constructs. There may be cultural variations in the behavioural manifestation as well as in the personality constructs themselves. I will refer to the Chinese translation of the MMPI-2 to show some of the challenges and related guidelines involved in translating an American-based personality test for use in a Chinese context. Examples in the process of the adaptation and standardization of the Chinese MMPI-2 will be used to illustrate considerations in the translation and adaptation of the test items, testing the equivalence of the items and scales in the Chinese language and culture, and validating the Chinese MMPI-2 for clinical use with Chinese patients.

 

The SEM Approach to Testing for Assessment Scale Equivalence across Culture
Barbara Byrne

Over at least the past two decades, there has been a rapidly growing interest in cross-national comparisons. Although such comparisons traditionally have been the province of cross-cultural psychologists, a review of the extant literature reveals this investigative work now to be of substantial interest within mainstream psychology with researchers, whose work has previously been conducted within a single culture, now extending across national borders. One pervasive weakness in multigroup comparison research in general, and cross-cultural research in particular, is the fallacious assumption that both the assessment scale and the construct it is designed to measure are operating equivalently (i.e., they are invariant) across the groups of interest. Unquestionably, unless these assumptions are tested statistically and found to hold, all findings remain suspect. Structural equation modeling (SEM), within the framework of a hierarchical set of confirmatory factor analytic (CFA) models provides a systematic approach to testing for both measurement and structural equivalence (i.e., that perception of item content and dimensionality of the associated construct are group-equivalent). When tests for assessment scale equivalence involve less than four cross-cultural groups, procedures can be relatively straightforward. However, the process becomes increasingly more complex and tedious as the number of different cultural groups expands and their socio-geographical roots become more widely divergent. In this presentation, I review: (a) the critical importance of equivalence as it relates to both the assessment scale and the construct it is designed to measure; (b) the major threats to such equivalence and exemplify several complexities that can impact the validity of cross-cultural comparisons; (c) the SEM hierarchical set of steps necessary in testing for equivalence of an assessment scale across groups; and (d) difficulties encountered in testing for equivalence across a large number of cultural groups, and elucidate a new statistical approach capable of addressing these complications.