Introduction

An issue surrounding the use of psychological tests is whether these tests perform equally across varying cultural and linguistic backgrounds. Moreover, our program of research has found that certain characteristics of items on a test interact with linguistic background and have an effect on responding.

Objectives

The present study examined whether cultural background had an effect over and above English fluency on responses to a modified version of the College Self-Efficacy Inventory. Additionally, we probed the effect of linguistic features of the items and whether these effects differed across levels of English fluency and cultural background.

Methodology

A sample of 690 participants was drawn from a diverse student population consisting of domestic, international, immigrant, and first generation Canadian students. Analyses included English Fluency, cultural background, and linguistic features such as concreteness, referring to others, and presence of possessives or plurals. Cultural background was assessed using the Asian Orientation Scale and Western Orientation Scale, on which individuals rated how often they participate in Western and Asian cultural activities.

Results

Using linear mixed models, results showed that cultural orientation had a statistically significant effect on responses over and above English fluency. In addition, we found statistically significant effects for linguistic features. Certain linguistic features interacted with cultural orientation as well as English Fluency. Finally, there was a statistically significant interaction effect between Asian and Western Orientations.

Conclusions

Our results emphasize that the linguistic features of test items can impact responding. Additionally, both the linguistic background and the cultural orientation of individuals impacted their responses. While language and culture often go together, researchers should not assume that a particular linguistic background serves as a proxy for cultural alignment. Therefore, in multicultural testing settings, results must be interpreted with respect to the interplay between individual characteristics and linguistic features of the test items.