Font Size: 

Sequence of presentation matters. This has been shown in various decision contexts ranging from effects related to the presentation sequence of political candidates on ballots to the position of food options on menus (e.g. Koppell & Steen, 2014). Usually, extreme positions were found to be more attractive and thus more likely to be chosen. Consequently, such effects can also be expected in multiple-choice (MC) items, especially when the test taker is not entirely sure about the correct answer to the question or engagement is low due to a lack of result-related consequences (e.g. in low-stakes assessment contexts). Previous research on this topic has been inconclusive whether there is an effect of the correct answer position and mostly focused on adult students. Studies showing an effect, were inconclusive whether extreme or middle positions of the correct answer were easier (e.g. Tellinghuisen & Sulikowski, 2008).

The aim of the present paper is to provide the most comprehensive analysis of this question till date and to draw on data of the educational large-scale assessment context that due to economic reasons mostly relies on MC. We analyzed the effect of correct answer position in four different data sets (N > 13000) from Austria, Germany, and Luxembourg that varied concerning students’ age (between 10 and 19 year-olds) and tested subjects (reading comprehension, mathematics, vocabulary, and science). To overcome methodological limitations of previous studies, we used Fischer’s LLTM (1973) that is based within the IRT framework.

Results suggest that effects of correct answer position can be neglected, especially when students are older than 10 years and if stakes are high. However, in younger students such effects are present but not consistent. Given an increase of large scale assessments also in younger cohorts, it is discussed what actions could be taken to tackle this position effect.