Building: Pinnacle
Room: 3F-Port of Hong Kong
Date: 2016-07-02 03:30 PM – 05:00 PM
Last modified: 2016-05-22
Abstract
Introduction
The success with chance is one of the major factors causing the error in measurement results. Therefore, it is already one of the most research topic on the multiple choice test. The effects of guessing on differential item functioning, equating, internal consistency and criterion validity is among the most studied topics. There are few studies in the literature regarding the effect on test dimensionality. Since guessing effects the variance of test scores, items variance, covariance and correlation between items, test' reliability and validity, effects of guessing on achievement tests’ dimensionality and size of the explained variance by each dimension can be expected.
Objectives
This study aims to determine to effects of guessing on number of tests dimension, the variance explained by the dimension, the root mean square error of approximation (RMSEA), goodness of fit value and McDonald's omega reliability.
Method
This study was conducted on simulation data. Research has been studied in one-dimensional and multi-dimensional simple structures. Test length, guessing level, the correlation between the dimensions, number of items in the dimensions, distribution of the student ability and sample size are taken into account. Analysis of test dimensionality was done by Factor 10.3 software. In determine of factor number, Velicer’s MAP test, Optimal Parallel Analysis and Hull methods were used. In factor analysis, tetrachoric correlations between dimensions are taken into account.
Results
According to the results, guessing increases number of factors and reduces the explained variance of the dominant factors. Interaction of guessing with distribution of students’ ability and number of items in dimensions were found high. Also when chance of success increases the goodness of fit and reliability reducing and the root mean square error of approximation is increasing.
Conclusions
When using multiple-choice tests, guessing should be taken into consideration and statistical techniques should be utilized to fix it.