Introduction/Objectives

As recent research has demonstrated that distinct and reliable subscores can be masked for a subgroup of examinees when only evaluating the total sample (Rios, 2015), there has been a call to assess individual-level subscore added value (Raymond & Feinberg, 2015). However, as there has been insufficient methodological research on this topic, the objective of this study was to compare the Mahalanobis distance (MD) and H^T indices in identifying whether an individual’s data significantly departs from unidimensionality, which is a prerequisite for possessing subscores with added value (Sinharay, 2010).

Design/Methodology

Data were generated separately for two groups that were administered an n multiple-choice item test comprised of four subdomains using a correlated-traits three-parameter logistic model. Specifically, data for Group 1 were generated to possess an essentially unidimensional structure across subdomains (i.e., inter-subdomain correlations were constrained equal to .95), while the inter-subdomain correlations were generated to be multidimensional for Group 2. Three independent variables were manipulated: a) subdomain test length (10, 25, 50 items), b) proportion of Group 2 simulees in the total sample (10%, 20%, 30%), and c) Group 2 inter-subdomain correlations (0.3, 0.5, 0.7). Data from both groups were then combined for power and Type I error analyses.

Results/Conclusions

Across subdomain test lengths, Type I error rates were maintained at or below 5% for both indices. In terms of power, the H^T index was found to possess rates near 100% across all conditions; whereas, power rates for the MD index (ranged from 13% to 61%) were found to increase when: a) subdomain test length increased, b) proportion of Group 2 simulees decreased, and c) Group 2 inter-subdomain correlations decreased. These results suggest that the H^T index shows some promise in identifying examinees with distinct subscores, which may be the first step to providing valid individual-level subscores.