The availability of proper norms is essential to sensibly interpret test scores. For psychological tests scores that change with age, the state-of-the-art methodology for estimating norms is the so-called continuous norming, rather than norming per subgroup. The core idea is to build a statistical model that relates age to the distribution of test scores. This distribution is then subsequently used to derive the norms. In this way, the available information from the whole norm group is used in estimating the norms. A promising continuous norming approach is the use of the generalized additive models for location, scale, and shape (GAMLSS, originating from Rigby & Stasinopoulos, 2005). GAMLSS allows for modelling differences in center, spread, skewness, and kurtosis as a function of age. This can be implemented as a kind of polynomial regression. However, applying GAMLSS for norming to empirical norming data involves the selection of a specific model. For instance, one needs to select the degree of the polynomial. In our study, we will compare different stepwise model selection procedures and evaluate their precision in estimating norms using a simulation study. Specifically, we compare two stepwise model selection procedures (the default in the GAMLSS R-package vs. a newly developed, more flexible procedure) using four model selection criteria (AIC, BIC, GAIC(3) and cross-validation). The data complexity is varied by manipulating the complexity of the relationship between age and the different model parameters. The performance is evaluated in terms of the difference between the true and estimated distributions of test scores against age. We provide an overview of the comparative performance of the procedures and selection criteria in the various conditions of data complexity. We will discuss implications for model selection in continuous norming with the GAMLSS approach.

Reference

Rigby, R. A., & Stasinopoulos, D. M. (2005). Generalized additive models for location, scale and shape. Applied Statistics, 54(3), 507-554.