In this workshop I will introduce the basic concepts of computational psychometrics (CP; von Davier, 2015), focusing on data mining, machine learning, and data visualization with applications in assessment. CP merges the data driven approaches with the theoretical (cognitive) models to provide a rigorous framework for the measurement of skills in the presence of Big Data. I will discuss five types of big data in educational assessment: a) ancillary information about the test takers; b) process data from simulations and games; c) data from collaborative interactions; d) data from multimodal sensors; and e) large data sets from tests with continuous administrations over time.

Learning technology offers rich data for developing novel educational assessments. The data obtained from learning and assessment technologies are typically “complex,†in the sense that they involve sources of statistical dependence that violate the assumptions of conventional psychometric models. These data are also referred to as process data. Complex assessment data have also been hypothesized to involve a variety of “non-cognitive†factors such as motivation and social skills, thereby increasing the dimensionality of the measurement problem and the potential for measurement bias. The question is how can one measure, predict, and classify test takers’ skills in a simulation or game-based assessment? DM and ML tools merged with the theoretical (cognitive) models may help answer this question.

As mentioned before, there are other types of assessment data that may benefit from the DM techniques. For example, data that consist of test scores and background questionnaire data over many administrations from a test with an almost continuous administration mode. In this situation, one research question may be about patterns in the data, another may be whether test scores for specific subpopuations can be predicted as part of the quality control efforts [4].

I will discuss similarities and differences between machine learning and statistical inference [5] and will present several free software available for machine learning and visualization [6], such as WEKA, RapidMiner, MIRAGE, and routines in R. I will also present examples of research projects and applications in assessment.