This paper focuses on assessing one specific 21^st century skill – collaborative problem solving (CPS) in an online environment with paired students. Using log file analysis methods and an Item Response Theory approach, it combines methods of psychometrics and educational assessment to develop person ability estimates of this 21^st century skill. Recent economic, educational and psychological research has highlighted the shifting workplace requirements and the change that is required in education and training to suitably equip the emerging workforce with the skills for the 21^st century. As a response to these emerging demands frameworks and assessments for 21^st century skills have emerged. As the economy changes towards digital, knowledge based, 21^st century workplaces the importance of new methods of assessment and measurement increases. This paper presents a template for estimating person ability in collaborative problem solving online. A template provides an outline of an assessment allowing a clear definition of the parameters of an assessment for reference before, during and after test construction. The design of this assessment template from conceptualisation to implementation centres on its generalisable application with the initial intention for the template to define a generalisable measure for the assessment of collaborative problem solving. It includes an assessment framework, a task template and scoring procedures. Systematic methods of coding computer log file data that will yield valid and reliable assessments are presented.

Findings from the study present stable psychometric properties of the template across different tasks built from the template. This paper aims to present an approach that can inform new measurement practices in relation to previously unattended latent traits and their processes. The assessment template provides a cost and time efficient approach to development of automated assessments of CPS and potentially other 21^st century skills.