Importance of the Workshop Topic: Though machine scoring of essays and short-form constructed responses has taken on an increasing role in both summative and formative assessment, getting access to the scoring algorithms is a challenge at best. Unlike CAT or other assessment technologies, there are no readily available commercial resources that one can purchase and only one program that is available the public domain. This makes it difficult to both research the topic and to develop tools that might be employed in instructional settings. The session is geared towards those with assessment or research responsibilities who might be interested in employing automated essay scoring for either assessment or instruction (or both).

Relevance and usefulness to the conference participations: The first portion of this six-hour workshop will focus on the capabilities and research involving the six main commercial automated essay scoring engines, e-rater, Intelligent Essay Assessor, CRASE, Bookette, Project Essay Grade, Intellimetric). The second portion of the workshop will be devoted to work with LightSide, a public-domain automated essay scoring engine, to analyze a set of essays that can be used for either formative or summative assessment. Those who attend this session are encouraged to download and install the LightSide software which is available at: http://lightsidelabs.com/services/research/. Essay data will be provided for those using the software during the session. Workshop participants are expected to have their own laptops with either a relatively recent version of Windows or MacOs.

Background of the facilitator(s): Mark D. Shermis has worked in the area of machine scoring for over 15 years. He and co-editor Jill Burstein have published two books on the topic Automated Essay Scoring: A Cross-Disciplinary Approach (2003) and Automated Essay Evaluation: Current Applications and New Directions (2013) which have served as primary reference material for researchers and developers in the discipline.

Logistics and/or Equipment: Workshop participant are expected to bring their own laptop computers (Windows or MacOs).  They can either download the LightSide software prior to the workshop or obtain it onsite.