Designing and executing the test administration process is critical to any successful program. The administration should be executed in a way to maximize the validity of the interpretation of scores, ensure fairness for all examinees, maintain the security of the test content, and be efficient from a resource perspective (AERA, APA, & NCME, 2014, McCallin, 2016). The complexities of test administration increase when a program is administered in a large-scale international manner. The CFA Institute (CFA) administers over 100,000 exams annually in over 90 countries. The program uses a paper-based administration that is carried out simultaneously around the world. Although most professional credentialing programs have migrated to computer-based testing, the current model affords the CFA the opportunity to employ numerous security strategies to maintain the integrity of the testing program and resulting scores.

Looking towards the future, the CFA is considering alternative administration options to take advantage of the benefits of computer-based administration platforms (Luecht, 2016). However, with these options come different challenges and potential threats to validity and security. To make such a decision, the CFA is engaging in an evaluation of different test administration options. The purpose of this presentation is to review this evaluation strategy, the current benefits and challenges associated with each option (including the current approach), and how well each would help the CFA achieve their programmatic goals. The presenters in this session will discuss each option from a validity, fairness, security, logistical, and resource perspective. Session attendees will understand the numerous considerations that go into designing an administration strategy, how to evaluate such options in light of programmatic needs, and how to make informed decisions supported by adequate validity evidence.

References:

American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (2014). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association.

Luecht, R. (2016). Computer-based test delivery models, data, and operational implementation issues. In F. Drasgow (Ed.). Technology and Testing (pp. 179-205). New York: Routledge.

McCallin, R. (2016). Test administration. In S. Lane, M. Raymond, & T. Haladyna (Eds.). Handbook of Test Development, 2nd Edition. (pp. 567-584). New York: Routledge