Open Conference Systems, ITC 2016 Conference

Font Size: 
POSTER: Response-Style Influenced Amplification or Moderation: Revamping Interpretative Guidelines for and Norming of Assessment Scores
Carina Fiedeldey-Van Dijk

Building: Pinnacle
Room: 2F-Harbourside Ballroom
Date: 2016-07-03 03:30 PM – 05:00 PM
Last modified: 2016-05-21

Abstract


Recent advances in dealing with the impact of response style (RS) patterns on behaviour and validity include procedures such as item calibration, score standardization (Van de Vijver, 2014), ipsative/nipsative response formats (Cheung & Chan, 2002), vignette anchoring (King & Wand, 2004), over-claiming (Kyllonen & Bertling, 2014), construct equivalence (Inceoglu et al., 2014), and cultural contextualization (Bartram, 2013; Baena, Van de Vijver, & Garcia, 2014).  Test takers’ RS is treated as distortions in need of reduction/elimination that should be construct-irrelevant across countries; a problem that “just wouldn’t go awayâ€.  This paper argues for continuation of the aforementioned efforts AND that RS patterns can be utilized as critical information in score interpretation in novel and standardized ways.  We will demonstrate how test takers’ patterned styles of responding to assessments, and by extension to functioning in the workplace and elsewhere, is interpretational information in itself.  RS patterns of a large, composite sample of n = 3,810 EQ-i completions were statistically analyzed and modeled onto 11 distinct cohorts, currently in its sixth edition.  (The 133-item EQ-i uses a five-point normative, interval scale of graded agreement grouped into 15 scales.)  Each RS cohort was profiled according to total EQ score, strengths and areas for development/opportunity, scale clusters where functional challenges are evident (e.g., in resilience, or in a particular leadership style), highest response validity indices (inconsistency, positive and negative impressions,) and response time.  The findings suggest that the current practice of using a standardized scale for score interpretation, often accompanied by descriptive guidelines in categories/band demarcated by fixed cut-offs, is inaccurate in the face of RS patterns that deviate from that of the norm population at large.  RS cohort norm options will be beneficial for development applications.  Existing demographic/performance norms can be calibrated by targeting specific scales profiled in RS cohorts, thereby enhancing test validity.


An account with this site is required in order to view papers. Click here to create an account.