Chair: Author 1

Introduction: Assessment for learning (AfL) policy frameworks are implemented by school teachers, so their understanding of what AfL is and how to do it mediates the policy. In contrast to testing where validity and reliability are discussed and expected, classroom assessment appears to have been less concerned with these important measurement issues, placing greater focus on 'improvement'.  However, improvement interpretations and actions lack robustness without confidence in the validity and reliability of classroom assessments. Hence, a better understanding of the beliefs, classroom conditions, and social values which may systematically undermine quality assessment need examination. This symposium draws together four contributors from the Handbook of Social and Human Conditions in Assessment (Routledge, July 2016) who examine the challenges and tensions teachers face. The papers examine factors which may undermine validity without teachers and other stakeholders being aware that this is occurring. These include teacher beliefs, teacher preparation, classroom practice, and contemporary contexts which must be considered during implementation of AfL policies.

Contributions:

Paper 1 addresses the beliefs and attitudes teachers have towards the competing tensions of using assessment formatively for improved outcomes and summatively for evaluation and accountability purposes.

Paper 2 addresses the teaching of preservice teachers to work in an AfL environment. Teacher education has to prepare students for the complexities of environments where both summative and formative practices co-exist.

Paper 3 focuses on classroom assessment realities where teacher practice has to incorporate assessment that contributes to improved student learning and can be used for summative purposes.

Paper 4 focuses on the tensions between beneficial assessment practices and students’ need for and right to privacy.

Conclusions: These papers remind us of the complexity facing teachers in school assessment policies and practices. They also point to research-based pathways that show teachers can implement AfL while meeting accountability requirements.

Teachers’ Perceptions about Assessment: Competing Narratives
Sarah M. Bonner, Hunter College

Perceptions shape and are shaped by beliefs, attitudes and values. They provide the basis for dispositions to act, which are profound predictors of behaviors. In theory, teachers’ beliefs, values, and perceptions about assessment influence their assessment actions. Thus, teachers’ perceptions about assessment matter.

Objectives

This paper synthesizes empirical research demonstrating teachers’ conflicting perceptions about assessment for learning (AfL) and accountability testing. Directions for research and recommendations as to why policy-makers should consider teachers’ perceptions when designing assessment systems are discussed.

Design/Methodology

The paper reviews the last 20 years of empirical literature on teacher concepts, beliefs, attitudes and values about K-12 student assessment. The term “perception†is used as an umbrella because concepts, attitudes, values, and beliefs are all shaped by mental interpretations about perceived information and stimuli.

Results

Teachers endorse general AfL practices such as using assessment to guide instructional responses. Teachers are more variable in their perceptions about peer- and self-assessment and feedback. Teachers’ perceptions about externally-mandated assessments are generally negative. Negative perceptions about externally-mandated tests may be particularly salient in contexts where policies encourage extensive use of standardized test data.

Conclusions

Teachers find it challenging to balance conflicting perceptions about test-based accountability with perceptions about AfL, when AfL and accountability assessment appear to be at odds. Internal conflicts such as these have been associated with negative emotions ranging from resignation to anger. Further research is needed to pinpoint specific perceptual conflicts about assessment, for instance among teachers working at different student developmental levels or within different disciplines. Teacher education programs should include methods to help preservice teachers gain self-efficacy in assessment and adaptively cope with negative perceptions. More broadly, assessment policies that better integrate AfL and accountability will help teachers achieve valued assessment goals while reducing perceived conflicts.

Moving From Student to Teacher: Changing Perspectives about Assessment Through Teacher Education.
Mary F. Hill, University of Auckland; Gayle Eyers, Waikato University

Introduction

Despite the importance of preparing assessment confident and competent beginning teachers, evidence has suggested that teachers are often ill-prepared for their assessment roles. Campbell (2013) argues that teacher preparation has contributed to this situation by being less than adequate and that such preparation may also be overridden by both teaching practice experiences and certain personal characteristics of the student teachers themselves.

Objectives

This paper reviews the empirical evidence regarding effective techniques for preparing teachers for their classroom assessment role and focuses particularly on involving student teachers in the practical use of assessment in classrooms during their preparation influences their learning.

Design/Methodology

This paper reviews the empirical literature regarding teacher preparation in assessment carried out in the compulsory schooling sector internationally since 2000. Following a key word search, 22 relevant studies were found and analysed thematically

Findings

Four major themes regarding teacher preparation for assessment were detected in the literature reviewed.  These related to the influence of student teacher assessment conceptions, the relationships between teaching and student teacher learning, the influence of involving student teachers in the practical use of assessment in classrooms, and a category examining other issues.

Conclusions

Despite variability in assessment education outcomes, and in contrast with Richardson and Placier’s (2001) review of the teacher change literature, many studies in this review demonstrated that with deliberate effort and planning, PSTs can begin teaching with positive attitudes, knowledge and ability to use assessment appropriately in their social context.

Campbell, C. (2013). Research on teacher competency in classroom assessment. In J. H. McMillan (Ed.), Sage handbook of research on classroom assessment (pp. 71-84). Thousand Oaks, CA: Sage Publications.

Richardson, V. & Placier, P. (2001). Teacher change. In V. Richardson (Ed.) Handbook of Research on Teaching (4^th Ed.) (pp. 905-947). Washington, D.C.: American Educational Research Association.

Classroom Processes that Support Effective Assessment.
Christine Harrison, King’s College London

Introduction

Since Black & Wiliam (1998,) much has been done to introduce and strengthen assessment for learning.  Assessment for learning has developed to encompass aspects of student action with increasing emphasis on students taking an active role in their own learning and assessment processes. Hence, assessment activities must address immediate student needs (e.g., certification, feedback to students on their current learning status) while also contributing to their future learning.

Objectives

This paper reports on classroom assessment realities where teacher practice has to incorporate assessment that can generate and use information on student learning to inform teaching, report on learning and assist students to monitor and progress their learning.  This paper looks at such classroom assessment practices and some of the factors that inhibit or support a formative approach.

Design/Methodology

In this paper, we build on a completed review by looking at how classroom assessment impacts self-efficacy for teachers and students and constructs and measures curriculum.  We approach curriculum, pedagogy, and assessment, with an eye to identifying productive and contradictory relationships between them.

Results

Our analysis suggests how classroom assessment affects learners and teachers and the curriculum and highlights the tensiond and dilemmas that can occur in the classroom. We also touch on some of the solutions that teachers use to foster a formative approach.

Conclusions

There is clearly a need to consider classroom assessment in relation to curriculum, pedagogy, and classroom culture in order to enact formative classroom assessment for quality learning.  There are still concerns that the potency of high stakes summative assessment and the uptake of sometimes superficial strategies to introduce assessment for learning that continue to inhibit formative processes.  By unravelling some of the complexities involved in a formative approach to classroom assessment, we outline some of the thinking, professional learning, and support needed in this approach.

 

Privacy in classroom assessment.
Martha Koch, University of Manitoba; Robin Tierney, Research-for-Learning

Introduction

Educational policy, across Canada and internationally, increasingly requires teachers to use assessment for learning (AfL). Digital technologies have the potential to enhance AfL because they facilitate the collection and sharing of information, allow for varied assessment forms, and enable prompt, personal feedback. At the same time, students need privacy as they learn, and maintaining the privacy of their personal information is a legal right. Teachers are faced with a dilemma in that the benefits of digital technologies for AfL practice are accompanied by increased risks of violating students’ privacy.

Objectives

This paper explores this dilemma by asking: How can teachers protect student privacy while using AfL to its full potential? This discussion of privacy is framed with key definitions and a brief historical overview. The paper then presents several ways assessment practice can conflict with privacy needs or rights and consider the implications for policy and practice.

Design/Methodology

Our chapter in the Handbook of Social and Human Conditions in Assessment, discusses privacy from a multi-national perspective. In this paper, we build on our chapter by looking more closely at the Canadian context. In particular, policies and guidelines teachers follow in several provinces around student privacy and assessment practice are analysed.

Results

The analysis describes how privacy in classroom assessment is currently being addressed in Canada in light of the ideas and recommendations from our Handbook chapter.

Conclusions

Concerns about privacy have surged in recent years, especially with the rapid development and adoption of new technologies. In response, governments have enacted legislation and released resources about privacy. However, very little of this focuses on classroom assessment practice and guidance for teachers is limited. Changes in teacher education and assessment policy are needed to encourage both respect for privacy and effective implementation of AfL.

 

Teachers’ Assessment Literacy and Conceptions of Assessment in a Performative Culture
Kim Koh, University of Calgary; Gavin Brown, University of Auckland; Eugene Kowch, University of Calgary; Olive Chapman, University of Calgary; Bryan Szumlas, Calgary Catholic School District

The implementation of assessment for learning practices in teachers’ daily classroom instruction cannot be realized if teachers do not perceive that improved teaching and student learning are the true functions of assessment. Teachers’ conceptions of assessment tend to be ecologically rational; that is, aligned with the social, cultural, and policy priorities of their school context. Therefore, when performativity on high-stakes, standardized testing is a priority of both school and system, assessment is likely to be perceived as mainly serving accountability functions. Assessment for learning practices can also be impeded if teachers have a low level of assessment literacy. The purpose of this study is to examine preservice and inservice teachers’ assessment literacy and conceptions of assessment in a high performative culture. Approximately 436 Canadian teacher candidates and teachers were administered the Canadian Assessment Literacy Inventory (CALI) and the Brown Teachers’ Conceptions of Assessment abridged inventory (TCoA-IIIA). The CALI is adapted by the first author for use in the Canadian context. It consists of 25 multiple-choice items to measure preservice and inservice teachers’ levels of assessment literacy. The TCoA-IIIA consists of 27 6-point Likert items, which measure four major conceptions of assessment: assessment improves teaching and learning, assessment makes students accountable, assessment makes teachers and schools accountable, and assessment is irrelevant. Confirmatory factor analysis is used to validate the factor structures of the two sets of data. In addition, the relationship between teachers’ assessment literacy and conceptions of assessment will be examined using structural equation modeling. The full data analysis will be completed for our presentation at the ITC conference in July 2016. The study’s findings will inform planning and review of assessment curriculum in teacher education and professional development programs in Canada and other high performative countries that share similar reform visions in assessment.

 

Evaluating the Validity of Student Grades based on Instructor-Constructed Multiple-Choice Items
Gavin Brown, University of Auckland; Hasan Abdulnabi, AMA International University

Introduction

Multiple-choice questions (MCQs) are commonly used in higher education assessment tasks for their ease and efficiency in covering instructional content in a short time. In contrast to most testing programs, statistical analysis of items and IRT-score determination is rare in higher education. Studies evaluating MCQs used in higher education assessments have found many flaws in the writing of items relative to conventional best practice, resulting in misleading insights about student performance and contaminating grading decisions.

Objectives

To evaluate instructor-written MCQs used operationally to award grades and determine (a) item quality, (b) impact on test score, and (c) impact on grades.

Methodology

Secondary analysis of 100 instructor-written MCQs used in an undergraduate midterm test (50 items) and final exam (50 items), making up 50% of the course grade was conducted. Data were obtained from 380 students enrolled in one 1st-year undergraduate general education course. Item difficulty, discrimination, and chance properties were determined using four statistical models (i.e., CTT, 1PL, 2PL, & 3PL) and contrasted to the conventional raw score approach. Software used for analyses included: SPSS version 21 for CTT, SPSS v21 with the Rasch model extension for 1PL, and Hansen’s ICL software for 2PL and 3PL.

Results

For each model and test, the effect on individual student assessment (grades) and course grades was evaluated. The mid-term test had significantly more problematic items than the final exam. The 3PL model kept most of the items, produced more reliable test scores, and resulted in more students passing the course than all three alternatives and the raw score approach.

Conclusions

The analyses show that many MCQs were flawed, despite judgement-based quality assurance procedures. Consequently, many students received an indefensible score and course grade. Higher education institutions need to integrate item analysis and standard setting processes before score and grade determination is conducted.