ERIC Number: EJ907914
Record Type: Journal
Publication Date: 2010-Aug
Pages: 19
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0311-6999
EISSN: N/A
Using the Method of Pairwise Comparison to Obtain Reliable Teacher Assessments
Heldsinger, Sandra; Humphry, Stephen
Australian Educational Researcher, v37 n2 p1-19 Aug 2010
Demands for accountability have seen the implementation of large scale testing programs in Australia and internationally. There is, however, a growing body of evidence to show that externally imposed testing programs do not have a sustained impact on student achievement. It has been argued that teacher assessment is more effective in raising student achievement levels. However, it is also often argued that teacher assessments are less reliable than the results of testing programs. This paper presents a study in which teachers judged writing scripts using the process of pairwise comparison to generate a scale. The analysis showed high internal consistency of the teacher judgements. The scale locations from pairwise comparisons were highly correlated with scale estimates for the same students from a large-scale testing program. The results demonstrate it is possible to efficiently obtain highly reliable and valid teacher judgements using the process of pairwise comparison. Reliability indices are also provided for a series of small-scale assessments that used the same methodology in a range of other domains. The results support the findings of the main study. The article discusses the benefits of using the method to supplement and validate results from large-scale testing programs. (Contains 4 tables and 1 figure.)
Descriptors: Testing Programs, Testing, Academic Achievement, Measures (Individuals), Foreign Countries, Accountability, Comparative Analysis, Correlation, Test Reliability, Test Validity, Student Evaluation, Writing Tests, Statistical Analysis, Elementary Education, Measurement
Australian Association for Research in Education (AARE). P.O. Box 71, Coldstream, Victoria 3770, Australia. Tel: +61-0359-649-031; Fax: +61-0359-649-586; e-mail: aare@aare.edu.au; Web site: http://www.aare.edu.au
Publication Type: Journal Articles; Reports - Evaluative
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Australia
Grant or Contract Numbers: N/A