ERIC Number: EJ1070135
Record Type: Journal
Publication Date: 2015-Sep
Abstractor: As Provided
Equating a Large-Scale Writing Assessment Using Pairwise Comparisons of Performances
Humphry, Stephen M.; McGrane, Joshua A.
Australian Educational Researcher, v42 n4 p443-460 Sep 2015
This paper presents a method for equating writing assessments using pairwise comparisons which does not depend upon conventional common-person or common-item equating designs. Pairwise comparisons have been successfully applied in the assessment of open-ended tasks in English and other areas such as visual art and philosophy. In this paper, pairwise comparisons were used to equate writing tests for two consecutive calendar years of Australia's large-scale assessment program. The viability of the method was demonstrated by the very high internal reliability of the pairwise scale, acceptable fit between data and the models applied, and a very high correlation between the pairwise scale and the rubric scores for both calendar years. An equating constant was shown to be not statistically significant, consistent with stable population means for the writing assessment across calendar years. The method also provided external validation of the rubric assessments. The limitations and advantages of the method are described and broader implications of the findings of the results of the study are discussed.
Descriptors: Writing Evaluation, Evaluation Methods, Comparative Analysis, Writing Tests, Foreign Countries, Reliability, Goodness of Fit, Correlation, Scoring Rubrics, Validity, Equated Scores
Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: firstname.lastname@example.org; Web site: http://www.springerlink.com
Publication Type: Journal Articles; Reports - Descriptive
Education Level: N/A
Authoring Institution: N/A
Identifiers - Location: Australia
Grant or Contract Numbers: N/A