NotesFAQContact Us
Search Tips
ERIC Number: EJ1162840
Record Type: Journal
Publication Date: 2017
Pages: 4
Abstractor: ERIC
ISSN: ISSN-1553-541X
Evaluating Comparability in the Scoring of Performance Assessments for Accountability Purposes
Lyons, Susan; Evans, Carla
Voices in Urban Education, n47 2017
This brief summarizes "Comparability in Balanced Assessment Systems for State Accountability," published in "Educational Measurement: Issues and Practice" (Evans & Lyons 2017). The study evaluated comparability claims in local scoring of performance assessments across districts participating in New Hampshire's Performance Assessment of Competency Education (PACE) pilot project. The study applied two judgmental methods for estimating comparability that are used in international contexts: consensus scoring and rank ordering. Across all districts, the consensus scoring yielded scores that were positive, meaning they were a bit lower than the teacher-given scores. Analysis revealed uneven scoring across districts, suggesting that there remains a need for additional training on scoring and within-district calibration, as well as for increased cross-district calibration. The rank-order method results revealed scoring differences across districts, most notably in one district in which teachers were scoring their student work a full standard deviation below (more rigorous) where the judges placed the same student work within the sample. Researchers found that applying the two methods in the context of the PACE system highlighted some strengths and limitations of each method. [For "Comparability in Balanced Assessment Systems for State Accountability," see EJ1154674.]
Annenberg Institute for School Reform at Brown University. Brown University, Box 1985, Providence, RI, 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: New Hampshire
Grant or Contract Numbers: N/A