ERIC Number: EJ1154674
Record Type: Journal
Publication Date: 2017
Pages: 11
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0731-1745
EISSN: N/A
Comparability in Balanced Assessment Systems for State Accountability
Evans, Carla M.; Lyons, Susan
Educational Measurement: Issues and Practice, v36 n3 p24-34 Fall 2017
The purpose of this study was to test methods that strengthen the comparability claims about annual determinations of student proficiency in English language arts, math, and science (Grades 3-12) in the New Hampshire Performance Assessment of Competency Education (NH PACE) pilot project. First, we examined the literature in order to define comparability outside the bounds of strict score interchangeability and explored methods for estimating comparability that support a balanced assessment system for state accountability such as the NH PACE pilot. Second, we applied two strategies--consensus scoring and a rank-ordering method--to estimate comparability in Year 1 of the NH PACE pilot based upon the expert judgment of 85 teachers using 396 student work samples. We found the methods were effective for providing evidence of comparability and also detecting when threats to comparability were present. The evidence did not indicate meaningful differences in district average scoring and therefore did not support adjustments to district-level cut scores used to create annual determinations. The article concludes with a discussion of the technical challenges and opportunities associated with innovative, balanced assessment systems in an accountability context.
Descriptors: Academic Achievement, Language Arts, Mathematics Achievement, Science Achievement, Elementary Secondary Education, Accountability, School Districts, Scoring, Cutting Scores, Evaluation Methods, State Standards, Performance Based Assessment, Competence
Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: New Hampshire
Grant or Contract Numbers: N/A