NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1019623
Record Type: Journal
Publication Date: 2013-Dec
Pages: 35
Abstractor: As Provided
ISSN: ISSN-1076-9986
Measuring Test Measurement Error: A General Approach
Boyd, Donald; Lankford, Hamilton; Loeb, Susanna; Wyckoff, James
Journal of Educational and Behavioral Statistics, v38 n6 p629-663 Dec 2013
Test-based accountability as well as value-added asessments and much experimental and quasi-experimental research in education rely on achievement tests to measure student skills and knowledge. Yet, we know little regarding fundamental properties of these tests, an important example being the extent of measurement error and its implications for educational policy and practice. While test vendors provide estimates of split-test reliability, these measures do not account for potentially important day-to-day differences in student performance. In this article, we demonstrate a credible, low-cost approach for estimating the overall extent of measurement error that can be applied when students take three or more tests in the subject of interest (e.g., state assessments in consecutive grades). Our method generalizes the test-retest framework by allowing for (a) growth or decay in knowledge and skills between tests, (b) tests being neither parallel nor vertically scaled, and (c) the degree of measurement error varying across tests. The approach maintains relatively unrestrictive, testable assumptions regarding the structure of student achievement growth. Estimation only requires descriptive statistics (e.g., test-score correlations). With student-level data, the extent and pattern of measurement-error heteroscedasticity also can be estimated. In turn, one can compute Bayesian posterior means of achievement and achievement gains given observed scores--estimators having statistical properties superior to those for the observed score (score gain). We employ math and English language arts test-score data from New York City to demonstrate these methods and estimate the overall extent of test measurement error is at least twice as large as that reported by the test vendor.
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Grade 3; Grade 4; Grade 5; Grade 6; Grade 7; Grade 8
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: New York
Grant or Contract Numbers: N/A