ERIC Number: ED449174
Record Type: Non-Journal
Publication Date: 2000-Aug
Reference Count: N/A
How Out-of-Level Testing Affects the Psychometric Quality of Test Scores. Out-of-Level Testing Report 2.
Bielinski, John; Thurlow, Martha; Minnema, Jane; Scott, Jim
This report is a review and analysis of the psychometric literature on the topic of out-of-level testing. Out-of-level testing refers to the practice of using a level of the test other than the test taken by most of the students in a student's current grade level. Much of the research on out-of-level testing was conducted in the 1970s and 1980s, and the emphasis on chance-level scoring of these studies has detracted from the understanding of how out-of-level testing actually works to affect score reliability. The studies said little about the important concept of measurement error, and there was little discussion about the detrimental effects of translating scores from an out-of-level test back into the scale of the in-level test. The literature indicates that the benefit of out-of-level testing is that it is a cost-effective method for increasing the precision with which low performing students' ability is measured. However, an unreported downside is that the process by which scores on out-of-level testing are converted back into the scale of the in-level test reduces measurement precision. This report focuses on how out-of-level testing affects test score quality. It emphasizes the links among test difficulty, individual ability, and measurement error and discusses the neglected topic of measurement error that is introduced by the equating process used to translate the out-of-level test scores onto the in-level scale. The dramatic increase in the use of out-of-level testing that appears to be taking place across the United States is certainly not justified by the findings of this analysis, and the application of an approach developed in a norm-referenced context seems inappropriate in the current context of standards-based assessments. (Contains 4 figures and 42 references.) (SLD)
Descriptors: Achievement Tests, Elementary Secondary Education, Equated Scores, Error of Measurement, Literature Reviews, Psychometrics, Scores, Standardized Tests, Student Evaluation, Test Format, Test Use
National Center on Educational Outcomes, University of Minnesota, 350 Elliott Hall, 75 East River Road, Minneapolis, MN 55455 ($10). Tel: 612-624-8561; Fax: 612-624-0879; Web site: http://www.coled.umn.edu/NCEO.
Publication Type: Information Analyses
Education Level: N/A
Sponsor: Special Education Programs (ED/OSERS), Washington, DC.
Authoring Institution: National Center on Educational Outcomes, Minneapolis, MN.