ERIC Number: ED479797
Record Type: Non-Journal
Publication Date: 2003-Mar
Reference Count: N/A
Errors in Standardized Tests: A Systemic Problem.
Rhoades, Kathleen; Madaus, George
The nature and extent of human error in educational testing over the past 25 years were studied. In contrast to the random measurement error expected in all tests, the presence of human error is unexpected and brings unknown, often harmful, consequences for students and their schools. Using data from a variety of sources, researchers found 103 errors that occurred in the past 25 years. The incidence of such errors has risen dramatically since 1998. Errors were categorized as "active" (those arising from individual mistakes) and "latent" (those arising form poor managerial decisions). Active causes are attributed to most errors, but latent causes were found for 24 of the errors, with 2 cases, both in the United Kingdom, confirmed. Latent error has been characterized as more problematic, since its existence is then connected to the production of active error. The existence of latent error has only been confirmed in the United Kingdom, but confirmation requires a systemic review of the sort that is not often done. Human error has been most likely to affect public school students in grades 4 through 12, and it is estimated that well over 1.5 million students and 4,000 schools have been affected. Most errors are not detected by testing contractors and personnel; they are generally found at local school districts, state departments of education, and those intimately involved with the testing. To reduce the incidence and impact of errors, the use of multiple measures is recommended, and the use of monitoring procedures that minimize error is also suggested. Four appendixes list errors not detected by contractors, errors detected by contractors, errors in school rankings, and gray areas. (Contains 31 endnotes, and 314 references.) (SLD)
Publication Type: Reports - Evaluative
Education Level: N/A
Sponsor: Ford Foundation, New York, NY.
Authoring Institution: National Board on Educational Testing and Public Policy, Chestnut Hill, MA.