ERIC Number: ED358113
Record Type: Non-Journal
Publication Date: 1993-Apr
Reference Count: N/A
Test Form Accuracy.
As high-stakes use of tests increases, it becomes vital that test developers and test users communicate clearly about the accuracy and limitations of the scores generated by a test after it is assembled and used. A procedure is described for portraying the accuracy of test scores. It can be used in setting accuracy targets during form construction and in communicating information to test users. The procedure is discussed in the context of the Armed Services Vocational Aptitude Battery. The general procedure for reviewing current test accuracy profiles and for setting new targets includes: (1) developing accuracy profiles for current forms; (2) using expert judgment to review and revise accuracy goals for new forms at key points on the target scale; (3) using item response theory analyses to calibrate new items, adjust for sample differences, and estimate classification error rates for trial forms; (4) developing preliminary tolerances for compliance with accuracy targets; and (5) checking initial form accuracy profiles. The procedures for producing accuracy profiles are illustrated with 4,000 cases from the Profile of American Youth Study. Advantages and limitations of the procedure are discussed. Twelve two-part graphs illustrate the analyses. (SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: Defense Manpower Data Center, Monterey, CA.
Identifiers - Assessments and Surveys: Armed Services Vocational Aptitude Battery