NotesFAQContact Us
Search Tips
ERIC Number: ED400314
Record Type: Non-Journal
Publication Date: 1996-Apr
Pages: 29
Abstractor: N/A
Reference Count: N/A
Some New Methods for Mapping Ratings to the NAEP Theta Scale To Support Estimation of NAEP Achievement Level Boundaries.
Davey, Tim; And Others
Some standard-setting methods require judges to estimate the probability that an examinee who just meets an achievement standard will answer each of a set of items correctly. These probability estimates are then used to infer the values on some latent scale that, in theory, determines an examinee's responses. The paper focuses on the procedures used to convert the probability estimates into performance standards. A number of procedures are described that have been traditionally used, including simple summation, simple ability averaging, weighted ability averaging, and least squares statistics. Some new procedures for estimating achievement standards from subjective probability estimates include an unweighted least squares approach, maximum likelihood (MLE), and modifications to MLE that consider posterior distributions. MLE, unweighted least squares, and simple ability averaging techniques were applied to data from the 1992 National Assessment of Educational Progress reading and mathematics tests for grades 4 and 12. These three approaches were evaluated with a jackknife design. Results for the different procedures were different, with lower achievement results and smaller standard errors from the MLE and least squares procedures than from the traditional simple ability averaging. The most desirable approach, however, may still depend on other than statistical criteria. (Contains one figure, six references, and five tables.) (SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: National Assessment of Educational Progress