NotesFAQContact Us
Search Tips
ERIC Number: ED451258
Record Type: Non-Journal
Publication Date: 2001-Apr-11
Pages: 31
Abstractor: N/A
Reference Count: N/A
A Generalizability Theory Approach toward Estimating Standard Errors of Cutscores Set Using the Bookmark Standard Setting Procedure.
Lee, Guemin; Lewis, Daniel M.
The Bookmark Standard Setting Procedure (Lewis, Mitzel, and Green, 1996) is an item-response-theory-based standard setting method that has been widely implemented by state testing programs. The primary purposes of this study were to: (1) estimate standard errors for cutscores that result from Bookmark standard settings under a generalizability theory model; and (2) to investigate the effects of different universes of generalization and several error sources on the standard errors. Data were obtained from a Bookmark standard setting conducted in grades 5, 8, and 11 mathematics for a large-scale assessment. The findings show that different patterns of error scores are found for different cut scores. Therefore, it seems necessary to estimate separate variance components for each cut score and to apply them to estimate the corresponding standard error. In addition, different universes of generalization produce different standard error estimates. As a result, policymakers should consider which universe is appropriate for the proposed use of the cutscores. There were also nonnegligible effects of participants and groups among several error sources. Increasing the number of small groups tended to be more efficient than increasing the number of participants per group given a fixed number of participants to decrease the standard errors. (Contains 5 tables, 7 figures, and 10 references.) (Author/SLD)
Publication Type: Numerical/Quantitative Data; Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A