NotesFAQContact Us
Search Tips
ERIC Number: ED422352
Record Type: Non-Journal
Publication Date: 1998-Apr
Pages: 27
Abstractor: N/A
Reference Count: N/A
A Meta-Analytic Assessment of Empirical Differences in Standard Setting Procedures.
Bontempo, Brian D.; Marks, Casimer M.; Karabatsos, George
Using meta-analysis, this research takes a look at studies included in a meta-analysis by R. Jaeger (1989) that compared the cut score set by one standard setting method with that set by another. This meta-analysis looks beyond Jaeger's studies to select 10 from the research literature. Each compared at least two types of standard setting method. To assess the difference in cut scores produced by each standard setting method, a common metric was used for every cut-score comparison. The standardized magnitude of the difference between two compared cut scores, called the effect size, was calculated, with the Angoff method results treated as the control group in effect size calculations. Depending on the conceptualization chosen, results of this analysis may be interpreted differently. For a fixed effects model approach, some standard setting methods produce significantly different cut scores. This conclusion must be tempered by the amount of heterogeneity in the model. If a random effects model is endorsed, it is recognized that no significant differences between methods are produced. This conclusion also must be taken with caution because of the relationship between the within-method variation and the between-method variation. Although the analysis results do not make it possible to make conclusive statements about systematic differences in effect sizes and cut scores produced by the standard setting methods presented, meta-analysis holds much promise in its ability it answer these questions in the future. (Contains 4 tables, 3 figures, and 26 references.) (SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A