ERIC Number: EJ782115
Record Type: Journal
Publication Date: 2008
Pages: 17
Abstractor: Author
ISBN: N/A
ISSN: ISSN-0013-1644
EISSN: N/A
Estimating Standard Errors of Cut Scores for Item Rating and Mapmark Procedures: A Generalizability Theory Approach
Yin, Ping; Sconing, James
Educational and Psychological Measurement, v68 n1 p25-41 2008
Standard-setting methods are widely used to determine cut scores on a test that examinees must meet for a certain performance standard. Because standard setting is a measurement procedure, it is important to evaluate variability of cut scores resulting from the standard-setting process. Generalizability theory is used in this study to estimate standard errors of cut scores resulting from two standard-setting methods: item rating (Angoff-based) and mapmark (bookmark-based) methods. In this study, two different generalizability (G) study designs and four different decision (D) study designs were examined, and the impact of varying different aspects of the study design and universes of generalization was examined. Results suggest that cut scores were generally consistent for both methods. The first round standard setting contributed the most to the overall variability for the mapmark method. Also, it is clear that there is no "one" standard error associated with a certain cut score. (Contains 5 tables.)
Descriptors: Generalizability Theory, Standard Setting, Cutting Scores, Test Items, Error of Measurement, Standards, Research Design
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A