NotesFAQContact Us
Search Tips
ERIC Number: ED413362
Record Type: Non-Journal
Publication Date: 1996
Pages: 30
Abstractor: N/A
Reference Count: N/A
Interjudge Variability and Intrajudge Consistency Using the Cognitive Components Model for Standard Setting.
McGinty, Dixie; Neel, John H.; Hsu, Yu-Sheng
The cognitive components standard setting method, recently introduced by D. McGinty and J. Neel (1996), asks judges to specify minimum levels of performance not for the test items, but for smaller portions of items, the component skills and concepts required to answer each item correctly. Items are decomposed into these components before judges convene. A preliminary study supported the usefulness of the approach and suggested that it was equal to the Angoff method in most respects and resulted in lower interjudge variability. In this study, the original study was replicated with a different set of judges. Twelve judges, all third- and fourth-grade teachers, set standards for a Georgia criterion-referenced mathematics test for third grade. Forty-five test items were used. Each judge provided ratings using both the Angoff method and the cognitive components method. Results are encouraging with regard to the potential of the cognitive components model as an alternative to standard setting. As in the initial study, the cognitive components model resulted in lower variability among judges at all levels of the process. This may suggest that a substantial proportion of the disagreement among judges using the Angoff method is due to judges' differing abilities to perceive the important characteristics of items. (Contains 6 tables and 21 references.) (SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A