NotesFAQContact Us
Search Tips
ERIC Number: ED464946
Record Type: Non-Journal
Publication Date: 2001-Apr
Pages: 26
Abstractor: N/A
Reference Count: N/A
Interpreting the Results of Three Different Standard Setting Procedures.
Green, Donald Ross
Different procedures for setting cut points on achievement test scales provide the standard-setting participants with different information to support the unique judgment task associated with each procedure. This study examined how participants in standard settings used the different information from three different procedures in Kentucky in 2000. Three cut points had to be established on the scales for each of six content areas in each of three grades for Kentucky's assessment system. The procedures were: (1) a contrasting groups procedure; (2) a holistic examination of student work on the tests as described by R. Jaeger and C. Mills (1998); and (3) a bookmark procedure (D. Lewis and others, 1998). Using Kentucky's five-step standard-setting process, the judges (teachers) established preliminary descriptors, conducted each of the three approaches, and then conducted a synthesis process to put together the results of the preceding steps. All told, 1,599 teachers participated in the exercises. Results show that each of the three standard setting procedures produced sets of cut scores for the various tests that can be said to be reasonable and defensible. Given the common backgrounds of the teachers in each of the three sets of judges and the carefully developed sets of descriptors of the categories, it is evident that much of the difference in cut scores stems from the different kinds of information the procedures provide. Of course, it is time consuming to use more than one procedure, and if only a single procedure is wanted, these data suggest the bookmark approach because of an apparently clearer relationship to instruction. The possibilities of the Jaeger-Mills process for helping schools improve could prove real, as many teachers wanted to see more full papers during the synthesis process. (SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Kentucky