ERIC Number: ED372113
Record Type: Non-Journal
Publication Date: 1994-Apr
Reference Count: N/A
Using an Extended Angoff Procedure To Set Standards on Complex Performance Assessments.
Hambleton, Ronald K.; Plake, Barbara S.
The number of performance-based assessments is increasing rapidly, but to date there is no established procedure for setting standards on these assessments. This paper describes several extensions to the Angoff procedure to accommodate the characteristics of a performance-based assessment and presents the results of research in applying this procedure. The extensions included a revised task for panelists that involved the specification of expected scores for just barely certifiable candidates on polytomously scored exercises and weights to reflect the relative importance of scoring dimensions at the exercise level and of the exercises themselves. Results obtained from using the new procedure with 12 panelists (teachers of adolescents with experience in pilot tests of the assessment) were mixed. On one hand, panelists seemed to find the new procedure straightforward to apply and were able to reach a high level of agreement among themselves about the standards. Confidence levels in the resulting standards were high. However, when given the choice, panelists definitely preferred a standard-setting procedure that was at least partly conjunctive as opposed to the extended Angoff procedure that was completely compensatory in nature. Ten tables and one figure present study findings. (Contains 7 references.) (SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A