ERIC Number: ED390897
Record Type: RIE
Publication Date: 1993-May
Reference Count: N/A
Logical Consistency of the Angoff Method of Standard Setting.
DeMauro, Gerald E.; Powers, Donald E.
Standard setting on licensure and certification tests is difficult both to execute and to defend. There may, however, be certain minimum standards for standard setting on which most will be able to agree. One such standard is logical consistency. M. T. Kane (1984, 1986) has suggested an approach to evaluating the logical consistency of one widely used method to set passing scores--the Angoff procedure (W. H. Angoff, 1971). This approach is applied here to the standard setting data from a study of the NTE Specialty Area Test for School Psychology. In brief, the Angoff procedure involves obtaining judges estimates of the probability with which minimally competent examinees can be expected to answer correctly each item in a test. For this study, 130 items from the test were used with 19 panelists who were members of the National Association of School Psychologists. The study provided modest evidence that the Angoff procedure does yield results that display a relatively high degree of logical consistency, especially as judged from the mean estimates provided by a panel of judges. (Contains 3 tables and 15 references.) (Author/SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: Educational Testing Service, Princeton, NJ.
Identifiers: Angoff Methods; Consistency (Behavior); NTE Specialty Area Tests
Note: Version of a paper presented at the Annual Meeting of the National Council on Measurement in Education (Boston, MA, April 17-19, 1990).