ERIC Number: ED340737
Record Type: Non-Journal
Publication Date: 1991-Nov
Reference Count: N/A
Sources of Variability in the Angoff Standard-Setting Process.
Halpin, Glennelle; McLean, James E.
Although the standard-setting method of W. H. Angoff (1971) has broad-based support in the research literature, inconsistencies in the resulting standards do occur. Sources of these inconsistencies are examined in a study of judges, competencies (items), rounds (replications), and the interactions among them. A modified Angoff approach was used to set standards on the Alabama High School Graduation Examination (AHSGE), Second Edition. Thirty-four subject matter high school teachers (judges) convened to set standards for the AHSGE. Data from round-two estimates if made, or round-one estimates otherwise, were analyzed using three-way factorial analysis of variance, with judges, rounds, and competencies as independent variables. The analysis was repeated for reading, language, and mathematics. For language and mathematics, judges accounted for the largest amount of variability. For reading, competencies were the largest source of variability. Although variability was the focus, the high degree of agreement found is noted, lending credence to the standards set despite some variability. Three tables present study data, and a 15-item list of references is included. (SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Alabama High School Graduation Examination