ERIC Number: ED282903
Record Type: Non-Journal
Publication Date: 1987-Apr
Pages: 15
Abstractor: N/A
ISBN: N/A
ISSN: N/A
EISSN: N/A
An Experimental Study of the Effect of Judges' Knowledge of Item Data on Two Forms of the Angoff Standard Setting Method.
Garrido, Mariquita; Payne, David A.
Minimum competency cut-off scores on a statistics exam were estimated under four conditions: the Angoff judging method with item data (n=20), and without data available (n=19); and the Modified Angoff method with (n=19), and without (n=19) item data available to judges. The Angoff method required free response percentage estimates (0-100) percent, whereas the Modified Angoff method used a seven point scale (5, 20, 40, 60, 75, 90, 95). Item data were in the form of frequency distributions of multiple-choice exam option selections for five student ability levels. Results indicated significantly: (1) greater intra-judge variability for the Angoff method; and (2) higher minimum competency cut-off scores for judges with performance data available for reference than for judges who did not have data available. Acceptable levels of inter-judge reliability, judge internal consistency, and item internal consistency were also observed. (Author)
Descriptors: Academic Standards, Comparative Analysis, Criterion Referenced Tests, Cutting Scores, Difficulty Level, Higher Education, Interrater Reliability, Judges, Minimum Competency Testing, Multiple Choice Tests, Standard Setting (Scoring), Statistical Distributions, Statistics, Test Items, Test Theory
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Audience: Researchers
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A