ERIC Number: ED248270
Record Type: RIE
Publication Date: 1984-Apr
Reference Count: 0
An Evaluation of Methods for Setting Standards on the Essay Portion of the National Teacher Examinations.
Busch, John Christian; Jaeger, Richard M.
This study addressed seven questions regarding the methods used in setting passing scores on the essay subtest of the National Teacher Examinations (NTE) Communication Skills test for the North Carolina State Board of Education. North Carolina uses these tests to screen prospective applicants to teacher education programs. The judges (five college faculty members and two public school teachers) rated two sets of eleven essay responses to a question from the 1982 national NTE administration provided by Educational Testing Service (ETS). The first essay set was presented in order of increasing ETS assigned scores, the second in random order. Judges rated the essays as "acceptable" or "unacceptable." They were given the ETS holistic scoring for each essay, decided which ETS rating represented a minimally acceptable essay, and rank ordered the essays. A minimal standard for each judge was computed by two methods. The resulting data were analyzed descriptively due to the small sample size. Results suggested that to produce the smallest variance among judges' standards, essays should be presented in increasing order of quality (as scored by ETS readers), but without prior knowledge of ETS ratings. Rating knowledge created an apparent positive bias in recommended standards. (BS)
Publication Type: Speeches/Meeting Papers; Reports - Evaluative
Education Level: N/A
Authoring Institution: N/A
Identifiers: Angoff Methods; Educational Testing Service; National Teacher Examinations; North Carolina
Note: Paper presented at the Joint Annual Meeting of the American Educational Research Association and the National Council on Measurement in Education (New Orleans, LA, April 23-27, 1984).