ERIC Number: ED190604
Record Type: RIE
Publication Date: 1980-Apr
Reference Count: 0
A Comparison of Item-Writing Methods for Criterion-Referenced Tests.
Roid, Gale; And Others
Using informal, objectives-based, or linguistic methods, three elementary school teachers and three experienced item writers developed criterion-referenced pretests-posttests to accompany a prose passage. Item difficulites were tabulated on the responses of 364 elementary students. The informal-subjective method, used by many achievement test developers, allowed maximum freedom of wording and yielded significantly more difficult items. The objectives-based and linguistic methods, in which the item writer chose the foils (distractors), were susceptible to item writer bias. In contrast, the method which provided maximum control, linguistic-based algorithmic foil method, yielded the items which were easy and not subject to bias. It also created items which were insensitive to the pretest-posttest shift in difficulty. Therefore, algorithmic foil methods are promising because they control item writer differences; more research is needed before reasonable item difficulties and instructional sensitivity can be obtained. Teacher-produced items were more sensitive to instruction than those of experienced writers. It is concluded that item-writer differences are real and that field testing is important to identify these differences. (CP)
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Sponsor: Advanced Research Projects Agency (DOD), Washington, DC.
Authoring Institution: Oregon State System of Higher Education, Monmouth. Teaching Research Div.
Identifiers: Distractors (Tests)
Note: Paper presented at the joint Annual Meetings of the American Educational Research Association and the National Council on Measurement in Education (Boston, MA, April 7-11, 1980).