ERIC Number: ED395032
Record Type: RIE
Publication Date: 1990-Jun
Reference Count: N/A
Toward a Framework for Constructed-Response Items.
Bennett, Randy Elliot; And Others
A framework for categorizing constructed-response items was developed in which items were ordered on a continuum from multiple-choice to presentation/performance according to the degree of constraint placed on the examinee's response. Two investigations were carried out to evaluate the validity of this framework. In the first investigation, 27 test development staff assigned 46 items of various formats to the categories. Overall, agreement with the intended item categorizations was good, with a median of 2 of a possible 27 judges disagreeing with a given item's classification. In the second investigation, responses of 40 examinees each to 4 sets of items were scored by test development staff, with each set scored by 4 individuals. Results showed scoring agreement to be highest for a category requiring the examinee to choose a response from an extended stimulus array and lowest for items requiring that the stimulus be reordered to form a correct sequence. Whether the reported agreement levels represent sufficient accuracy to permit the widespread use of such items in standardized tests depends on whether some degree of scoring error, however small, can be accepted. Appendix A gives sample items organized by category, and Appendix B is a scoring guide organized by item category. (Contains 1 figure, 7 tables, and 13 references.) (Author/SLD)
Publication Type: Reports - Evaluative
Education Level: N/A
Authoring Institution: Educational Testing Service, Princeton, NJ.