ERIC Number: ED297281
Record Type: RIE
Publication Date: 1988-Jul
Reference Count: 0
Informal Reasoning Assessment: Using Verbal Reports of Thinking to Improve Multiple-Choice Test Validity. Technical Report No. 430.
Norris, Stephen P.
A study examined whether the process of gathering verbal reports of subjects' thinking while taking multiple-choice critical thinking tests could be used to infer the reasoning process used and identify test items which do not require critical thinking skills. Four factors can render an inference of a subject's critical thinking skills untrustworthy: (1) the degree of informal reasoning sophistication of the subject; (2) the background empirical beliefs of the subject; (3) the assumptions which the subject brings to test items; and (4) the political and religious ideologies of the subject. Subjects, 343 senior high school students from four high schools, were divided into five groups for taking a multiple choice test. One group took the test in the normal manner while the other four groups gave verbal reports of their thinking for each question. Examiners asked subjects in these four groups leading questions to investigate whether the thinking processes of the subjects would be altered. Subjects were given a performance score equal to the number of correct answers and a "thinking score" which indicated the quality of thinking displayed in the verbal reports. Results indicated that there were no statistically significant differences in the performance scores of the five groups nor in the thinking scores of the four groups that used verbal reports. (One table of data and 30 references are attached.) (RS)
Publication Type: Reports - Research
Education Level: N/A
Sponsor: Social Sciences and Humanities Research Council of Canada, Ottawa (Ontario).
Authoring Institution: Illinois Univ., Urbana. Center for the Study of Reading.; Bolt, Beranek and Newman, Inc., Cambridge, MA.
Note: For a related document, see CS 009 232.