ERIC Number: ED248264
Record Type: RIE
Publication Date: 1983-Jul
Effect of Examinee Certainty on Probabilistic Test Scores and a Comparison of Scoring Methods for Probabilistic Responses.
Suhadolnik, Debra; Weiss, David J.
The present study was an attempt to alleviate some of the difficulties inherent in multiple-choice items by having examinees respond to multiple-choice items in a probabilistic manner. Using this format, examinees are able to respond to each alternative and to provide indications of any partial knowledge they may possess concerning the item. The items used in this study were 30 multiple-choice analogy items. Examinees were asked to distribute 100 points among the four alternatives for each item according to how confident they were that each alternative was the correct answer. Each item was scored using five different scoring formulas. Three of these scoring formulas were reproducing scoring systems. Results from this study showed a small effect of certainty on the probabilistic scores in terms of the validity of the scores but no effect at all on the factor structure or internal consistency of the scores. Once the effect of certainty on the probabilistic scores had been ruled out, the five scoring formulas were compared in terms of validity, reliability, and factor structure. There were no differences in the validity of the scores from the different methods. (Author/BW)
Publication Type: Reports - Research
Education Level: N/A
Sponsor: Office of Naval Research, Arlington, VA. Personnel and Training Research Programs Office.
Authoring Institution: Minnesota Univ., Minneapolis. Dept. of Psychology.