ERIC Number: ED457164
Record Type: Non-Journal
Publication Date: 2001-May
Pages: 34
Abstractor: N/A
ISBN: N/A
ISSN: N/A
EISSN: N/A
Measurement Decision Theory.
Rudner, Lawrence M.
This paper describes and evaluates the use of decision theory as a tool for classifying examinees based on their item response patterns. Decision theory, developed by A. Wald (1947) and now widely used in engineering, agriculture, and computing, provides a simple model for the analysis of categorical data. Measurement decision theory requires only one key assumption--that the items are independent. The model has a very simple framework in that one starts with the conditional probabilities of examinees in each mastery state responding correctly to each item. The model is attractive for a number of measurement situations. As background to the study presented in this paper, an overview of measurement decision theory and its key concepts are presented and illustrated using a binary classification (pass/fail) test and a sample three-item test. The research section presents an evaluation of the model by examining the: (1) classification accuracy of tests scored using measurement decision theory; (2) differential sequential testing procedures by comparing classification accuracy against that of the best case item response theory scenario; (3) the number of items needed to make a classification; and (4) the number of examinees needed to calibrate measurement decision theory item parameters satisfactorily. Two sets of simulated data were used to address these issues, and item parameters were based on samples of items from the 1999 Colorado State Assessment program fifth grade mathematics test and the 1996 National Assessment of Educational Progress Eighth Grade Mathematics Assessment. The research shows that, using Wald's SPRT, a large percentage of examinees can be classified accurately with very few items. The research also shows that very few pilot test examinees are needed to calibrate the system. The model is clearly a simple yet powerful and widely applicable model. (Contains 4 figures, 7 tables, and 29 references.) (SLD)
Descriptors: Classification, Mathematical Models, Measurement Techniques, Responses, Scoring, Test Items
Publication Type: Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: National Inst. on Student Achievement, Curriculum, and Assessment (ED/OERI), Washington, DC.
Authoring Institution: N/A
Grant or Contract Numbers: N/A