PDF pending restoration
ERIC Number: ED457195
Record Type: RIE
Publication Date: 2000-Nov
Reference Count: N/A
Analyzing Multidimensional Response Data Structure Represented by Unidimensional IRT Models To Increase the Precision of Scoring Using Out-of-Scale Information.
Capar, Nilufer K.
This study investigated specific conditions under which out-of-scale information improves measurement precision and the factors that influence the degree of reliability gains and the amount of bias induced in the reported scores when out-of-scale information is used. In-scale information is information that an item provides for a composite trait to which it is specifically classified by a content review. Out-of-scale information is defined as information that an item provides for a composite trait other than the composite to which it is specifically classified. Computer simulated two-dimensional data were used to investigate the effects of various factors in the precision of reported scores computed with and without out-of-scale information. Different testing conditions were specified to compare the performance of the traditional information computation method (in-scale information alone; Method A) and the alternative method (in- and out-of-scale information together; Method B). The findings for standard error indicate that random measurement error is more evident with Method A. Bias results show that ability estimates at high and low theta levels are pulled toward the mean with both methods. The root mean square errors reflecting the combined effects of standard errors and bias differed with different conditions and test lengths, as discussed. The choice between the in-scale and out-of-scale procedures would seem to depend primarily on standard error and bias. Because Method B resulted in lower standard errors and usually performed better with the short test moderate discrimination conditions, this alternative method is likely to be attractive. Using out-of-scale information is likely to produce reliable subscores that can be used for diagnostic purposes. (Contains 4 tables, 11 figures, and 25 references.) (SLD)
Descriptors: Ability, Adaptive Testing, Computer Assisted Testing, Item Response Theory, Mathematics, Reading Comprehension, Scaling, Scores, Simulation, Test Bias, Test Items
Paper presented at the Annual Meeting of the Florida Educational Research Association (45th Tallahassee, FL, November 8-10, 2000).
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A