ERIC Number: ED441027
Record Type: Non-Journal
Publication Date: 1999-Apr
Reference Count: N/A
The Effects of Judgment-Based Stratum Classifications on the Efficiency of Stratum Scored CATs.
Finney, Sara J.; Smith, Russell W.; Wise, Steven L.
Two operational item pools were used to investigate the performance of stratum computerized adaptive tests (CATs) when items were assigned to strata based on empirical estimates of item difficulty or human judgments of item difficulty. Items from the first data set consisted of 54 5-option multiple choice items from a form of the ACT mathematics assessment. Items from the second data set were drawn from a computerized algebra test with sample sizes of 250 to 300 examinees for each of the 140 options. Each of the 11 judges independently sorted items into difficulty strata for one of the two datasets. It was found that stratum CATs based on empirical item difficulties (both p-values and b-parameters) have increased efficiency and precision relative to a conventional fixed-length test. It was also shown that efficiency and precision increased as the number of strata increased. Also, under certain conditions, the stratum CAT was able to match or exceed the precision and efficiency of the traditional CAT. These findings provide evidence of the promise stratum CATs have as a non-Item Response Theory adaptive testing method that requires minimal item data. The results from the stratum CAT based on human judgment were not promising. Findings suggest that stratum CATs based on human judgments do not provide increased efficiency or precision over a conventional fixed-length test. (SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A