ERIC Number: ED395035
Record Type: RIE
Publication Date: 1995-Mar
Reference Count: N/A
Estimating the Effects of Test Length and Test Time on Parameter Estimation Using the HYBRID Model.
The traditional indicator of test speededness, missing responses, clearly indicates a lack of time to respond (thereby indicating the speededness of the test), but it is inadequate for evaluating speededness in a multiple-choice test scored as number correct, and it underestimates test speededness. Conventional item response theory (IRT) parameter estimation ignores the mixture of random response during calibration; consequently, estimated parameters are biased. The HYBRID model (K. Yamamoto, 1989) was extended (Yamamoto, 1990) to characterize when each examinee switches from an ability-based response strategy to a strategy of responding randomly. The model has allowed the evaluation of test speededness by estimating the proportions of examinees who switch strategies at any possible point in the test. The estimated IRT parameters based on the HYBRID model were more accurate than the ordinary IRT-only analysis. With the extended HYBRID model applied to the data taken from an experimental form of the Test of English as a Foreign Language it was found that: (1) test length had a small impact on the proportion of examinees affected by test speededness; (2) a greater proportion of examinees was affected by speededness of a test with a 50-minute time limit than a test with a 55- or 60-minute time limit; and (3) the difference in the proportions of examinees affected by the speededness of tests under 55- and 60-minute time limits was small. However, nearly 20% of examinees were affected by speededness after completing 80% of the test, so that the last 20% of responses of 20% of examinees did not represent their true ability. (Contains 4 tables, 16 figures, and 18 references.) (Author/SLD)
Publication Type: Reports - Evaluative
Education Level: N/A
Authoring Institution: Educational Testing Service, Princeton, NJ.
Identifiers: HYBRID Model; Item Parameters; Missing Data; Speededness (Tests); Test of English as a Foreign Language