ERIC Number: EJ724937
Record Type: Journal
Publication Date: 2005
Reference Count: 0
IRT Item Parameters and the Reliability and Validity of Pretest, Posttest, and Gain Scores
May, Kim; Jackson, Tameika S.
International Journal of Testing, v5 n1 p63-73 2005
The effect of different combinations of item response theory (IRT) item parameters (item difficulty, item discrimination, and the guessing probability) on the reliability and construct validity (correlation with the latent trait being measured) of pretest, posttest, and gain scores was analytically examined using the 3-parameter logistic (3PL) model (Birnbaum, 1968). The analyses were performed at the single-item level using items with specified combinations of item parameters. In general, the pretest item validities and reliabilities were much higher than those of the gain score. It is also the case that the reliability of the gain score is at a minimum when that of the pretest score is at a maximum, namely, near the middle of the distribution of item difficulty (i.e., for moderately difficult tests). Furthermore, using IRT we show that the regression of the gain score on the latent trait being measured is nonmonotonic, meaning a person at a given low ability level has the same expected gain score as one of the corresponding high ability. These results not only are of theoretical interest but also may be of interest to applied practitioners and test developers.
Descriptors: Pretests Posttests, Guessing (Tests), Probability, Scores, Reliability, Validity, Construct Validity, Item Response Theory, Evaluation Methods, Data Analysis
Lawrence Erlbaum Associates, Inc., Journal Subscription Department, 10 Industrial Avenue, Mahwah, NJ 07430-2262. Tel: 800-926-6579 (Toll Free); e-mail: firstname.lastname@example.org.
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Authoring Institution: N/A
Identifiers: Gain Scores; Item Parameters