ERIC Number: ED389714
Record Type: RIE
Publication Date: 1991
Reference Count: N/A
DIFferential Testlet Functioning Definitions and Detection. Program Statistics Research Technical Report No. 91-9.
Wainer, Howard; And Others
It is sometimes sensible to think of the fundamental unit of test construction as being larger than an individual item. This unit, dubbed the testlet, must pass muster in the same way that items do. One criterion of a good item is the absence of differential item functioning (DIF). The item must function in the same way as all important subpopulations of examinees. In this paper, "testlet DIF" is defined, and a statistical methodology is provided to detect it. This methodology parallels the item response theory-based likelihood ratio procedures explored previously by D. Thissen, L. Steinberg, and H. Wainer (1988). The methodology is illustrated with analyses of data from a testlet-based experimental version of the Scholastic Aptitude Test. The illustration demonstrated that, in this instance, not modeling the testlet structure overestimated the reliability of the test by an amount equivalent to a test of doubled length. In addition, testlets made up of what appeared to be exemplary items exhibited significant sex DIF when testlets were considered "in toto." It was also found that testlets with modest DIF in both directions can still be fair at score and proficiency levels. (Contains 4 tables, 10 figures, a technical appendix, and 27 references.) (Author/SLD)
Publication Type: Reports - Evaluative
Education Level: N/A
Authoring Institution: Educational Testing Service, Princeton, NJ.
Identifiers: Likelihood Ratio Criterion; Scholastic Aptitude Test; Testlets