ERIC Number: ED331878
Record Type: RIE
Publication Date: 1991-Mar-5
Reference Count: N/A
Influence of the Criterion Variable on the Identification of Differentially Functioning Test Items Using the Mantel-Haenszel Statistic. Lab Report 198.
Clauser, Brian E.; And Others
This paper explores the effectiveness of the Mantel-Haenszel (MH) statistic in detecting differentially functioning test items when the internal criterion is varied. Using a data set from the 1982 statewide administration of a 150-item life skills examination (the New Mexico High School Proficiency Examination), a randomly selected sample of 1,000 Anglo-American and 1,000 Native American examinee item response sets was analyzed. The MH procedure was first applied to all of the items involved. The items were then categorized as belonging to one or more of four subtests based on the skills or knowledge needed to choose the correct response. Each subtest was then analyzed as a separate test, using the MH procedure. Three control subtests were also established using random assignment of test items, and they were analyzed using the MH procedure. Based on three runs of a MH computer program analyzing a total of 91 items, 22 items were identified as functioning differentially in the two examinee groups. The choice of criterion--total test score versus subtest score--had a substantial influence on the classification of items according to whether or not they were differentially functioning in the Anglo-American and Native American groups. Evidence for the convergence of judgmental and statistical procedures was found in the unusually high proportion of differentially functioning items within one of the classifications, and in the results of the reanalysis of this group of items. Two tables present data from the analyses. (Author/SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Sponsor: Massachusetts Univ., Amherst. School of Education.
Authoring Institution: N/A
Identifiers: Criterion Variables; Mantel Haenszel Procedure; New Mexico High School Proficiency Examination
Note: Paper presented at the Annual Meeting of the American Educational Research Association (Boston, MA, April 16-20, 1990).