ERIC Number: ED393941
Record Type: RIE
Publication Date: 1989-Jan
Reference Count: N/A
Context Bias in the Test of English as a Foreign Language.
Angoff, William H.
This study was undertaken to test the hypothesis that items of the Test of English as a Foreign Language (TOEFL) containing reference to American people, places, customs, etc., tend to favor examinees who have spent some time living in the United States. Two samples of examinees were drawn from the March 1987 TOEFL administration, one tested in the United States consisting of 5,799 persons living in the country more than 1 year, and the other consisting of individuals with less than 1 month residence in the United States tested in their native countries (n=21,652 worldwide). Mantel Haenszel analyses were carried out for each of the 146 operational items of the test. In a separate part of the study, five raters were asked to rate test items to determine explicit references to some aspect of America. Ratings were highly reliable, and all but 16 of the 146 items were unanimously judged to have or not to have some reference to Americana. Of the TOEFL items, however, only one gave a consistent advantage, found in every region studied, to persons with U.S. residence experience. No support was provided for the hypothesis that TOEFL items give an advantage to persons who have lived in the United States for some time. Four appendixes contain the distribution of candidates by region, instructions for a pilot study and the formal study, and a display of raters' assignments of Americana to each test item. (Contains 3 figures, 22 tables, and 11 references.) (Author/SLD)
Publication Type: Reports - Research; Numerical/Quantitative Data
Education Level: N/A
Authoring Institution: Educational Testing Service, Princeton, NJ.
Identifiers: Americanisms; Item Bias Detection; Mantel Haenszel Procedure; Test of English as a Foreign Language; United States
Note: Table 10 contains very small print.