ERIC Number: ED391812
Record Type: Non-Journal
Publication Date: 1991-Feb
Reference Count: N/A
Development of Procedures for Resolving Irregularities in the Administration of the Listening Comprehension Section of the TOEFL Test.
Way, Walter D.; McKinley, Robert L.
Two procedures were developed to determine whether examinees in a given test center were affected by a testing irregularity on the Listening Comprehension section of the Test of English as a Foreign Language (TOEFL). One approach employed analysis of covariance (ANCOVA) on Listening Comprehension (Section 1) means using scores on Structure and Written Expression (Section 2) and scores on Reading and Vocabulary (Section 3) as covariates. The second procedure entailed a Bayesian approach that used prior information about performance at the center in question. Analyses using these two procedures were carried out with simulated data and data from actual testing irregularities at 5 centers involving 639 affected examinees and numerous comparisons. Results indicated that both ANCOVA and Bayesian approaches provided useful information. They usually agreed, but differences occurred in specific situations as discussed. Although both procedures should be incorporated into the operational procedures for resolving testing irregularities, results should be interpreted with caution, particularly if they produce discrepant results. An appendix contains five tables of summary statistics. (Contains 1 figure, 10 tables, and 3 references.) (Author/SLD)
Publication Type: Reports - Evaluative
Education Level: N/A
Authoring Institution: Educational Testing Service, Princeton, NJ.
Identifiers - Assessments and Surveys: Test of English as a Foreign Language