ERIC Number: ED366172
Record Type: RIE
Publication Date: 1993-Apr
Reference Count: N/A
Decision Dependability of Subtests, Tests, and the Overall TOEFL Test Battery.
Brown, James Dean; Ross, Jacqueline A.
This study investigates the Test of English as a Foreign Language (TOEFL), in particular the relative contributions to score dependability (analogous to classical theory reliability) of various numbers of items and subtests as well as the decision dependability at different cut points. Research questions that apply to the overall TOEFL battery and to its tests and subtests address classical theory reliability estimates; relative contributions to error variance of persons, items, subtests, and their interactions; dependability for varying numbers of items and subtests; and the effect on score dependability of various cutpoints. The study was based on the item responses of 20,000 test takers from 15 different language backgrounds. Data were collected from the May 1991 administration of the TOEFL at foreign and domestic test centers. Analyses included descriptive statistics, classical theory reliability estimates, generalizability theory, and decision dependability estimates for various cut points. Test dependability analyses indicate that subtests can make substantial contributions to the variance of test scores and thus may affect dependability in important ways. However, these results also make it clear that, in some cases, subtests may have a negligible impact on dependability. Thus, while inclusion of subtests or the expansion of the number of subtest on a test may have a substantial beneficial effect on the dependability of the scores on that test, this relationship cannot be taken as a forgone conclusion. Findings also indicate that on the present TOEFL the lowest dependabilities along the range are still very high. Analyses are appended. (Contains 20 references.) (AA)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: Hawaii Univ., Manoa.; Educational Testing Service, Princeton, NJ.
Identifiers: Test of English as a Foreign Language
Note: Paper presented at the Annual Meeting of the Language Testing Research Colloquium (Cambridge, England, United Kingdom, April 1993).