ERIC Number: EJ906075
Record Type: Journal
Publication Date: 2010
Abstractor: As Provided
Reference Count: 40
A Matter of Significance: Can Sampling Error Invalidate Cloze Estimates of Text Readability?
O'Toole, John Mitchell; King, Robert A. R.
Language Assessment Quarterly, v7 n4 p303-316 2010
This quantitative study intends to better understand the impact of the location of the first deleted word upon the estimation of text difficulty yielded by successive cloze tests based on random deletion from a single passage. The variation in sampling of language features across five cloze tests based on the same passage is random and thus not statistically significant. However, there is a statistically significant difference in result distributions of matched groups after scoring for exact replacement. These differences in performance appear partly due to differences in the frequencies of nouns, verbs, and grammatical metaphor between the different tests derived from the base passage. Although the differences between the tests in this investigation appear to fall within the tolerances indicated by commonly used criterion scores, the range of mean scores for particular school grades suggests that more generalised estimation of text difficulty should be based on exhaustive sampling rather than relying on one test based on a specific passage. This study is significant because it illuminates widespread concerns regarding the usefulness of cloze techniques for the purpose for which they were first devised and suggests reasons for the impact of two forms of sampling error. (Contains 1 figure and 4 tables.)
Descriptors: Cloze Procedure, Readability, Nouns, Figurative Language, Science Tests, Statistical Analysis, Sampling, Matched Groups, Verbs, Grammar, Scores, Error of Measurement, Secondary School Students, Science Instruction, Language Patterns, Foreign Countries
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Evaluative
Education Level: Secondary Education
Authoring Institution: N/A
Identifiers - Location: Australia