ERIC Number: ED263120
Record Type: Non-Journal
Publication Date: 1985-Feb
Reference Count: N/A
A Preliminary Study of Raters for the Test of Spoken English.
Bejar, Isaac I.
The feasibility of reducing scoring costs for the Test of Spoken English (TSE) by using one rater was investigated. Currently, two raters are used. It was found that, because of the possibility of different standards used by potential raters, it does not appear feasible to use a single rater as the sole determiner of speaking proficiency under the current system. Other possible alternatives were also examined. One approach was the development of a quality control index which would predict the extent of the disagreement between two raters, but the index that was developed could not be validated. The best predictors of rater disagreement were the identities of the raters. Their disagreements, however, resulted from the differing standards they used. Raters agreed substantially about the ordering of examinees, but varied slightly in the severity of their ratings. (Author/GDC)
Descriptors: Analysis of Covariance, Cost Effectiveness, English (Second Language), Evaluation Criteria, Evaluation Methods, Evaluators, Factor Structure, Feasibility Studies, Higher Education, Interrater Reliability, Language Tests, Oral English, Scoring, Speech Tests, Standards, Test Reliability, Test Validity
Publication Type: Reports - Research
Education Level: N/A
Authoring Institution: Educational Testing Service, Princeton, NJ.