ERIC Number: ED417598
Record Type: Non-Journal
Publication Date: 1996-Sep-25
Reference Count: N/A
A Study of Raters' Scoring Tendency of Speaking Ability through Verbal Report Methods and Questionnaire Analysis.
Journal of Communication Studies, n5 p3-17 Sep 1996
To find ways to improve rater reliability of a tape-mediated speaking test for Japanese university students of English as a Second Language, two studies gathered information on: how raters actually made their choices on rating sheets of students' speaking ability; determined what criteria teachers think they use and actually use in rating students' speaking ability; and drew implications for improving rater reliability. In the first study, subjects were six native English speakers and three Japanese teachers of English. They answered a 16-item questionnaire about rating techniques and criteria, then four months later, rated 30 student tapes, answered the questionnaire again, and gave a retrospective self-report about the internal process of evaluating student skills. One of the subjects also gave a self-report concurrent with rating tapes. In the second study, six audio- and videotape raters analyzed their rating tendencies through verbal reports, self introspective and retrospective reports, and interviews. Results of the two studies and implications for teaching and testing are reported. The instruments (questionnaires, retro/introspective questions, and interview questions) are appended. Contains 3 references. (MSE)
Descriptors: English (Second Language), Evaluation Criteria, Foreign Countries, Interrater Reliability, Language Research, Language Tests, Rating Scales, Scoring, Second Language Instruction, Speech Skills, Student Evaluation, Surveys, Testing, Verbal Tests
The "Journal of Communication Studies" is published by Keizia University, Tokyo, Japan.
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Authoring Institution: N/A
Identifiers - Location: Japan