ERIC Number: ED360844
Record Type: Non-Journal
Publication Date: 1993-Aug-3
Reference Count: N/A
Evaluating the Efficacy of Rater Self-Training.
Kenyon, Dorry; Stansfield, Charles W.
This paper examines whether individuals who train themselves to score a performance assessment will rate acceptably when compared to known standards. Research on the efficacy of rater self-training materials developed by the Center for Applied Linguistics for the Texas Oral Proficiency Test (TOPT) is examined. Rater self-materials are described and analysis of data is reported from a study conducted during their development. Eight individuals worked through the materials on their own and submitted qualitative feedback on the materials and on their experience with them. They also participated in a calibration study in which they independently rated recorded segments from a TOPT administration. These ratings were analyzed by traditional approaches and by a multifaceted Rasch approach. Findings indicate that the raters as a group scored the TOPT consistently, although differences in rater severity led to some important disagreements with the rating key. The study illustrates the role of background characteristics and motivation in the success of rater self-training. (Contains 8 references.) (JL)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: Center for Applied Linguistics, Washington, DC.