ERIC Number: ED193731
Record Type: RIE
Publication Date: 1980-May
Reference Count: 0
Increasing Inter-Judge Agreement in Forensic Events.
Lewis, James J.; Larsen, Janet K.
A study was designed to see how judge experience and training sessions influenced interjudge agreement during high school speech contests. Three groups of six judges each viewed a videotape of five poetry readings, gave the performers ratings and rankings, engaged in 30-minute group training sessions, viewed a second videotape of five poetry readings, and rated and ranked the second set of performers. The groups of judges ranged from inexperienced through moderately experienced to highly experienced as oral interpretation judges. Correlations from rankings revealed that training increased the agreement among judges, with agreement after training sessions being quite high; and that training the experienced judges caused the greatest increases in agreement (the more experience judges had, the less they agreed). Experienced judges seemed to use different criteria from one another when discriminating between readers of similar ability, but consistently agreed regarding larger differences of ability between readers. Inexperienced judges seemed to apply one common standard to similar readers, but had no ability to determine overall quality of the group. Training did seem to give inexperienced judges a yardstick of quality. These results suggested ways of maximizing agreement between judges, depending on the number of groups of speakers to be judged, the experience of the judges, training opportunities, and whether each judge evaluated all speakers. (RL)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Note: Paper presented at the Annual Meeting of the International Communication Association (30th, Acapulco, Mexico, May 18-23, 1980).