The purpose of this study was to rate oral retellings of fifth graders to determine how passages, raters, and rating occasions affect those ratings, and to identify what combination of those elements produce reliable retelling ratings. A group of 36 fifth grade students read and orally retold three contemporary realistic fiction passages. Two researchers rated these retellings on two separate occasions using the Reader Retelling Rating Scale (3-RS). These ratings were analyzed using generalizability software. Results showed that to obtain high reliability coefficients, retellings of a minimum of four (preferably six) passages should be rated by at least two raters on one occasion. (Contains 1 table and 6 figures.)