This study examined the effects of different test administration conditions on reading comprehension test scores. Evidence of performance differences across district testing conditions might imply that the meanings and interpretations associated with the corresponding test scores have limited generalizability (i.e., knowing how well one reads under one set of conditions might not generalize to performance under other conditions). The issues addressed by this research pertain to the validity of scores from passage-based reading comprehension tests. Three test administration factors were manipulated at grades 3, 5, and 7: (1) whether examinees read a passage before or after the test questions; (2) whether examinees were allowed to review a passage while they were answering questions about it; and (3) whether examinees received prior training (practice) consistent with the conditions under which they took the test. The average number of subjects in each condition was about 66. Iowa Tests of Basic Skills Vocabulary scores were treated as an individual difference variable. The primary dependent variables were total test scores (obtained under standard and extended time limits), skills scores (facts, inferences, and generalizations), and work rates (at 20 and 42 minutes). The results suggest that alternative testing conditions can have complex effects on work rates and test scores that can interact with the ability and grade levels of students. As a main effect, training had little influence on work rates or test scores. No-passage-review administrations were associated with greater working rates, but lower test scores under extended time limits. Questions-first administrations were associated with lower working rates and lower test scores under standard time limits. This finding does not support the beliefs that some hold about the advantages of reading the questions before the passages. (Contains 1 table, 4 figures, and 12 references.) (Author/SLD)