South Carolina's School Incentive Reward Program Program (SIRP) based on year-to-year gains in reading and mathematics achievement, which seemed to have face validity, was evaluated, particularly its use of the relative gains of students in only the highest grade in the school as the basis for the awards. When this requirement was relaxed so that the results of the students in the next lower grade in the school were used, award decisions based on the two analyses were inconsistent, essentially random. The data were obtained from South Carolina's Basic Skills Assessment Program's (BSAP) criterion referenced tests administered to all students in grades 1-3, 6, 8, and 11, and the Comprehensive Tests of Basic Skills (CTBS) administered in grades 4, 7, and 10. The percentage passing the BSAP tests and the school mean normal curve equivalents on the CTBS subtests defined school-grade performance. Schools with high achievement gains or generally high performance were considered eligible to receive SIRP funds. Of 984 schools studied, 315 were identified as high performance schools based on the highest grade level, and 310 based on the next lower grade. One hundred and three schools were identified on both analyses. It was suggested that policymakers be aware of these problems in effective schools research. (Author/GDC)