Researchers in higher education frequently evaluate institutional effectiveness as the difference between an actual and predicted graduation rate, but little is known about whether such a method is reliable or valid. This study examines the measurement properties of effectiveness scores derived from regression residuals for community colleges in the United States (n = 875). The test--retest reliability of the resulting effectiveness measure over two consecutive years was acceptable to good (r = 0.63 to 0.72). A test of convergent validity indicated that effectiveness scores were positively associated with students' perceptions of a supportive campus environment (r = 0.26). A test of discriminant validity revealed relatively small correlations between effectiveness scores and institutional attributes, such as the percent of low-income students (r = -0.01). The results support the use of effectiveness scores as a complementary performance indicator for community colleges to better articulate the value added to the student experience.