Although anecdotal evidence and research alike espouse the benefits of service learning, some researchers have suggested that more rigorous testing is required in order to determine its true effect on students. This is particularly true in the case of academic development, which has been inconsistently linked to service learning. It has been proposed that this discrepancy is due to three complications: grades not reflecting higher order thinking skills, self-selection bias, and different grading methods. The study described in this article attempted to circumvent these complications using a test-retest methodology and measuring academic development in three ways: course grades, an assignment that directly tested course-specific comprehension, and self-reported improvement. In addition, improvements in civic responsibility, interpersonal skills, and practical skills were measured via self-report. Although students who participated in service learning self-reported greater improvement in civic responsibility, interpersonal skills, and academic development, they only demonstrated more academic development in terms of concrete course concepts, showing no differences in final examination marks or generation of detailed examples. These findings suggest that academic improvement through service learning may not be adequately assessed by typical methods used to evaluate academic development at universities.