Perceived shortage of principals, increased accountability demands, the changing role of principals, and the growing influence of the state over school administration have created a set of challenges for principal certification and licensure that have propelled a renewed need for analysis of certification and licensure processes. Recent calls to hold preparation programs accountable for outcomes have prompted states to develop and adopt preparation program accountability systems. One primary feature of these systems is the success of program graduates on a state test leading to principal certification. Yet, little research has examined the validity and reliability of such state tests in assessing even the likely effectiveness of aspiring principals, let alone predicting their actual effectiveness once employed as a principal. This conceptual study utilizes theoretical analysis of extant literature and empirical analysis of extant data from one state (Florida) to reach conclusions about the appropriateness of using a state test to (1) credential aspiring educational leaders; and (2) evaluate principal-preparation programs.