There is a growing call for evaluation capacity building (ECB); although the area currently lacks a rich research base, there are few robust methods and practice through which to define it. The argument in this article is that the impact of evaluation is mediated by program stakeholders' engagement in evaluation activities. This mediation provides a foundation for a consideration of the merit of ECB. This study sought to find confirmation that stakeholder engagement in evaluation could influence the outcomes of a program and thus subsequently provide evidence as to the merit and significance of evaluation engagement. Multiple forms of evaluation data were collected for two long-term public health evaluations and coded and collated again on an evidence-based rubric in close consultation with the community. The influence of stakeholder's willingness and capacity to engage in evaluation activity was also investigated. The analysis revealed that stakeholders' engagement in evaluations provided reasonably unique contributions to the overall program outcomes. This article provides the impetus for evaluation-based capacity building, in that it presents empirical evidence that willingness, capacity to engage in evaluation activities, and the use of evaluation information increases the probability of achieving desired outcomes and sustainability.