As publication pressure has increased in the world of higher education, more journals, books, and other publication outlets have emerged. Thus it is critical to develop clear criteria for effectively evaluating the quality of publication outlets. Without such criteria funding agencies and promotion committees are forced to guess at how to evaluate a scholar's portfolio. In this article, we explore the perils of evaluating journals based on a single quantitative measure (e.g., the Impact Factor rating of the Institute for Science Information). We then discuss key considerations for evaluating scholarship, including three main criteria: rigor, impact, and prestige. Finally, we conclude with examples of how these criteria could be applied in evaluating scholarship.