Context: Among small rural hospitals, there is a growing recognition of the need to measure and report on the use of resources and the safety and quality of the services provided. Dashboards, clinical value compasses, and balanced scorecards are approaches to performance measurement that have been adopted by many health care organizations. However, there exists very little comparative performance data specific for critical access hospitals. Purpose: To identify how comparative performance data for critical access hospitals (CPD-CAH) might facilitate performance and quality improvement, to assess the potential benefits and drawbacks of such data, and to identify some of the critical issues in the development and implementation of CPD-CAH. Methods: Assessment of discussions by participants at a rural hospital performance improvement summit and authors' analyses. Findings: CPD-CAH potentially could improve quality of care and patient outcomes, provide comparative data and benchmarks, inform policy development, facilitate collaboration, and enhance community relations. However, CPD-CAH could also impose an unaffordable cost, produce poor information, require complex coordination, induce a negative public reaction, and result in perverse hospital behavior. Development and implementation of CPD-CAH would require including stakeholders' assessment of its desirability and feasibility, setting objectives, establishing guiding principles, developing a method, collecting and analyzing data, and disseminating results. Conclusions: CPD-CAH could significantly advance CAH performance and quality improvement. However, development and implementation would be a complicated exercise requiring academic expertise and practitioner consultation. The potential value of CPD-CAH should be carefully weighed against its potential cost.