Using computers to evaluate teachers based on student test scores is more difficult than it seems. Value-added modeling is a genuinely serious attempt to grapple with the difficulties. Value-added modeling carries the promise of measuring teacher quality automatically and objectively, and improving school systems at minimal cost. The essence of value-added models lies in the precise way they calculate expected scores for the students of each teacher. The mathematical ideas on which they are based are complicated and appear inaccessible to anyone with less training than upper-division university statisticians. A fairly small community of scholars, made up of both advocates and skeptics, has been responsible for developing the calculations (McCaffrey et al. 2003; National Research Council 2010). All of these experts agree that the results should be used with caution. However, because numbers in official printouts are so specific and appear so authoritative, it will prove problematic in practice to prevent them from dominating decisions about promotion and dismissal. Some concerns that have been raised previously about value-added modeling include the possible influence of missing information such as student mobility, large variations in results from year to year, the need for many years of data to obtain reliable results, and the absence of suitable pretests in some subject areas. In this article, the author describes a very particular worry he has had for some time, but for which he only recently was able to obtain any evidence. Deem this an invitation to grapple with the sorts of decisions that lurk behind the mathematics. (Contains 1 figure.)