In his influential 1960 paper "The Unreasonable Effectiveness of Mathematics in the Natural Sciences", Eugene P. Wigner raises the question of why something that was developed without concern for empirical facts--mathematics--should turn out to be so powerful in explaining facts about the natural world. Recent philosophy of science has developed "Wigner's puzzle" in two different directions: First, in relation to the supposed indispensability of mathematical facts to particular scientific explanations and, secondly, in connection with the idea that aesthetic criteria track theoretical desiderata such as empirical success. An important aspect of Wigner's article has, however, been overlooked in these debates: his worries about the underdetermination of physical theories by mathematical frameworks. The present paper argues that, by restoring this aspect of Wigner's argument to its proper place, Wigner's puzzle may become an instructive case study for the teaching of core issues in the philosophy of science and its history.