Stirling's approximation to ln "n"! is typically introduced to physical chemistry students as a step in the derivation of the statistical expression for the entropy. However, naive application of this approximation leads to incorrect conclusions. In this article, the problem is first illustrated using a familiar "toy model" example, the two-state system of "N" classical spins, where it is shown that two different physical situations lead to the same computed value of the entropy. Retention of additional terms in the approximation of the factorial is required to yield an accurate expression for the statistical weight of the most probable configuration in such model systems, but generates only a little extra accuracy in entropy calculations, and then only in the limit of very small numbers of particles. Additionally, inclusion of these terms makes the entropy nonextensive. We show here that, in the standard derivation of the entropy of the microcanonical ensemble, it is the freedom to allow the ensemble size to be infinite that makes the Boltzmann entropy expression S = k[subscript B] ln"W" exact, a fact that is not widely understood.