Theoretical perspectives and empirical research suggest that acceptance of evolution may be contingent upon what is evolving (e.g., plants vs. humans) and the scale of change (microevolution vs. macroevolution). The Inventory of Student Evolution Acceptance (I-SEA) is the only instrument designed to measure acceptance at different evolutionary scales and in different evolutionary contexts. Nevertheless, current validity testing for this instrument remains limited and grounded in Classical Test Theory. In this study, we examine patterns of evolution acceptance using the I-SEA instrument in a large sample of undergraduates (n > 2,000 participants) from six semesters of an evolution-focused biology class. We examine three research questions: (RQ1) Does Rasch analysis support the psychometric properties of the I-SEA discussed by Nadelson and Southerland? (RQ2) How variable are I-SEA measures across semesters, and are they sensitive to evolution instruction? (RQ3) Are I-SEA measures comparable to evolution acceptance measures produced by the Generalized Acceptance of EvolutioN Evaluation? Our empirical results support theoretical claims that evolution acceptance does not appear to be a unidimensional construct among novice learners. We identify limitations of the I-SEA and recommend multiple modifications to improve instrument quality.