Confirmatory factor analysis was used to explore the internal validity of scores on the TerraNova Comprehensive Tests of Basic Skills/5 using samples from a southwestern school district and standardization samples reported by the publisher. One of the strengths claimed for battery-type achievement tests is provision of reliable and valid samples of student achievement in specific content areas. Results of the present study showed that specific content areas may not be clearly represented in the test structure as there was little difference in goodness of fit between two- or three-factor structures of subtest scores. In addition, a nested series of invariance tests showed that all parameters of the three-factor model cross-validated across samples. Together with large intercorrelations between the latent achievement factors and large subtest uniquenesses, these results raise questions about the differentiation among subtest scores as well as how scores should be used and interpreted. (Contains 3 tables.)