We perform a factor analysis on a "Force Concept Inventory" (FCI) data set collected from 2109 respondents. We address two questions: the appearance of conceptual coherence in student responses to the FCI and some consequences of this factor analysis on the teaching of Newtonian mechanics. We will highlight the apparent conflation of Newton's third law with Newton's first law in one of the FCI questions and suggest an approach to teaching that may avoid this issue. We also note the absence of a distinct factor interpretable as relating specifically to kinematics. Furthermore, we identify and discuss some of the technical difficulties which may be encountered when performing factor analysis on categorical data sets, such as is the case with FCI data sets. (Contains 2 figures and 4 tables.)