The objective of this study was to examine the nature of preservice teachers' evaluations of elementary mathematics problems using the Mathematical Tasks Framework (MTF), a model designed to discriminate among tasks according to their cognitive complexity. We also tested the relationship between mathematics content knowledge and problem length on the preservice teachers' evaluations. Twenty-six undergraduate students enrolled in an elementary mathematics methods course at a large urban university were introduced to the MTF and cognitive complexity during a class lecture and were subsequently required to sort 32 mathematics problems according to the framework. Results demonstrated that overall, the preservice teachers had more difficulty accurately classifying problems considered to represent high levels of cognitive complexity compared to problems that were less complex. Those with strong mathematics content knowledge, as measured by a standardized test, were able to sort the problems more accurately than those with weaker content knowledge. Two open-ended items assessing content knowledge were not related to sorting performance. Finally, the preservice teachers were influenced by the surface characteristic of task length; the data indicated that the teachers tended to label short problems as less cognitively demanding and long problems as more so. Implications for preservice professional development include an increased emphasis on mathematics content knowledge as well as expert modeling of the identification of deep conceptual principles at the heart of the mathematics curriculum.