Common tests of morphological awareness measure both morphology and syntax by requiring participants to fit words and pseudowords into sentences by adding or removing affixes. We report the results of a study testing a new word level task. College students viewed transparent words (without phonological or orthographic shifts) and used a keyboard to indicate whether the items contained 1, 2, 3, or 4 morphemes. Morpheme counting accuracy was strongly and significantly correlated with sentence level tests of morphological awareness, also grouping with them in a factor analysis, suggesting that the tasks measure a similar construct. Morpheme counting accuracy was also strongly and significantly correlated with the word identification and passage comprehension measures from the WJ-IV. Crossed random-effects modeling showed that all tasks were sensitive to word frequency and vocabulary. However, different MA tasks varied in their sensitivity to the sublexical properties of words. Responses in the sentence level tasks were sensitive to word and bigram frequency while responses in the word level task was sensitive to base frequency and the number of morphemes. Our findings suggest that conscious knowledge of root words and affixes can be directly measured at the word level without a syntactic component to the task and that responses do capture variation in the ability to decompose complex words into their component morphemes.