Annual National Assessment (ANA) performance in Mathematics across the primary grades in South Africa indicates a decrease in mean performance across Grades 1-6. In this paper, we explore the apparently high performance in Grade 1 through a comparative investigation of learner responses on two assessments: the Grade 1 ANA taken in February 2011 by Grade 2 learners and a diagnostic oral interview test drawn from the work of Wright et al. (2006), administered at the same time. Our findings point to a predominant pattern of high performance on the ANA and low performance on Wright et al.'s tests. In-depth analysis of the responses of two learners in this group indicates that this discrepancy is due to acceptance in the ANA of correct answers produced through highly rudimentary counting strategies. The diagnostic test, in contrast, awards lower marks when correct answers are produced in inefficient ways. We conclude with concerns that acceptance of low-level counting strategies in the ANA may well work against persuading Grade 1 and 2 teachers to work towards more sophisticated strategies. (Contains 4 tables.)