This paper discusses a teaching dilemma faced by mathematics teachers in the Malaysian Chinese primary schools in coping with the latest changes in language policy. In 2003, Malaysia launched a new language policy of teaching mathematics using English as the language of instruction in all schools. However, due to the complex sociocultural demands of the Malaysian Chinese community, mathematics is taught in both Mandarin and English in the Malaysian Chinese primary schools. Teachers faced the dilemma of code-switching between English and Mandarin particularly in low-performing classes. A substantial amount of class time was spent in translating, especially the mathematical terminologies. Findings of this study imply that further research studies are needed to examine the effectiveness of teaching mathematics bilingually versus monolingually in a Malaysian context.