This article explores how language and the multisemiotic nature of mathematics can present potential challenges for English language learners (ELLs). Based on two qualitative studies of the discourse of mathematics, we discuss some of the linguistic challenges of mathematics for ELLs in order to highlight the potential difficulties they may have when reading and doing mathematics. Examples, based on a systemic-functional linguistic analysis of texts, are used to show how different mathematical meanings are constructed. (Contains 2 tables.)