There is increasing realization that mathematics-related self-efficacy expectations are a strong predictor of an array of significant mathematics outcomes. It is also evident that the curricular practice in schools largely neglects development of a student understanding in the unique language of mathematics. Consequently, this study probes how much influence difficulties with different forms of Terms, Symbols, Morphology, Syntax, Semantics and Pragmatics of Mathematics language exert on Self-efficacy belief in learning mathematics of upper primary Malayalam medium students. Mathematical language test administered on 200 standard VII students in Kozhikode and Malappuram districts of Kerala provided 21 categories of language related difficulties. Learner report on self-efficacy in learning Mathematics was summarised as "can learn" or "cannot learn" mathematics. Chi square analyses revealed that difficulty with every components of Mathematics language other than semantics increases the risk of low self-efficacy in learning Mathematics. In general, the risk of low self-efficacy belief in mathematics is observed in nearly 50% to 25% more students with difficulty in Terms, Symbols, Morphology, Syntax, and Pragmatics of Mathematics language than among those without difficulty in these elements of mathematics. This study reiterated the significance of language of mathematics and its various components in achievement and performance of mathematics principles and skills as well as for affective factors, especially self-efficacy beliefs, which in turn impact student motivation to learn mathematics further. Along with suggestions to strengthen language of mathematics among the students, the study indicate to the importance of further exploring and analysing the difficulties arising while teaching mathematics in diverse language settings including Malayalam.