The ability to associate different types of number representations referring to the same quantity (symbolic Arabic numerals, signed/spoken number words, and nonsymbolic quantities), is an important predictor of overall mathematical success. This foundational skill--mapping--has not been examined in deaf and hard-of-hearing (DHH) children. To address this gap, we studied 188 4 1/2 to 9-year-old DHH and hearing children and systematically examined the relationship between their language experiences and mapping skills. We asked whether the timing of children's language exposure (early vs. later), the modality of their language (signed vs. spoken), and their rote counting abilities related to mapping performance. We found that language modality did not significantly relate to mapping performance, but timing of language exposure and counting skills did. These findings suggest that early access to language, whether spoken or signed, supports the development of age-typical mapping skills and that knowledge of number words is critical for this development.