Object labels are valuable communicative tools because their meanings are shared among the members of a particular linguistic community. The current research was conducted to investigate whether 13-month-old infants appreciate that object labels should not be generalized across individuals who have been shown to speak different languages. Using a visual habituation paradigm, Experiment 1 tested whether infants would generalize a new object label that was taught to them by a speaker of a foreign language to a speaker from the infant's own linguistic group. The results suggest that infants do not expect 2 individuals who have been shown to speak different languages to use the same label to refer to the same object. The results of Experiment 2 reveal that infants do not generalize a new object label that was taught to them by a speaker of their native language to an individual who had been shown to speak a foreign language. These findings offer the first evidence that by the end of the 1st year of life, infants are sensitive to the fact that the conventional nature of language is constrained by the language that a person has been shown to speak.