From early in life, infants synchronize with others on a physiological level, a process thought to underlie social connections and group cohesion. This synchronization is seen, for example, when their pupils dilate in response to observing another person with dilated pupils -- known as "pupillary contagion." There is mixed evidence on whether arousal synchrony is modulated by interpersonal similarity factors, such as race, and even in studies that find such an effect, confounding visual factors could play a role. In the current study, language was used to manipulate interpersonal similarity for 10-month-old infants who saw speakers' pupils dilate or constrict, while their own pupil size and gaze were assessed. Results from the first half of the study show that only own-language speakers elicited pupillary contagion and increased attention when their pupils dilated. While in the second half of the study, when infants' level of attention was also decreasing, this effect did not hold. Together, the results indicate that infants' sharing of arousal is modulated by shared language, though further research can help to clarify how these effects unfold over time.