How and when do children become aware that speakers have different accents? While adults readily make a variety of subtle social inferences based on speakers' accents, findings from children are more mixed: while one line of research suggests that even infants may be acutely sensitive to accent unfamiliarity, other studies suggest that 5-year-olds have difficulty identifying accents as different from their own. In an attempt to resolve this paradox, the current study assesses American children's sensitivity to American vs. Dutch accents in two situations. First, in an eye-tracked sentence processing paradigm where children have previously shown sensitivity to a salient social distinction (gender) from voice cues, 3-5-year-old children showed no sensitivity to accent differences. Second, in a social decision-making task where accent sensitivity has been found in 5-year-olds, an age gradient appeared, suggesting that familiar accent preferences emerge slowly between 3 and 7 years. Counter to claims that accent is an early, salient signal of social group, results are more consistent with a protracted learning hypothesis that children need extended exposure to native-language sound patterns in order to detect that an accent deviates from their own. A video abstract of this article can be viewed at: https://www.youtube.com/watch?v=BQAgy3IFYXA