When learning basic color vocabulary, young children show a selective delay in the acquisition of brown and gray relative to other basic color terms. In this study, we first establish the robustness of this finding and then investigate the extent to which perception, language, and color preference may influence color conceptualization. Experimental tasks were designed to measure different aspects of perceptual color processing (discrimination and saliency), color preference and objective counts of color term frequency in preschool-directed language (books and mothers' speech) were used to compare the acquisition of three groups of colors: primary colors, secondary colors (orange, pink, and purple) that appear at the same time as the primary colors, and secondary colors (brown and gray) that appear late. Although our results suggest that perception does not directly shape young children's color term acquisition, we found that children prefer brown and gray significantly less than basic colors and that these color terms appear significantly less often in child-directed speech, suggesting that color preference, linguistic input, and developing color cognition may be linked.