Is processing of conceptual information as robust as processing of perceptual information early in development? Existing empirical evidence is insufficient to answer this question. To examine this issue, 3- to 5-year-old children were presented with a flexible categorization task, in which target items (e.g., an open red umbrella) shared category membership with one test item (e.g., a folded umbrella) and perceptual characteristics with another test item (e.g., a red mushroom). Participants were instructed to either categorize stimuli by the same dimension (i.e., perceptual similarity or category membership) in both phases of the task, or switch from categorizing by one dimension to categorizing by the other dimension. Results pointed to a strong asymmetry in switch costs: conceptual switch costs were higher than perceptual switch costs. These results suggest that processing of perceptual information remains more robust than processing of conceptual information at least until 5 years of age.