The organization of category-selective regions in ventral visual cortex is well characterized in human adults. We investigated a crucial, previously unaddressed, question about how this organization emerges developmentally. We contrasted the developmental trajectories for face-, object-, and place-selective activation in the ventral visual cortex in children, adolescents, and adults. Although children demonstrated adult-like organization in object- and place-related cortex, as a group they failed to show consistent face-selective activation in classical face regions. The lack of a consistent neural signature for faces was attributable to (1) reduced face-selectivity and extent of activation within the regions that will become the FFA, OFA, and STS in adults, and (2) smaller volumes and considerable variability in the locus of face-selective activation in individual children. In contrast, adolescents showed an adult-like pattern of face-selective activation, although it was more right-lateralized. These findings reveal critical age-related differences in the emergence of category-specific functional organization in the visual cortex and support a model of brain development in which specialization emerges from interactions between experience-dependent learning and the maturing brain.