Early post-natal nutrition influences later development, but there are no studies comparing brain function in healthy infants as a function of dietary intake even though the major infant diets differ significantly in nutrient composition. We studied brain responses (event-related potentials; ERPs) to speech sounds for infants who were fed either breast milk (BF), milk-based formula (MF), or soy formula (SF) during the first 6 months of life. Two syllables presented in an oddball paradigm elicited a late positive wave (P350) from temporal and frontal brain regions involved in language processes. All groups showed significantly greater response amplitudes to the infrequent syllable across sites at 3 months and frontally at 6 months, but significant discrimination at temporal sites was only observed at 6 months in BF infants. Decreases in response amplitudes from 3 to 6 months were greater for the frequently presented syllable, most prominent in BF infants, and greater in females than males. The results indicate greater syllable discrimination in BF than formula-fed infants, but whether this can be attributed to dietary influences alone remains unclear. Feeding method and background factor differences between breastfed and formula-fed infants may also contribute to the observed differences. The general absence of differences between formula-fed groups is notable and suggests that milk-based formula and soy formula equally support brain development and function during the first post-natal 6 months. Finally, the results indicate gender differences in the development of neural and temporal processes involved in sensory discrimination, and suggest that at 6 months these processes are better developed in females.