Autism spectrum disorder (ASD) is a complex neurodevelopmental condition that occurs within the first 3 years of life, which is marked by social skills and communication deficits along with stereotyped repetitive behavior. Although great efforts have been made to clarify the underlying neuroanatomical abnormalities and brain-behavior relationships in adolescents and adults with ASD, literature is still limited in information about the neurobiology of ASD in the early age of life. Brain images of 50 toddlers with ASD and 28 age, gender, and developmental quotient matched toddlers with developmental delay (DD) (control group) between ages 2 and 3 years were captured using combined magnetic resonance-based structural imaging and diffusion tensor imaging (DTI). Structural magnetic resonance imaging was applied to assess overall gray matter (GM) and white matter (WM) volumes, and regional alterations were assessed by voxel-based morphometry. DTI was used to investigate the white matter tract integrity. Compared with DD, significant increases were observed in ASD, primarily in global GM and WM volumes and in right superior temporal gyrus regional GM and WM volumes. Higher fractional anisotropy value was also observed in the corpus callosum, posterior cingulate cortex, and limbic lobes of ASD. The converging findings of structural and white matter abnormalities in ASD suggest that alterations in neural-anatomy of different brain regions may be involved in behavioral and cognitive deficits associated with ASD, especially in an early age of 2-3 years old toddlers.