Language and communication deficits are among the core features of autism spectrum disorder (ASD). Reduced or reversed asymmetry of language has been found in a number of disorders, including ASD. Studies of healthy adults have found an association between language laterality and anatomical measures but this has not been systematically investigated in ASD. The goal of this study was to examine differences in gray matter volume of perisylvian language regions, connections between language regions, and language abilities in individuals with typical left lateralized language compared to those with atypical (bilateral or right) asymmetry of language functions. Fourteen adolescent boys with ASD and 20 typically developing adolescent boys participated, including equal numbers of left- and right-handed individuals in each group. Participants with typical left lateralized language activation had smaller frontal language region volume and higher fractional anisotropy of the arcuate fasciculus compared to the group with atypical language laterality, across both ASD and control participants. The group with typical language asymmetry included the most right-handed controls and fewest left-handers with ASD. Atypical language laterality was more prevalent in the ASD than control group. These findings support an association between laterality of language function and language region anatomy. They also suggest anatomical differences may be more associated with variation in language laterality than specifically with ASD. Language laterality therefore may provide a novel way of subdividing samples, resulting in more homogenous groups for research into genetic and neurocognitive foundations of developmental disorders. (Contains 5 figures and 4 tables.)