Children with autism experience difficulties in understanding social affective cues, and it has been suggested that such deficits will generalize to music. In order to investigate this proposal, typically developing individuals and children with autism and Down syndrome were compared on tasks measuring perception of affective and movement states in music. The results showed that discrimination performance on both experimental conditions depended on chronological or verbal mental age rather than diagnosis. The findings suggest that emotion-processing deficits in the social domain do not generalize to music, and that musical understanding is closely related to the level of language development.