Mentalizing refers to making inferences about other people's mental states, whereas visuospatial perspective taking refers to inferring other people's viewpoints. Both abilities seem vital for social functioning; yet, their exact relationship is unclear. We directly compared mentalizing and visuospatial perspective taking in nineteen adults with Asperger syndrome (AS) and fifteen control participants with the same stimulus material. Stimuli depicted virtual characters surrounded by two different objects. Virtual characters expressed a preference for one of the objects indicated by facial expression, gestures or head/body orientation. Compared to controls, participants with AS showed significantly increased reaction times and decreased accuracy for mentalizing (i.e., when inferring the virtual character's preference from the character's nonverbal bodily cues). By contrast, there were no significant group differences in perspective taking (i.e., by mental own-body transformations). These findings demonstrate, first, specific deficits in AS when mental states have to be inferred from nonverbal social cues. Second, visuospatial perspective taking may not necessarily be related to social impairments occurring in autism spectrum disorders.