Prior studies have indicated brain abnormalities underlying social processing in autism, but no fMRI study has specifically addressed the differential processing of direct and averted gaze, a critical social cue. Fifteen adolescents and adults with autism and 14 typically developing comparison participants viewed dynamic virtual-reality videos depicting a simple but realistic social scenario, in which an approaching male figure maintained either direct or averted gaze. Significant group by condition interactions reflecting differential responses to direct versus averted gaze in people with autism relative to typically developing individuals were identified in the right temporoparietal junction, right anterior insula, left lateral occipital cortex, and left dorsolateral prefrontal cortex. Our results provide initial evidence regarding brain mechanisms underlying the processing of gaze direction during simple social encounters, providing new insight into the social deficits in individuals with autism.