Social avoidance and anxiety are prevalent in fragile X syndrome (FXS) and are potentially mediated by the amygdala, a brain region critical for social behavior. Unfortunately, functional brain resonance imaging investigation of the amygdala in FXS is limited by the difficulties experienced by intellectually impaired and anxious participants. We investigated the relationship between social avoidance and emotion-potentiated startle, a probe of amygdala activation, in children and adolescents with FXS, developmental disability without FXS (DD), and typical development. Individuals with FXS or DD demonstrated significantly reduced potentiation to fearful faces than a typically developing control group (p < 0.05). However, among individuals with FXS, social avoidance correlated positively with fearful-face potentiation (p < 0.05). This suggests that general intellectual disability blunts amygdalar response, but differential amygdala responsiveness to social stimuli contributes to phenotypic variability among individuals with FXS.