Autism spectrum disorder (ASD) and autism-like traits are associated with deficits in face memory ability, although it is not yet clear whether this deficit reflects a specific aspect of the ASD/autism-like phenotype. We addressed this issue using a neurotypical sample of adolescent twins (N[subscript complete pairs] = 782) drawn from the Twins Early Development Study who were assessed on face and object memory performance alongside two core aspects of autism-like traits: (i) difficulties with social behavior/interactions, and (ii) attention to detail. We observed a negative association between face memory ability and difficulties with social behavior/interactions. This association reflected an overlapping genetic etiology: heritable influences acting on face memory ability are associated with the social difficulties aspects of autism-like traits.