We investigated the effect of familiarity on people's perception of facial likeness by asking participants to choose which of two mirror-symmetric chimeric images (made from the left or right half of a photograph of a face) looked more like an original image. In separate trials the participants made this judgment for their own face and for the face of a close friend; half of them matched to a true image of the original and half matched to a mirror image of the original. In the case of matching to a friend's face presented in the familiar orientation, over 80% of participants chose the left-left composite to be a better likeness to the original, whereas only 62% showed the same left-side bias when matching to a mirror image. The difference is significant, and the result contrasts markedly with a second experiment where participants who were unfamiliar with the faces showed comparable left-side biases when matching to true or mirror reversed images. The result suggests that perceptual asymmetries are retained in our long-term memory for highly familiar faces. While matching to images of self also showed an effect of familiarity, the data in this condition show less evidence of perceptual asymmetry and are discussed in relation to recent research on the representation of one's own face.