Humans' tendency to follow others' gaze is considered to be rather resistant to top-down influences. However, recent evidence indicates that gaze following depends on prior eye contact with the observed agent. Does observing two people engaging in eye contact also modulate gaze following? Participants observed two faces looking at each other or away from each other before jointly shifting gaze to one of two locations. Targets appeared either at the cued location or at the non-cued location. In three experiments gaze cueing effects (faster responses to objects appearing at the cued location) were found only when the two faces had looked at each other before shifting gaze. In contrast, no effects of gaze following were observed when the two faces had looked away from each other. Thus, the attentional relation between observed people modulates whether their gaze is followed. (Contains 4 figures.)