Research has demonstrated a link between perspective taking and working memory. Here we used eye tracking to examine the time course with which working memory load (WML) influences perspective-taking ability in a referential communication task and how motivation to take another's perspective modulates these effects. In Experiment 1, where there was no reward or time pressure, listeners only showed evidence of incorporating perspective knowledge during integration of the target object but did not anticipate reference to this common ground object during the pretarget-noun period. WML did not affect this perspective use. In Experiment 2, where a reward for speed and accuracy was applied, listeners used perspective cues to disambiguate the target object from the competitor object from the earliest moments of processing (i.e., during the pretarget-noun period), but only under low load. Under high load, responses were comparable with the control condition, where both objects were in common ground. Furthermore, attempts to initiate perspective-relevant responses under high load led to impaired recall on the concurrent WML task, indicating that perspective-relevant responses were drawing on limited cognitive resources. These results show that when there is ambiguity, perspective cues guide rapid referential interpretation when there is sufficient motivation and sufficient cognitive resources.