Recent studies have shown that listeners use verbs and other predicate terms to anticipate reference to semantic entities during real-time language comprehension. This process involves evaluating the denoted action against relevant properties of potential referents. The current study explored whether action-relevant properties are readily available to comprehension systems as a result of the embodied nature of linguistic and conceptual representations. In three experiments, eye movements were monitored as listeners followed instructions to move depicted objects on a computer screen. Critical instructions contained the verb "return" (e.g., "Now return the block to area 3"), which presupposes the previous displacement of its complement object--a property that is not reflected in perceptible or stable characteristics of objects. Experiment 1 demonstrated that predictions for previously displaced objects are generated upon hearing "return," ruling out the possibility that anticipatory effects draw directly on static affordances in perceptual symbols. Experiment 2 used a referential communication task to evaluate how communicative relevance constrains the use of perceptually derived information. Results showed that listeners anticipate previously displaced objects as candidates upon hearing "return" only when their displacement was known to the speaker. Experiment 3 showed that the outcome of the original act of displacement further modulates referential predictions. The results show that the use of perceptually grounded information in language interpretation is subject to communicative constraints, even when language denotes physical actions performed on concrete objects. (Contains 7 figures and 1 table.)