Processing semantically related stimuli creates interference across various domains of cognition, including language and memory. In this study, we identify the locus and mechanism of interference when retrieving meanings associated with words and pictures. Subjects matched a probe stimulus (e.g., cat) to its associated target picture (e.g., yarn) from an array of unrelated pictures. Across trials, probes were either semantically related or unrelated. To test the locus of interference, we presented probes as either words or pictures. If semantic interference occurs at the stage common to both tasks, that is, access to semantic representations, then interference should occur in both probe presentation modalities. Results showed clear semantic interference effects independent of presentation modality and lexical frequency, confirming a semantic locus of interference in comprehension. To test the mechanism of interference, we repeated trials across 4 presentation cycles and manipulated the number of unrelated intervening trials (zero vs. two). We found that semantic interference was additive across cycles and survived 2 intervening trials, demonstrating interference to be long-lasting as opposed to short-lived. However, interference was smaller with zero versus 2 intervening trials, which we interpret to suggest that short-lived facilitation counteracted the long-lived interference. We propose that retrieving meanings associated with words/pictures from the same semantic category yields both interference due to long-lasting changes in connection strength between semantic representations (i.e., incremental learning) and facilitation caused by short-lived residual activation.