Theories of grounded cognition propose that modal simulations underlie cognitive representation of concepts [Barsalou, L. W. (1999). "Perceptual symbol systems." "Behavioral and Brain Sciences, 22"(4), 577-660; Barsalou, L. W. (2008). "Grounded cognition." "Annual Review of Psychology, 59", 617-645]. Based on recent evidence of modality-specific resources in perception, we hypothesized that verifying properties of concepts encoded in different modalities are hindered more by perceptual short-term memory load to the same versus different sensory modality as that used to process the property. We manipulated load to visual and auditory modalities by having participants store one or three items in short-term memory during property verification. In the high (but not low) load condition, property verification took longer when the property (e.g., yellow) involved the same modality as that used by the memory load (e.g., pictures). Interestingly, similar interference effects were obtained on the conceptual verification and on the memory task. These findings provide direct support for the view that conceptual processing relies on simulation in modality-specific systems. (Contains 2 tables and 4 figures.)