Two experiments examined the role of compatibility of input and output (I-O) modality mappings in task switching. We define I-O modality compatibility in terms of similarity of stimulus modality and modality of response-related sensory consequences. Experiment 1 included switching between 2 compatible tasks (auditory-vocal vs. visual-manual) and between 2 incompatible tasks (auditory-manual vs. visual-vocal). The resulting switch costs were smaller in compatible tasks compared to incompatible tasks. Experiment 2 manipulated the response-stimulus interval (RSI) to examine the time course of the compatibility effect. The effect on switch costs was confirmed with short RSI, but the effect was diminished with long RSI. Together, the data suggest that task sets are modality specific. Reduced switch costs in compatible tasks may be due to special linkages between input and output modalities, whereas incompatible tasks increase cross-talk, presumably due to dissipating interference of correct and incorrect response modalities. (Contains 2 footnotes, 1 table, and 2 figures.)