Reaction time is often shorter when the irrelevant graspable handle of an object corresponds with the location of a keypress response to the relevant attribute than when it does not. This object-based Simon effect has been attributed to an affordance for grasping the handle with the hand to the same side. Because a grasping affordance should differentially affect keypress responses only when they are made with different hands, we conducted three experiments that measured the object-based Simon effect for frying pan stimuli using between- and within-hand response sets. When the relevant stimulus dimension was color, neither the object-based Simon effect nor the location-based Simon effect varied across response sets. When upright-inverted orientation judgments were made for the frying pan and for nongraspable stimuli derived from it, there again was no significant difference in size of the between- and within-hand Simon effects for any of the stimuli. The results provide evidence that the Simon effect for graspable frying pan stimuli is because of relative location of the handle and not to a grasping affordance. (Contains 2 figures, 1 table and 1 footnote.)