A sequential matching task was used to compare how the difficulty of shape discrimination influences the achievement of object constancy for depth rotations across haptic and visual object recognition. Stimuli were nameable, 3-dimensional plastic models of familiar objects (e.g., bed, chair) and morphs midway between these endpoint shapes (e.g., a bed-chair morph). The 2 objects presented on a trial were either both placed at the same orientation or were rotated by 90[degrees] relative to each other. Discrimination difficulty was increased by presenting more similarly shaped objects on mismatch trials (easy: bed, then lizard; medium: bed, then chair; hard: bed, then bed-chair morph). For within-modal visual matching, orientation changes were most disruptive when shape discrimination was hardest. This interaction for 3-dimensional objects replicated the interaction reported in earlier studies presenting 2-dimensional pictures of the same objects (Lawson & Bulthoff, 2008). In contrast, orientation changes and discrimination difficulty had additive effects on within-modal haptic and cross-modal visual-to-haptic matching, whereas cross-modal haptic-to-visual matching was orientation invariant. These results suggest that the cause of orientation sensitivity may differ for visual and haptic object recognition. (Contains 7 figures.)