Past research has shown that variation in the target objects depicting a given spatial relation disrupts the formation of a category representation for that relation. In the current research, we asked whether changing the orientation of the referent frame depicting the spatial relation would also disrupt the formation of a category representation for that relation. Experiments 1 to 3 provided evidence that 6- and 7-month-olds formed a category representation for BETWEEN when a diamond shape was depicted in different locations between two vertical or horizontal reference bars during familiarization and in a novel location between the same orientation of bars during test. By contrast, in Experiment 4, same-age infants did not form a category representation for BETWEEN when the diamond shape was depicted between two vertical (or horizontal) bars during familiarization and between two horizontal (or vertical) bars during test. Moreover, in Experiment 5, 9- and 10-month-olds did form a category representation for BETWEEN when the orientation of the referent bars depicting the relation changed from familiarization to test. The findings suggest that the formation of category representations for spatial relations by infants is affected by changes to either target (figure) or referent (ground). (Contains 4 tables and 2 figures.)