Visual perceptual learning (PL) and perceptual expertise (PE) traditionally lead to different training effects and recruit different brain areas, but reasons for these differences are largely unknown. Here, we tested how the learning history influences visual object representations. Two groups were trained with tasks typically used in PL or PE studies, with the same novel objects, training duration and parafoveal stimulus presentation. We observed qualitatively different changes in the cortical representations of these objects following PL and PE training, replicating typical training effects in each field. These effects were also modulated by testing tasks, suggesting that experience interacts with attentional set and that the choice of testing tasks critically determines the pattern of training effects one can observe after a short-term visual training. Experience appears sufficient to account for prior differences in the neural locus of learning between PL and PE. The nature of the experience with an object's category can determine its representation in the visual system. (Contains 9 figures, 8 tables and 3 footnotes.)