Stress can enhance or impair memory performance. Both cortisol release and sympathetic nervous system responses have been implicated in these differential effects. Here we investigated how memory retrieval might be affected by stress-induced cortisol release, independently of sympathetic nervous system stress responses. Thirty-two healthy participants (16 women) learned emotionally arousing and neutral words. One hour later, half of the participants underwent a stressor (cold pressor test) and the other half, a control warm water exposure, both followed by a delayed free recall task. The stressed participants were split into those who did (responders, N = 8) and those who did not (nonresponders, N = 6) show a cortisol response. Both responders and nonresponders showed comparable sympathetic nervous system activity (skin conductance level) during the cold pressor. The cortisol responders recalled significantly fewer words compared to nonresponders, and compared to control participants; this effect was most pronounced for moderately arousing words (compared to highly arousing and neutral words). These results suggest that individual differences in cortisol reactivity affect memory retrieval performance, and help to explain the differential effects of stress on memory. (Contains 2 tables, 2 figures, and 1 footnote.)