Valence and arousal are thought to be the primary dimensions of human emotion. However, the degree to which valence and arousal interact in determining brain responses to emotional pictures is still elusive. This functional MRI study aimed to delineate neural systems responding to valence and arousal, and their interaction. We measured neural activation in healthy females (N = 23) to affective pictures using a 2 (Valence) x 2 (Arousal) design. Results show that arousal was preferentially processed by middle temporal gyrus, hippocampus and ventrolateral prefrontal cortex. Regions responding to negative valence included visual and lateral prefrontal regions, positive valence activated middle temporal and orbitofrontal areas. Importantly, distinct arousal-by-valence interactions were present in anterior insula (negative pictures), and in occipital cortex, parahippocampal gyrus and posterior cingulate (positive pictures). These data demonstrate that the brain not only differentiates between valence and arousal but also responds to specific combinations of these two, thereby highlighting the sophisticated nature of emotion processing in (female) human subjects. (Contains 4 tables and 4 figures.)