People can recognize the meaning or gist of a scene from a single glance, and a few recent studies have begun to examine the sorts of information that contribute to scene gist recognition. The authors of the present study used visual masking coupled with image manipulations (randomizing phase while maintaining the Fourier amplitude spectrum; random image structure evolution [RISE]; J. Sadr & P. Sinha, 2004) to explore whether and when unlocalized Fourier amplitude information contributes to gist perception. In 4 experiments, the authors found that differences between scene categories in the Fourier amplitude spectrum are insufficient for gist recognition or gist masking. Whereas the global 1/f spatial frequency amplitude spectra of scenes plays a role in gist masking, local phase information is necessary for gist recognition and for the strongest gist masking. Moreover, the ability to recognize the gist of a target image was influenced by mask recognizability, suggesting that conceptual masking occurs even at the earliest stages of scene processing.