External noise methods and observer models have been widely used to characterize the intrinsic perceptual limitations of human observers and changes of the perceptual limitations associated with cognitive, developmental, and disease processes by highlighting the variance of internal representations. The authors conducted a comprehensive review of the 5 most prominent observer models through the development of a common formalism. They derived new predictions of the models for a common set of behavioral tests that were compared with the data in the literature and a new experiment. The comparison between the model predictions and the empirical data resulted in very strong constraints on the observer models. The perceptual template model provided the best account of all the empirical data in the visual domain. The choice of the observer model has significant implications for the interpretation of data from other external noise paradigms, as well as studies using external noise to assay changes of perceptual limitations associated with observer states. The empirical and theoretical development suggests possible parallel developments in other sensory modalities and studies of high-level cognitive processes.