Many species, including humans, display group behavior. Thus, perceiving crowds may be important for social interaction and survival. Here, we provide the first evidence that humans use ensemble-coding mechanisms to perceive the behavior of a crowd of people with surprisingly high sensitivity. Observers estimated the headings of briefly presented crowds of point-light walkers that differed in the number and headings of their members (i.e., people in differently sized crowds had identical or increasingly variable directions of walking). We found that observers rapidly pooled information from multiple walkers to estimate the heading of a crowd. This ensemble code was precise; observer's perceived the behavior of a crowd better than the behavior of an individual. We also showed that this pooling provided tolerance against crowd variability and may cause a chaotic group to cohere into a unified Gestalt. Sensitive perception of a crowd's behavior required integration of human form and motion, suggesting that the ensemble code was generated in high-level visual areas. Overall, these mechanisms may reflect the prevalence of crowd behavior in nature and a social benefit for perceiving crowds as unified entities. (Contains 3 figures.)