Prior work showed that infants look longer at impossible figures than possible ones, although it is unclear whether they or older children understand "impossibility." We employed a series of matching and sorting tasks with pictures and objects to evaluate children's knowledge of this dimension. In Experiment 1, nearly all children successfully matched depicted and real objects, detected that possible and impossible objects differed, and identified structure-function relations of possible objects. In Experiment 2, children struggled to identify impossibility as a global characteristic for sorting. Few children sorted by impossibility, with a majority sorting by other shape similarities (e.g., round vs. straight). After receiving instructions, fewer than half sorted by possibility. Many children sorted by symmetry on their own, and the remainder performed perfectly after training. Children readily sorted by complexity. In Experiment 3, children sorted by possibility with few errors after elaborate instructions. Three-year-olds made significantly more errors than did older children, suggesting that selective attention to spatial geometry improves with experience. For children and adults, structural incoherence is challenging to recognize as a unifying property for grouping objects. Children may show implicit responses to impossible figures but often do not realize that they are in fact impossible.