We conducted the first study that estimates the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of a population-based autism spectrum disorders (ASD) surveillance system developed at the Centers for Disease Control and Prevention. The system employs a records-review methodology that yields ASD classification (case versus non-ASD case) and was compared with classification based on clinical examination. The study enrolled 177 children. Estimated specificity (0.96, [CI 0.95 = 0.94, 0.99]), PPV (0.79 [CI 0.95 = 0.66, 0.93]), and NPV (0.91 [CI 0.95 = 0.87, 0.96]) were high. Sensitivity was lower (0.60 [CI .95 = 0.45, 0.75]). Given diagnostic heterogeneity, and the broad array of ASD in the population, identifying children with ASD is challenging. Records-based surveillance yields a population-based estimate of ASD that is likely conservative.