Delayed appearance of early language milestones can be one of the first signs of a developmental disorder. In this study, we investigated how well late acquisition of language milestones predicted an outcome of specific language impairment (SLI). The sample included 150 children (76 SLI), aged 4 to 7 years old. Milestone information was collected via retrospective parent report in a case-control design. Children who produced first words after 15 months met late word producer (LWP) criteria, and children who produced word combinations after 24 months met late word combiner (LWC) criteria. Accuracy metrics indicated that children identified as LWCs were at significantly increased risk of developing SLI (positive likelihood ratio [LR+] = 8.18, 95% confidence interval [CI] = [3.43, 19.53]), whereas children identified as LWPs were not (LR+ = 2.50, 95% CI = [0.95, 6.75]). Notably, a sizable proportion of children with SLI met neither LWP nor LWC criteria. Further analyses indicated that consideration of genetic and environmental risk factors may play an important role in early identification. Our results suggest that delayed milestone achievement is associated with SLI, but that the type of delay matters when making clinical decisions.