Purpose: The primary purpose of this study was to compare percent correct word and sentence intelligibility scores for individuals with multiple sclerosis (MS) and Parkinson's disease (PD) with scaled estimates of speech severity obtained for a reading passage. Method: Speech samples for 78 talkers were judged, including 30 speakers with MS, 16 speakers with PD, and 32 healthy control speakers. Fifty-two naive listeners performed forced-choice word identification, sentence transcription, or visual analog scaling of speech severity for the Grandfather Passage (Duffy, 2005). Three expert listeners also scaled speech severity for the Grandfather Passage. Results: Percent correct word and sentence intelligibility scores did not cleanly differentiate speakers with MS, PD, or control speakers. In contrast, both naive and expert listener groups judged reading passages produced by speakers with MS and PD to be more severely impaired than reading passages produced by control talkers. Conclusion: Scaled estimates of speech severity appear to be sensitive to aspects of speech impairment in MS and PD not captured by word or sentence intelligibility scores. One implication is that scaled estimates of speech severity might prove useful for documenting speech changes related to disease progression or even treatment for individuals with MS and PD with minimal reduction in intelligibility.