Background: At present, percent syllables stuttered (%SS) is the gold standard outcome measure for behavioural stuttering treatment research. However, ordinal severity rating (SR) procedures have some inherent advantages over that method. Aims: To establish the relationship between Clinician %SS, Clinician SR and self-reported Speaker SR. To investigate whether Clinician SRs and Speaker SRs can be used interchangeably. Method & Procedures: Participants were three experienced speech-language pathologist (SLP) judges and 87 adults who stuttered. Adults who stuttered received a 10-min unscheduled telephone call at the conclusion of which they self-reported a SR using a nine-point scale. The SLPs measured the stuttering for these conversations with %SS and also with the SR scale. The mean scores for Clinician %SS and Clinician SR were compared with Speaker SR using appropriate indices of relative and absolute reliability. Relative reliability indices deal with the rank order of participants in a sample and whether they can be distinguished from each other. However, absolute reliability indices are related to the closeness of the measurement scores to each other and to a hypothetical true score. Outcomes & Results: Strong correlations were found between Clinician %SS and Clinician SR, and also between Clinician %SS and Speaker SR, although with higher values in the former case. Additionally, very high correlations showed acceptable relative reliability between Clinician SR and Speaker SR. However, absolute reliability in terms of standard error of measurement and limits of agreement was poor for Clinician SR and Speaker SR. Conclusions & Implications: The results suggest that Clinician SR and Speaker SR cannot be used interchangeably to measure temporal stuttering severity changes for an individual client. However, researchers might use these two measures interchangeably in research contexts, such as clinical trials, where changes of the entire group are of interest to determine and compare treatment effect size across trials.