Response shifts can be defined as a change in the way that a respondent interprets and responds to symptom questionnaire items, over and above true changes in their symptoms. Response shifts are liable to occur as a result of psychotherapy and can undermine evaluations of the effectiveness of psychotherapy interventions by making pre- and post- intervention scores non-comparable. We evaluated whether such response shifts were in evidence in the Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) and how this affected the comparisons of group-level scores before and after counselling intervention. We found that response shifts were minimal, making it likely that they can be easily addressed by testing group-level change within an appropriate latent variable model, rather than relying on observed scores.