The 'Sawtooth Effect' is where cohort performance on high-stakes assessments drops after assessment reform, and then improves over time as test familiarity increases. Despite its importance, limited research has been conducted to date. In this study, we took a novel approach of using changes in grade boundaries (cut scores) over time as a proxy measure for performance change across a large number of secondary school assessments. Findings were generally consistent with the Sawtooth Effect, suggesting that it took students/teachers around 3 years on average to become familiar with the new tests. Estimates of the magnitude of these changes were also produced. Findings support the need for caution when interpreting changes in performance for new assessments, as any gains that occur may be due to test familiarity, rather than meaningful gains to subject mastery.