Implementing assessment reform can be challenging. Proposed new assessments must be seen by stakeholders to be fit for purpose, and sometimes the perceptions of key stakeholders, such as teachers and students, may differ from the assessment developers. This article considers the recent introduction of a new high-stakes assessment of spoken proficiency for students of foreign languages in New Zealand high schools. The new assessment aims to measure spoken proficiency through the recording of a range of unstaged peer-to-peer interactions as they take place throughout the year. It contrasts with an earlier assessment that drew on a summative teacher-led interview. The article presents findings from a survey of teachers (n = 152), completed two years into the assessment reform, in which teachers were asked to consider the relative usefulness of the two assessment formats. Findings suggest that teachers consider the new assessment to be, in most respects, significantly more useful than the earlier model, and that the new assessment is working relatively well. Some challenges emerge, however, in particular around the feasibility and fairness of collecting ongoing evidence of spontaneous peer-to-peer performance. Findings raise issues to be considered if the new assessment is to work more successfully.