Current thinking encourages teachers to incorporate a range of assessment practices that are responsive to student thinking and promote student learning. At the same time, teachers are situated in a landscape of accountability, where they are often seen as technicians who implement prescribed curriculum, policies, and procedures with success measured by externally created assessments. This paper considers how teachers navigate this difficult terrain as they incorporate new assessment practices in their classrooms. We report on a two-year project undertaken with 42 teachers of mathematics (Grades 4-12) in Ontario, Canada. Using an analytic framework adapted from Windschitl ("Review of Educational Research," 72(2), 131-175.2002), we explore the dilemmas these teachers identified as they met in communities of practice. Our findings highlight the need for coherence in the assessment messages communicated to teachers and provide evidence of the critical role of ongoing collaboration and dialogue to support the development of teachers' professional judgement and enable change in their assessment practice.