Rubrics have become popular tools for assessing student writing both in classroom and standardized testing environments. Rubric construction and efficacy, however, is a topic that has been largely sidestepped in the literature and in teacher professional development. Composing an effective rubric--particularly for instructional or formative contexts--is a complex task that requires teachers to think metacognitively about their goals for a writing assignment, identify the assignment's purpose, weight the importance of various textual features, and align these elements to analytic scores. In this article, the authors conduct a textual analysis of initiating texts (i.e. rubrics and assignments) that teachers designed for use with a "Scholar" writing and peer response assignment. They identify three types of mismatch among the assignments, assessments, and purposes for writing; discuss implications of these mismatches for student writing and learning; and, finally, suggest ways in which online e-learning environments like "Scholar" might be designed to better support teachers' metacognition around assessment construction.