Recent studies report that the use of rubrics may not improve the reliability of assessment if raters are not well trained on how to design and employ them effectively. The intent of this two-phase study was to test if training pre-service and new in-service teachers in the construction, use, and evaluation of rubrics would improve the reliability of their evaluations of writing assignments. Results of Phase 1 showed that raters were significantly influenced by mechanical characteristics of students' writing, and that using rubrics may not improve the reliability of assessment attributes if raters are not well trained on how to design and employ them effectively. Researchers asserted that without high quality, intense training, participants (N=355) who use rubrics to assess students' writing are usually just as subjective in their evaluative marks as those who do not use rubrics. In Phase 2, participants (N=55) received intense rubrics training prior to being presented with the two writing samples. Results of Phase 2 showed that rubrics training led to more reliable assessment of both writing samples. (Contains 7 tables.)