Measurement problems which jeopardize the reliability and validity of competency-based writing assessments are analyzed. Methods to stabilize rating criteria and readers' application of them are necessary. Most writing assessment programs use guidelines from norm-referenced test methodology. Use of this method of criteria application based on ranking within-occasion endangers or precludes between-occasion uniformity. Judgment stability within a session and across sessions are other problems of measurement. The instability of ratings has been a major weakness of writing skills measures. Two indicators of rating variability are discussed. Rater drift is the rater's progressive deviation within a scoring session from previously shared criteria. Scale instability is the differential application of criteria by raters in different scoring sessions. During scale development and validation, assessments should collect separate ratings on component text features that comprise a total score. Rating methods should intersperse periodic checks. Frequency of checks and nature of feedback on scoring accuracy are important. Sound writing assessment requires scale stability. (Author/DWH)