This study investigated the relationship between the dimensionality of behaviorally anchored scales during construction and during their use. It was hypothesized that as the agreement among raters in assigning behaviors to dimensions increased, there would be an increase in the stability of the structure of the dimensions and in their independence during use. Undergraduate students enrolled in two introductory history classes and five introductory psychology classes were administered a teacher evaluation instrument containing 60 behaviorally based items. These items represented five performance dimensions and four ranges of rater agreement. For the analyses, class means were subtracted for each item, items were factor analyzed, and inter-dimension correlations were computed. The results demonstrated partial support for the hypothesis. At the higher levels of agreement among raters, more of the original five dimensions were interpretable, and they were more independent. However, in none of the factor analyses were all of the original dimensions reproduced. It was concluded that the degree of agreement among judges during dimension construction did influence the ability of factor analytic solutions to represent the original dimensionality. However, high agreement does not guarantee the maintenance of dimension independence. (Author)