Quality of performance data yielded by subjective judgment is of major concern to researchers in performance appraisal. However, some confusion exists in the analysis of quality on ratings obtained from different rating scale formats and from different raters. To clarify this confusion, a study was conducted to assess the quality of judgmental ratings provided by four rater sources. Six indices which seem to be meaningful for assessing the quality of judgmental ratings by different raters, i.e., leniency, range restriction, halo, dimensionality, inter-rater agreement, and predictive validity, were used. Middle level managers (N=344) were judged on their managerial role effectiveness by 272 superiors, 606 subordinates, and 470 peers, who rated ten specific roles, three overall performance variables, and completed the company's formal performance rating. Results indicated that self-ratings were slightly more lenient, but had the least halo. Superiors' ratings had the most restricted ranges and the highest level of halo. Peer ratings contained less halo than the ratings by superiors and subordinates, had less restricted ranges than superiors' ratings, and showed some level of predictive validity. Subordinate ratings had the least restricted ranges, but more halo than self-ratings, and had the lowest predictive validity. There was low inter-rater agreement on the effectiveness ratings across all the rater sources. The results indicate the need for further research to provide a better understanding of the nature of ratings provided by different segments of an organization. (JAC)