The development of some psychometric indices of performance rating quality based in judgement is traced. The conceptual and analytical confusion that characterizes contemporary research in this area is described. The operational definitions of several criteria of rating quality, including leniency and severity of the rating effect, halo effect, central tendency, and reliability are discussed. A set of positively-toned criteria for evaluating the quality of rating data as well as the suitability of some common research designs for generating statistical indices is presented. (Author/HLM)