This study investigated two approaches for obtaining job ratings in order to determine which procedure provided the most stable ratings, given a constant number of raters. Specifically, it compared reliability estimates determined by averaging across individually obtained job ratings and reliability estimates based on consensus ratings from interacting panels. To investigate reliability of job ratings obtained from individuals and from panels of raters, 450 basic airmen rated 100 brief job descriptions individually and in panels of three and five raters. Analysis revealed that estimates of mean reliability were greater for individual raters, and mean rating time increased with panel size. Consequently, the individual approach appears to be more economical and efficient than the panel procedure, in terms of reliability, time required, and number of raters required. (Author/BH)