ERIC Number: ED083314
Record Type: Non-Journal
Publication Date: 1973-Apr
Reference Count: N/A
Cross Validation and Generalization of a Content Analysis of the Narrative Sections of Navy Performance Evaluation for Senior Enlisted Personnel.
Ramsey-Klee, Diane M.; Richman, Vivian
In an earlier pilot study of the narrative sections of Navy performance evaluations for senior enlisted personnel in pay grade E-7, it was determined by content analytic techniques that it is possible to differentiate between the performance of typical and superlative chief petty officers based on the narrative content of Evaluation Reports. A second study attempted to cross validate the pilot study results on new Evaluation Reports for senior enlisted men in the same two occupational ratings (AT's and BT's) represented in the pilot study sample and to extend the content analysis to two different occupational ratings (CS's and RM's) in order to test the generalizability of the content analytic techniques developed earlier. As a further refinement, the cross validation and generalization samples were analyzed without any knowledge of the ratee's relative position in the upper half of the marking scale on Performance of Duty (the criterion variable). A reliability study was also conducted whose objectives were twofold: (1) to determine the level of agreement among four individuals all of whom independently would perform a content analysis of the same corpus of 48 Evaluation Reports, and (2) to investigate if nonresearchers could be trained successfully to apply the complex content analysis methodology developed in the pilot study. This report presents detailed evidence that the pilot study findings essentially were extendible to the blind cross validation sample consisting of the same two ratings and also were generalizable to two different occupational ratings. Despite the initial expectation that it would be extremely difficult to train nonresearch-oriented individuals to consistently index the narrative sections of Evaluation Reports, the surprising result in the reliability study was that in only six training sessions a quite respectable level of agreement was achieved. (Author)
Publication Type: N/A
Education Level: N/A
Sponsor: Office of Naval Research, Arlington, VA. Personnel and Training Research Programs Office.
Authoring Institution: R-K Research and System Design, Malibu, CA.