The purpose of this study was to examine the nature and degree of differences in expert versus novice knowledge structures, both before and after training, when judging the similarity of multiple-choice test items within a statistics and test theory (STT) domain. Subjects were employees of the Testing Division of the New York State Department of Civil Service (NYDCS), of whom 8 were trainees. Approximately 2 weeks prior to taking the agency training course in elementary STT, a 45-item multiple-choice domain-specific pretest was administered to the 8 trainees, as well as to 10 experienced NYDCS staff members. Scores on the test were then used as the basis for identifying novices and experts. The final subject pool (6 novices and 6 experts) was then administered a subset of 18 items from the pretest covering descriptive statistics; inferential statistics and experimental design; and validity, reliability, and test theory. Subjects judged on a 7-point scale how similar they perceived 153 different item pairs to be. At the end of the course, novices again sorted the 153 pairs of items using the 7-point scale. Subjects' multidimensional scaling judgments were entered into matrices and analyzed via INDSCAL, and an analysis of angular variation was computed. Prior to training, no significant difference between the two subject groups was found, although at that point novices shared only one of the experts' three dimensions, while, after training the groups shared two dimensions. The finding of no significant difference between the groups prior to training conflicts with logic, and the paper concludes with a discussion of possible reasons for the finding of non-significance. Nine transparencies illustrating the study design and results are appended. (RLC)