ERIC Number: ED335359
Record Type: RIE
Publication Date: 1990-Nov
Reference Count: N/A
Differences between Novice and Expert Knowledge Structure, Pre- and Post-Training, in a Statistics and Test Theory Domain.
Steinberg, Wendy J.
The purpose of this study was to examine the nature and degree of differences in expert versus novice knowledge structures, both before and after training, when judging the similarity of multiple-choice test items within a statistics and test theory (STT) domain. Subjects were employees of the Testing Division of the New York State Department of Civil Service (NYDCS), of whom 8 were trainees. Approximately 2 weeks prior to taking the agency training course in elementary STT, a 45-item multiple-choice domain-specific pretest was administered to the 8 trainees, as well as to 10 experienced NYDCS staff members. Scores on the test were then used as the basis for identifying novices and experts. The final subject pool (6 novices and 6 experts) was then administered a subset of 18 items from the pretest covering descriptive statistics; inferential statistics and experimental design; and validity, reliability, and test theory. Subjects judged on a 7-point scale how similar they perceived 153 different item pairs to be. At the end of the course, novices again sorted the 153 pairs of items using the 7-point scale. Subjects' multidimensional scaling judgments were entered into matrices and analyzed via INDSCAL, and an analysis of angular variation was computed. Prior to training, no significant difference between the two subject groups was found, although at that point novices shared only one of the experts' three dimensions, while, after training the groups shared two dimensions. The finding of no significant difference between the groups prior to training conflicts with logic, and the paper concludes with a discussion of possible reasons for the finding of non-significance. Nine transparencies illustrating the study design and results are appended. (RLC)
Descriptors: Adults, Cognitive Structures, Comparative Testing, Government Employees, Job Training, Knowledge Level, Mathematics Instruction, Multidimensional Scaling, Multiple Choice Tests, Occupational Tests, Pretests Posttests, Professional Education, Rating Scales, State Government, Statistics, Test Items, Test Theory
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers: Experts; New York
Note: Paper presented at the Annual Meeting of the Northeastern Educational Research Association (Ellenville, NY, November 1, 1990).