ERIC Number: ED335399
Record Type: RIE
Publication Date: 1991-Apr
The Effects of Academic Discipline on Generalizability of Student Evaluations of Instruction.
Barnes, Laura L. B.; Barnes, Michael W.
Although student evaluation data provide a reasonable basis for making decisions about instructors when generalizing across courses and students, when the course is the object of measurement (OM), data are less generalizable. This finding may be due to the type of evaluation items used or to academic discipline differences in the type of courses selected for study. This study addressed this problem by using A. Biglan's (1973) model for classifying disciplines along the dimensions of paradigmatic/preparadigmatic (hard/soft) and pure/applied. A nested sampling procedure yielded two sample types: courses within teachers, in which individual instructors taught more than one course; and teachers within courses, in which individual courses were taught by more than one instructor. For each sample type, evaluation forms for 20 courses within each discipline classification were sought. Thirty items from a survey, which required students at a private doctoral-granting institution in the Southwest to rate their instructors on a 0-5 scale, were used. The evaluation items for this study measured six dimensions of instruction: organization, breadth of coverage, group interaction, enthusiasm, grading, and individual rapport. Generalizability and decision studies were conducted in which, for one sample, teacher was the OM, and for the second sample, course was the OM. Reliable decisions about instructors could reasonably be made from all six of the evaluation dimensions. However, reliability for course decisions varied greatly with the evaluation dimension, being highest for breadth of coverage and lowest for grading. The same general pattern was noted for the paradigmatic disciplines and the preparadigmatic-applied disciplines, but not for the preparadigmatic-pure disciplines. Two figures and five tables present study data. (Author/RLC)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Note: Paper presented at the Annual Meeting of the National Council on Measurement in Education (Chicago, IL, April 4-6, 1991).