ERIC Number: ED283290
Record Type: RIE
Publication Date: 1974-Feb
On Neglect of the Independent Variable in Program Evaluation. Project MITT Occasional Paper.
Charters, W. W., Jr.; Jones, John E.
This paper illustrates the need for full description and measurement of differences between "experimental" and "control" situations in school program evaluation studies. Two studies were completed at the University of Oregon involving the innovative program--differentiated staffing (DS). The first study, a doctoral investigation, sought to assess the relative effects of DS on elementary students' achievement using two schools and restricted to fifth and sixth graders to which the Comprehensive Test of Basic Skills was administered twice. Neglecting the independent variable resulted in mixed findings--achievement scores did not consistently favor either school. In the second study, using the same schools, a research group at the Center for the Advanced Study of Educational Administration (CASEA), measuring staff role behavior and relationships, discovered that staff behavior did not meet program goals. Neither study could attribute student outcomes to an independent variable described as "differentiated staffing" versus "traditional" organization. Four program levels assist evaluation: (1) institutional commitment; (2) structural context; (3) role performance (staff perspective); and (4) student learning activities. Although schools' commitment and structure differed, investigations focused on level 3, program consequences that were unattained and prevented the school from description as a "differentiated staffing school." Investigators therefore evaluated a non-event. These studies demonstrate the futility of comparing student outcome measures between classrooms or schools that differ only at functionally remote levels of program description. Changes at level 3 (and preferably level 4) should document achievement data. Analytical language and measurement techniques are needed to describe the independent variable at all levels. (CJH)
Descriptors: Concurrent Validity, Differentiated Staffs, Elementary Education, Instructional Innovation, Measurement Techniques, Organizational Theories, Predictive Validity, Predictor Variables, Program Evaluation, Program Implementation, Research and Development Centers, Research Methodology, Research Needs, School Organization
Publication Type: Reports - Research
Education Level: N/A
Sponsor: National Inst. of Education (DHEW), Washington, DC.
Authoring Institution: Oregon Univ., Eugene. Center for Educational Policy and Management.