ERIC Number: ED335408
Record Type: RIE
Publication Date: 1991-Apr
Reference Count: 0
The Effects of "High Stakes" Certification Demands on the Generalizability and Dependability of a Classroom-Based Teacher Assessment System.
Ellett, Chad D.; And Others
This paper presents the results of ongoing analyses of the reliability of the System for Teaching and Learning Assessment and Review (STAR) as a comprehensive measure of classroom teaching and learning for making teacher certification decisions. Focus was on the effects of high stakes assessment conditions for certification on the generalizability evidence for the STAR when compared to more normal research conditions. The STAR contains 140 teacher effectiveness and student learning indicators, which are classified into four performance dimensions that are operationalized by 23 teaching and learning components (TLCs). Data were collected during the spring of 1990 using 144 teachers in 30 elementary and secondary schools in southeastern Louisiana school districts. In all, 864 observations were completed (144 teachers x 6 observations). Also, data from STAR assessments (16,524 observations) for 2,754 teachers and 2 random samples of about 100 teachers nested within this group were used. A four-facet (teachers, assessor type, occasion of measurement, and assessment indicators) General Purpose Analysis of Variance System was used. A generalizability coefficient (GC) was computed for each STAR TLC. Descriptive statistics for various STAR TLCs, GCs for components comparing models with two or three observers, GCs collected under research versus certification conditions, and variance component estimates for the TLCs for a statewide certification data file are reported. Assessment demand characteristics under certification conditions can alter the statistical reliabilities of classroom-based teacher assessment systems. These conditions typically skew performance distributions, resulting in decreased statistical reliabilities. The STAR's reliability seems best supported by coefficients established under research conditions with no high stakes certification demands. Nine data tables and a 16-item list of references are included. (RLC)
Descriptors: Classroom Observation Techniques, Educational Indicators, Elementary School Teachers, Evaluation Criteria, Evaluators, Generalizability Theory, Learning Strategies, Reliability, Secondary School Teachers, State Programs, Teacher Certification, Teacher Evaluation, Teaching Methods, Testing Programs
Publication Type: Numerical/Quantitative Data; Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers: Louisiana; System for Teaching Learning Assessment Review LA; Teacher Performance Appraisal System
Note: Paper presented at the Annual Meeting of the American Educational Research Association (Chicago, IL, April 3-7, 1991).