ERIC Number: ED334245
Record Type: RIE
Publication Date: 1991-Apr
Reference Count: N/A
Reliability of Performance Assessments: Let's Make Sure We Account for the Errors.
Trevisan, Michael S.
Some of the issues regarding the estimation of reliability for performance assessments are explored, and a methodology is suggested for determination of reliability. In performance assessment, the magnitude of the reliability coefficient may be less than that obtained for a standardized test, since part of the potential use of performance assessment is to obtain outcome data for targets difficult to assess with standardized tests. A further problem is the lack of training of many assessment specialists with regard to performance assessment. In addition, reliability may refer to more than internal consistency. There is a growing consensus that the intraclass correlation coefficient (ICC) provides the best means for establishing reliability for quantitative observation data. Three basic ICC models are discussed: (1) one-way analysis of variance; (2) two-way random effects; and (3) a two-way mixed model. The use of ICC methods is discussed, and computer softwares for ICC analyses are described. Four tables illustrate the discussion, and a 26-item list of references is included. (SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers: Intraclass Correlation
Note: Paper presented at the Annual Meeting of the National Council on Measurement in Education and the National Association of Test Directors (Chicago, IL, April 4-6, 1991).