ERIC Number: ED390921
Record Type: RIE
Publication Date: 1995-Apr
Reference Count: N/A
Cautionary Observations on Reliability and Equating of Forms in High Stakes Performance Assessment: The Problem of Granularity.
Cope, Ronald T.
This paper deals with the problems that arise in performance assessment from the granularity that results from having a small number of tasks or prompts and raters of responses to these tasks or prompts. Two problems are discussed in detail: (1) achieving a satisfactory degree of reliability; and (2) equating or adjusting for differences of difficulty among tasks or prompts. Empirical results from the Schul and Linacre (1995) study of writing assessment are used to amplify the discussion of reliability problems, and a set of hypothetical equating results are used to illustrate the problem of trying to make adjustments for difficulty when there are few scale points. The discussion concludes with suggestions for how performance assessment programs might attempt to deal with the problems that arise from high degrees of granularity. (An appendix contains a derivation of statistics used in the figures.) (Author)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers: Granularity (Statistics)
Note: Paper presented at the Annual Meeting of the National Council on Measurement in Education (San Francisco, CA, April 19-21, 1995).