ERIC Number: ED187761
Record Type: RIE
Publication Date: 1980-Apr
Reference Count: 0
A Consumers' Guide to Criterion-Referenced Test "Reliability".
Berk, Ronald A.
Seventeen statistics for measuring the reliability of criterion-referenced tests were critically reviewed. The review was organized into two sections: (1) a discussion of preliminary considerations to provide a foundation for choosing the appropriate category of "reliability" (threshold loss function, squared-error loss-function, or domain score estimation) for a particular application and (2) an evaluation of the indices for selection of a specific index within the chosen category. The former was related to information on the test forms assumption, the cutting score, probabilities of decision errors, score interpretation, type of decision, and the seriousness of losses associated with the decision errors. The latter suggested that the threshold loss function approaches of Hambleton and Novick (1973) and Huynh (1976) were the most useful, the squared-error loss function indices of Livingston (1972a) and Brennan (1980) provide meaningful information especially for placement tests, and the domain score estimation statistics have limited utility for individual decision making. Detailed tables are included. (Author/GSK)
Publication Type: Speeches/Meeting Papers; Information Analyses; Reports - Research
Education Level: N/A
Authoring Institution: N/A
Identifiers: Domain Score Estimation; Loss Function
Note: Paper presented at the Annual Meeting of the National Council on Measurement in Education (Boston, MA, April 8-10, 1980). Tables may be marginally legible due to small print.