ERIC Number: ED335374
Record Type: RIE
Publication Date: 1991-Apr
Reference Count: N/A
Component vs. Composite Grading: Outcome Quality and Passing Rate as Functions of Number of Tests, Test Correlation, and Reliability.
Credentialing agencies that use multi-test examinations (MTEs) should be concerned with the quality of pass/fail outcomes as well as with the proportion of candidates passing. This study addressed three questions: (1) the effects on passing rate of MTE reliability, number of tests comprising a MTE, and correlations between pairs of component tests; (2) the effects on outcome quality of test reliability and correlations between pairs of component tests; and (3) how component (multiple-cutoff) and composite (compensatory) grading differ in terms of outcome quality when evaluated against the same standard. Using three independent variables (the number of tests comprising a MTE, the tests' reliabilities, and their true-score correlations) that define 18 conditions, large-sample data sets (1,000 scores) were computer-generated and analyzed under classical true-error conditions. Results show that: chances of candidate success on component-graded MTEs varied dramatically with the technical characteristics of the MTE; for component-graded and composite-graded MTEs, the quality of pass/fail outcomes varied significantly according to test characteristics; and the two grading strategies produced different distributions of candidates across outcome-quality categories. Implications for licensing MTE construction are discussed. Four tables present study data. An attachment illustrates the formula for controlling correlations among the distributions. (SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers: Composite Scores; Multiple Cutoff Scores
Note: Paper presented at the Annual Meeting of the American Educational Research Association and the National Council on Measurement in Education (Chicago, IL, April 3-7, 1991).