ERIC Number: ED134612
Record Type: RIE
Publication Date: 1976-Dec
Reference Count: 0
Effects of Item Characteristics on Test Fairness. Research Report 76-5.
Pine, Steven M.; Weiss, David J.
This report examines how selection fairness is influenced by the item characteristics of a selection instrument in terms of its distribution of item difficulties, level of item discrimination, and degree of item bias. Computer simulation was used in the administration of conventional ability tests to a hypothetical target population consisting of a minority and a majority subgroup. Fairness was evaluated by three indices which reflect the degree of differential validity, errors in prediction (Cleary's model) and proportion of applicants exceeding a selection cutoff (Thorndike's model). Major findings were: (1) tests with a uniform distribution of difficulties had fairness properties generally superior to tests having a peaked distribution of item difficulties; (2) subgroup validity differences can be expected to occur when test items are biased against one of the subgroups; (3) when differential prediction is used, the Thorndike model reflects varying degrees of unfairness due to item bias and other test characteristics, while the Cleary and validity models do not; and (4) differential prediction provides fairer selection than the use of majority prediction only, regardless of the internal characteristics of the test, although substantial degrees of unfairness still exist under certain test item configurations. It was concluded that the internal characteristics of a selection instrument will affect the fairness of test scores in specific applications and that further research is needed to delineate which testing strategies and/or item characteristics are optimal in reducing unfairness. (Author)
Publication Type: Reports - Research
Education Level: N/A
Sponsor: Office of Naval Research, Arlington, VA. Personnel and Training Research Programs Office.
Authoring Institution: Minnesota Univ., Minneapolis. Dept. of Psychology.