ERIC Number: ED163036
Record Type: RIE
Publication Date: 1978-Aug
Reference Count: 0
A Comparison of the Fairness of Adaptive and Conventional Testing Strategies. Research Report 78-1.
Pine, Steven M.; Weiss, David J.
This report examines how selection fairness is influenced by the characteristics of a selection instrument in terms of its distribution of item difficulties, level of item discrimination, degree of item bias, and testing strategy. Computer simulation was used in the administration of either a conventional or Bayesian adaptive ability test to a hypothetical target population consisting of a minority and majority subgroup. Fairness was evaluated by three indices which reflect the degree of differential validity, errors in prediction (Cleary's model), and proportion of applicants exceeding a selection cutoff (Thorndike's model). Major findings are: (1) when used in conjunction with either the Bayesian or conventional test, differential prediction increased fairness and facilitated the interpretation of the fairness indices; (2) the Bayesian adaptive tests were consistently fairer than the conventional tests for all item pools above the alpha=.7 discrimination level for tests of more than 30 items; (3) the differential prediction version of the Bayesian adaptive test produced almost perfectly fair performance on all fairness indices at high discrimination levels; and (4) the placement of subgroup prior distribution in the Bayesian adaptive testing procedure can affect test fairness. (Author/CTM)
Descriptors: Adaptive Testing, Bayesian Statistics, Comparative Testing, Computer Assisted Testing, Item Analysis, Minority Groups, Occupational Tests, Personnel Selection, Prediction, Simulation, Test Bias, Test Construction, Test Items, Test Validity
Psychometric Methods Program, Department of Psychology, University of Minnesota, 75 East River Road, Minneapolis, Minnesota 55455 (no charge, while supplies last); National Technical Information Service, Springfield, Virginia 22161
Publication Type: Reports - Research
Education Level: N/A
Sponsor: Office of Naval Research, Arlington, VA. Personnel and Training Research Programs Office.
Authoring Institution: Minnesota Univ., Minneapolis. Dept. of Psychology.