THE VALIDITY OF AN EDUCATIONAL ACHIEVEMENT TEST DEPENDS UPON THE CORRESPONDENCE BETWEEN SPECIFIED EDUCATIONAL OBJECTIVES AND THE EXTENT TO WHICH THESE OBJECTIVES ARE MEASURED BY THE EVALUATION INSTRUMENT. THIS STUDY IS DESIGNED TO EVALUATE THE EFFECT OF STATISTICAL ITEM SELECTION ON THE STRUCTURE OF THE FINAL EVALUATION INSTRUMENT AS COMPARED WITH THE STRUCTURE OF ITS ORIGINAL ITEM POOL. THE ITEM POOL CONSISTED OF 379 FOUR AND FIVE OPTION MULTIPLE-CHOICE ITEMS USED IN AN INTRODUCTORY NATURAL SCIENCE TEST. THESE ITEMS WERE CLASSIFIED USING THE CATEGORIES OF BLOOM'S "TAXONOMY OF EDUCATIONAL OBJECTIVES"--KNOWLEDGE, COMPREHENSION, APPLICATION, AND ANALYSIS. ONE THOUSAND MALE AND 1000 FEMALE STUDENTS WHO HAD TAKEN THE EXAMINATIONS WERE RANDOMLY SELECTED. THE UPPER AND LOWER 27 PERCENT IN EACH DISTRIBUTION WERE USED TO COMPUTE INDICES OF ITEM DIFFICULTY AND DISCRIMINATION, USING DIFFERENCE AND DAVIS INDICES. BOTH INDICES WERE THEN USED TO SELECT THE 100-ITEM FINAL TEST FORM. TWO MAJOR CONCLUSIONS RESULTED--(1) STATISTICAL SELECTION OF ITEMS FROM THE TOTAL ITEM POOL HAD A BIASING EFFECT ON THE SELECTED TESTS, AND (2) STATISTICAL SELECTION OF ITEMS FROM THE TOTAL ITEM POOL OPERATED DIFFERENTIALLY FOR MALE AND FEMALE GROUPS. THESE CONCLUSIONS IMPLY THAT STATISTICAL ITEM SELECTION ALONE IS NOT SUFFICIENT FOR TEST CONSTRUCTION. THIS PAPER WAS PRESENTED AT THE ANNUAL MEETING OF THE NATIONAL COUNCIL ON MEASUREMENT IN EDUCATION (CHICAGO, FEBRUARY, 1965) AND IS A REPRINT FROM "JOURNAL OF EDUCATIONAL MEASUREMENT," VOLUME 2, 1965. (HW)