ERIC Number: ED460134
Record Type: Non-Journal
Publication Date: 1998-Aug
Reference Count: N/A
The Effects of Finite Sampling Corrections on State Assessment Sample Requirements. NAEP Validity Studies (NVS).
Chromy, James R.
States participating in the National Assessment of Educational Progress State Assessment program (state NAEP) are required to sample at least 2,500 students from at least 100 schools per subject assessed. In this ideal situation, 25 students are assessed for a subject in each school selected for that subject. Two problems have arisen: some states have too few schools, and some states have small schools. This paper explores the application of finite population correction factors to the between-school component of variance and examines how this might affect sample size requirements in states that currently require exemption from the minimum sample requirements for the state NAEP. The study also explores how the infinite population assumption for hypothesis testing relating to comparisons between domain means might be preserved. Hypothetical school and student population structures and hypothetical variance components are developed. For each variance component, the effective sample size resulting from the minimum state NAEP sample size requirements is determined when the infinite population assumptions are a good approximation to reality. The study then examines how this effective sample size can be maintained with an alternate sample design that recognizes the population structure and applies the finite population correction at the school stage of sampling. The investigation lends support to alternate sample size specifications in both states with few schools and states with many small schools. Some additional changes in optional designs should also be considered. Appropriate cost modeling and the application of optimum allocation theory could yield less costly designs that would most likely favor more sampling from the larger schools. Better estimates of variance components associated with strata, schools, and students for a variety of assessment measures would also be of great assistance in investigations such as this one. (Contains 10 tables.) (SLD)
Descriptors: Elementary Secondary Education, National Competency Tests, Research Design, Research Methodology, Sample Size, Sampling, School Size, State Programs
NAEP Validity Studies (NVS), American Institutes for Research, 1791 Arastradero Road, P.O. Box 1113, Palo Alto, CA 94302. Tel: 650-493-3550; Fax: 650-858-0958.
Publication Type: Numerical/Quantitative Data; Reports - Evaluative
Education Level: N/A
Sponsor: National Center for Education Statistics (ED), Washington, DC.
Authoring Institution: American Institutes for Research in the Behavioral Sciences, Palo Alto, CA.
Identifiers - Assessments and Surveys: National Assessment of Educational Progress