ERIC Number: ED222555
Record Type: RIE
Publication Date: 1982-Apr
Reference Count: 0
Design Effects and the Analysis of Survey Data.
Folsom, Ralph E.; Williams, Rick L.
The National Assessment of Educational Progress (NAEP), like most large national surveys, employs a complex stratified multistage unequal probability sample. The design provides a rigorous justification for extending survey results to the entire U.S. target population. Developments in the analysis of data from complex surveys which provide a straightforward method for taking account of the sample design through proper estimation of subpopulation estimates and their covariance matrix are reviewed. Relationships among subpopulations are then evaluated via large sample Wald statistics assumed to be asymptotically distributed as central chi-squared random variables. While these methods provide a mechanism for analyzing NAEP data, the computer software required to properly estimate sample design-based covariance matrices is not generally available to NAEP data users. Design effect methods for adjusting test statistics obtained from standard statistical methods which implicitly assume simple random sampling from an infinite population are presented with several new decompositions obtained which display the effects of multistage clustering, stratification, and unequal weighting on the covariance matrix. A comparison of asymptotically valid sample design-based chi-squared tests versus analogous simple random sampling tests and design effect adjusted tests is given. Design effect adjustments on NAEP mathematics data are shown. Primary type of information provided by the report: Procedures (Sampling); Results (Secondary Analysis). (Author/CM)
Publication Type: Reports - Research
Education Level: N/A
Sponsor: National Inst. of Education (ED), Washington, DC.
Authoring Institution: Education Commission of the States, Denver, CO. National Assessment of Educational Progress.; Research Triangle Inst., Research Triangle Park, NC.
Identifiers: Chi Square; National Assessment of Educational Progress; NIE ECS NAEP Item Development Project; Second Mathematics Assessment (1978)
Note: Appendix B is marginally legible due to small print; For related documents, see TM 820 707-712 and TM 820 716.