ERIC Number: ED343935
Record Type: RIE
Publication Date: 1992-Jan
Reference Count: N/A
Statistical Significance, Sample Splitting, and Generalizability of Results.
Ferrell, Charlotte M.
Statistical significance is often misinterpreted to mean replicability or generalizability of results, although a statistically significant difference does not equal a reliable difference. Sample splitting procedures may be a more accurate way of estimating research result generalizability. This type of cross-validation involves randomly dividing the original group and using one subgroup to develop the predictor equation. This equation is used to predict a criterion score for each person in the second subgroup, and predicted criterion scores for subjects in the second subgroup are correlated with actual criterion scores. To illustrate the sample splitting invariance procedure, a small data set (n=42 inservice teachers) from a study by L. G. Daniel and K. R. Okeafor (1987) is used. Results demonstrate that statistical significance testing may not always yield an accurate estimate of replicability when compared to analyses run using the sample splitting invariance procedure. There are five tables of analysis results and an appendix listing relevant computer codes for the computations. A 24-item list of references is included. (SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers: Criterion Referenced Scoring; Cross Validation; Invariance; Research Replication; Sample Splitting
Note: Paper presented at the Annual Meeting of the Southwest Educational Research Association (Houston, TX, January 31-February 2, 1992).