ERIC Number: ED415265
Record Type: Non-Journal
Publication Date: 1997-Nov
Reference Count: N/A
Has Testing for Statistical Significance Outlived Its Usefulness?
McLean, James E.; Ernest, James M.
The research methodology literature in recent years has included a full frontal assault on statistical significance testing. An entire edition of "Experimental Education" explored this controversy. The purpose of this paper is to promote the position that while significance testing by itself may be flawed, it has not outlived its usefulness. However, it must be considered in combination with other criteria. Specifically, statistical significance is but one of three criteria that must be demonstrated to establish a position empirically. Statistical significance merely provides evidence that an event did not happen by chance. However, it provides no information about the meaningfulness (practical significance) of an event or if the event is replicable. Consequently, statistical significance testing must be accompanied by judgments of the event's practical significance and replicability. However, the likelihood of a chance occurrence of an event must not be ignored. It is acknowledged that the importance of significance testing is reduced as sample size increases. In large sample experiments, particularly those involving multiple variables, the role of significance testing is diminished because even small differences are often statistically significant. In small sample studies where assumptions such as random sampling are practical, significance testing can be quite useful. It is important to remember that statistical significance is but one criterion useful to inferential researchers. In addition to statistical significance, practical significance, and replicability, researchers must also consider Type II errors and sample size. Furthermore, researchers should not ignore other techniques such as confidence intervals. While all of these statistical concepts are related, they provide different types of information that assist researchers in making decisions. (Contains 30 references.) (Author/SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A