NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ956145
Record Type: Journal
Publication Date: 2011-Dec
Pages: 12
Abstractor: As Provided
Reference Count: 28
ISSN: ISSN-1082-989X
Many Tests of Significance: New Methods for Controlling Type I Errors
Keselman, H. J.; Miller, Charles W.; Holland, Burt
Psychological Methods, v16 n4 p420-431 Dec 2011
There have been many discussions of how Type I errors should be controlled when many hypotheses are tested (e.g., all possible comparisons of means, correlations, proportions, the coefficients in hierarchical models, etc.). By and large, researchers have adopted familywise (FWER) control, though this practice certainly is not universal. Familywise control is intended to deal with the multiplicity issue of computing many tests of significance, yet such control is conservative--that is, less powerful--compared to per test/hypothesis control. The purpose of our article is to introduce the readership, particularly those readers familiar with issues related to controlling Type I errors when many tests of significance are computed, to newer methods that provide protection from the effects of multiple testing, yet are more powerful than familywise controlling methods. Specifically, we introduce a number of procedures that control the "k"-FWER. These methods--say, 2-FWER instead of 1-FWER (i.e., FWER)--are equivalent to specifying that the probability of 2 or more false rejections is controlled at 0.05, whereas FWER controls the probability of any (i.e., 1 or more) false rejections at 0.05. 2-FWER implicitly tolerates 1 false rejection and makes no explicit attempt to control the probability of its occurrence, unlike FWER, which tolerates no false rejections at all. More generally, "k"-FWER tolerates "k" - 1 false rejections, but controls the probability of "k" or more false rejections at [alpha] = 0.05. We demonstrate with two published data sets how more hypotheses can be rejected with "k"-FWER methods compared to FWER control. (Contains 4 tables and 8 footnotes.)
American Psychological Association. Journals Department, 750 First Street NE, Washington, DC 20002-4242. Tel: 800-374-2721; Tel: 202-336-5510; Fax: 202-336-5502; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A