ERIC Number: ED430047
Record Type: Non-Journal
Publication Date: 1999-Apr-21
Choosing a Multiple Comparison Procedure Based on Alpha.
Barnette, J. Jackson; McLean, James E.
Four of the most commonly used multiple comparison procedures were compared for pairwise comparisons and relative to control of per-experiment and experimentwise Type I errors when conducted as protected or unprotected tests. The methods are: (1) Dunn-Bonferroni; (2) Dunn-Sidak; (3) Holm's sequentially rejective; and (4) Tukey's honestly significant difference procedure (HSD). Monte Carlo methods were used to generate replications, and means and standard deviations of observed Type I error-rates and percentages of observed Type I errors within the 0.95 confidence intervals were determined for per-experiment and experimentwise conditions. Effects of numbers of groups and group sizes on these Type I error rates were examined. Of primary concern was the accuracy of these procedures compared with the nominal alpha. Results indicate that none of these tests should be conducted as protected tests, but only as unprotected tests, particularly when alpha is 0.05 or less. If the Type I error control philosophy is experimentwise, Tukey's HSD, as an unprotected test, is clearly the most accurate procedure across all three alpha levels. If the Type-I error control philosophy is per-experiment, the Dunn-Bonferroni, as an unprotected test, is clearly the most accurate procedure across all three alpha levels. (Contains 5 tables and 18 references.) (SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A