An Efficiency Comparison of Closed Multiple Test Methods for Population Means under Unequal Correlation Matrix

Abstract

The objective of this research is to compare five closed multiple test methods with stepwiseprocedure for testing the difference between two population means : Hotelling’s T2 method, Bonferroni-Holm method, Hommel’s method based on Simes’ test, Westfall-Young bootstrap method and ExactPermutational method, by considering their capacity of controlling type I error rate and their power of thetest under multivariate normal distributions with the same covariance matrix which equals to thecorrelation matrix for 3, 5 and 7 dependent variables; 10, 30, 50 and 70 equal sample sizes; unequalcorrelation design matrix with correlation coefficient equals to 0.0, 0.3, 0.5, 0.7 and 0.9 for the case ofunequal correlation design matrix at 0.05 significant level (α). Monte Carlo simulations was performed andrepeated 1,000 times for each scenario. The results showed that in most situations, Hotelling’s T2 methodhas empirical type I error rate less than lower bound of the tolerance type I error rate controllable criterion,while others have empirical type I error rate lies in the interval of the tolerance criterion. Sample size do notaffect the capacity of controlling type I error rate but the number of dependent variables affect the capacityof controlling type I error rate. In addition, Westfall-Young bootstrap method and Exact Permutationalmethod have empirical type I error rate lies in the interval of the tolerance criterion. For almost everysituations Westfall-Young bootstrap method has the highest empirical power but Hommel’s method basedon Simes’ test has the the highest empirical power when the correlation coefficient is low. Hotelling’s T2method has the lowest empirical power in all situations. Westfall-Young bootstrap method and ExactPermutational method has similar empirical power in all situations. In addition, empirical power variesaccording to the sample size and the number of dependent variables but varies inversely with thecorrelation coefficient.