Comparison of Model Selection Criteria for Classical Designed Experiments

Abstract

Model selection procedures play an important role in many researches especially quantitative research. In science research and Biological sciences, the analysis and model selection of experiments are often used. In this article, we studied a 22 and 32 factorial design. Only qualitative factors were considered. These designs are used in this study for model selections by using Akaike’s Information Criterion (AIC), using p-value from ANOVA (P-Value Method), and Stepwise Procedure. In additional, there are two types under this study of the model specified: Type I and Type II corresponding to a model having “relatively few” important effects and a model having “many” important effects respectively. The results of this study will be presented for the model selection performance and for comparisons of selected models that match between methods. For each design and types of models, a model is selected and the proportion of selected models that are the true or “correct” model is recorded. These results will point to which model selection methods select the correct model more often than others. Furthermore, the final models of two model selection methods will be compared using the proportion of selected models that match for each of the 500 simulated replications. These results indicate which methods tend to select the same or different final model. The conclusions for a 22 factorial design, the model selection methods that selected the correct models and the model that match related with replicates per experimental treatment and the value of parameter for both model types. For a 32 factorial design, the P-value and Stepwise Methods selected the corrected model often the other method and also selected the same model more often than the other pair of method for all replicates per experimental treatment (n=2, 4), all values of the parameters and both model types.