Study on the penalty functions of model selection criteria

Abstract

The aim of this paper is to study the penalty functions of the well-known model selection criteria, AIC , BIC , and KIC , which can unify their formulas as APIC = log(2)+(p+1)/n, called Adjusted Penalty Information Criterion. The appropriate value of  for APIC has been found to reduce the probabilities of over- and underfitting and also to overcome the weak signal-to-noise ratio. The value of  is selected based on four measurements: the probability of over- and underfitting, the signal-to-noise ratio, the probability of order selected, and the observed L₂ efficiency. Performance of APIC is examined by theoretical and extensive simulation study. The theoretical results show that, the probability of overfitting tends to zero and the signal-to-noise ratio tends to strong if the value of  tends to infinity. However, the simulation results show that, when the true model is weakly identifiable, the small value of  gives a high probability of correct order being selected. But, if the true model is very difficult to detect, the observed L₂ efficiency is a meaningful measurement than the probability of order selected. The observed L₂ efficiency suggests the large value of  causes the high efficiency of APIC which indicates that the correct model is also the closet model, except when the true model can be specified more easily and sample sizes are moderate to large, then the small value of is preferable. For the strongly identifiable true model, the large value of performs well, whereas if the regression coefficients are not large enough and the sample sizes are small to moderate, the value of  should be moderate.