Parameter Estimation for Robust Regression with Maximum Likelihood Estimation and S-estimation
Article Sidebar
Main Article Content
Abstract
For parameters estimation or coefficients estimation in linear model, the Least Square (LS) estimator of parameters has always turned out to be the best linear unbiased estimator. However, if the observations contain outliers or high leverage, this may affect the Least Square estimates. So, an alternative approach; the so-called robust regression method, is needed to obtain a better fit of the model or more precise estimator of parameters. This article presents the procedure of robust regression methods, namely, the maximum likelihood and S-estimation methods and shows the comparison of the efficiency of the parameter resulted from the maximum likelihood method when using the Huber’s function and that of the S-estimation method. The criterion for efficiency comparison was the mean square error (MSE) and the coefficient of determination. From the information collected from research reports, it was found that the parameters estimation with the S-estimation method was more effective than that of the maximum likelihood method when using the Huber’s function, in the case that the observations from independent variables having high leverage when the error distribution being normal.
Article Details
References
Adedia, D., Adebanji, A., Labeodan, M., and Adeyemi, S. (2016). Ordinary Least Squares and Robust Estimators in Linear Regression: Impacts of Outliers, Error and Response Contaminations. Mathematics & Computer Science, 13(4), 1-11.
Ahmad, W. M. A. W., and Shafig, M. (2013). High Density Lipoprotein Cholesterol Predicts Triglycerides Level in Three Distinct phases of Blood Pressure. Basic and Applied Research, 10(1), 38-46.
Ahmed, M. G., and Maha, E. Q. (2016). Regression Estimation in the Presence of Outliers: A Comparative Study. Probability and Statistics, 5(3), 65-72.
Alma, O. G. (2011). Comparison of Robust Regression Method. Contemp. Math. Science, 6(9), 409-421.
Almetwally, E. M., and Almohgy, H. M. (2018). Comparison Between M-estimation, S-estimation, and MM estimation Methods of Robust Estimation with Application and Simulation. Mathematical Archive, 9(11), 55-63.
Candan, M. (1995). Robust Estimators in Linear Regression Analysis. Thesis of the Degree of Master of Science Program in Statistics. Ankara, Turkey: Hacettepe University.
Meral, C., and Onur, T. (2011). The Comparing of S-estimator and M-estimator in Linear Regression. Science, 24(4), 747-752.
Montgomery, D. C., Peck, E. A. and Vining, G. G. (2006). Introductions to Linear Regression Analysis. 4th ed. New York: John Wiley & Sons.
Panik, M. (2009). Regression Modeling Methods, Theory, and Computation with SAS.New York: Taylor & Francis Group.
Pitselis, G. (2013). A review on robust estimators applied to regression credibility. Computational and Applied Mathematics, 239(8), 231-249.
Rousseeuw, P. J., and Leroy, A. M. (2003). Robust regression and outlier detection. New York: John Wiley & sons.
Shafiq, M., Amir, W. M., and Zafakali, N. S. (2017). Algorithm for Comparison of Robust Regression Methods In Multiple Linear Regression By Weighting Least Square Regression (SAS). Modern Applied Statistical Methods, 16(2), 490-505.
Yarmohammadi, M., and Mahmoudvand, R. (2010). The effect of outliers on robust and resistant coefficient of determination in the linear regression models. Academic Research, 2(3), 133-138.
Yu, C. and Yao, W. (2017). Robust Linear Regression: A Review and Comparison. Communication in Statistics, 46(8), 6261-6282.