Ordinary Least Squares and Robust Estimators in Linear Regression: Impacts of Outliers, Error and Response Contaminations
David Adedia *
Department of Basic Sciences, University of Health and Allied Sciences, P.M.B. 31, Ho, Ghana.
Atinuke Adebanji
Department of Mathematics, Kwame Nkrumah University of Science and Technology, P.M.B. KNUST, Kumasi, Ghana.
Moremi Labeodan
School of Business and Economics, Monash University (South African Campus), P/B X60, Roodepoort, South Africa.
Shola Adeyemi
Sirtex Medical Europe GmbH, Bonn, Germany.
*Author to whom correspondence should be addressed.
Abstract
The Ordinary Least Squares Estimator (OLSE) is the best method for linear regression if the classical assumptions are satis ed for estimating weights. When these assumptions are violated, the robust methods give more reliable estimates while the OLSE is strongly a ected adversely. In order to assess the sensitivity of some estimators using more than ve criteria, a secondary dataset on Anthropometric measurements from Komfo Anokye Teaching Hospital, Kumasi-Ghana, is used. In this study, we compare the performance of the Huber Maximum Likelihood Estimator (HMLE), Least Trimmed Squares Estimator (LTSE), S Estimator (SE) and Modified Maximum Likelihood Estimator (MMLE) relative to the OLSE when the dataset has normal errors; 10, 20 and 30 percent outliers; 20% error contamination and lognormal contamination in the response variable. In the assessment, we use coefficients and their standard errors, relative efficiencies, Root Mean Square Errors, and the coefficients of determination of the estimators. We also use the power of the test to assess the e ects of the aberrations on the post hoc power analysis of the estimators. The results show the SE and MMLE outperform the HMLE and LTSE while the OLSE breaks down completely. The LTSE performs well when the trimming is done to eliminate only the outliers. Also, SE and MMLE resist the e ect of all aberrations in the data and also have good post hoc power analysis.
Keywords: Ordinary least squares estimator, robust estimators, power of the test, outliers, errors