Comparison of Classical Linear Regression and Orthogonal Regression According to the Sum of Squares Perpendicular Distances

Abstract

Regression analysis is a statistical technique for investigating and modeling the relationship between variables. The purpose of this study was the trivial presentation of the equation for orthogonal regression (OR) and the comparison of classical linear regression (CLR) and OR techniques with respect to the sum of squared perpendicular distances. For that purpose, the analyses were shown by an example. It was found that the sum of squared perpendicular distances of OR is smaller. Thus, it was seen that OR line has appeared to present a much better fit for the data than CLR line. Depending on those results, the OR is thought to be a regression technique to obtain more accurate results than CLR at simple linear regression studies.

References

1. Adcock, R.J. (1878). A Problem in least squares, Annals of Mathematics, 5(2), 53-54.
2. Akdeniz, F. (2013). Olasılık ve İstatistik, Nobel Kitabevi, 18. Press, Ankara, 459-492.
3. Bahar, H.H. (2006). An Evaluation of KPSS Scores According to Grade Point Avarege and Gender, Education and Science, 31(140), 68-74.
4. Bağçeci, B., Döş, B. & Sarıca, R. (2011). An Analysis of Metacognitive Awareness Levels and Academic Achievement of Primary School Students, Mustafa Kemal University Journal of Social Sciences Institute, 8(16), 551-566.
5. Bayat, N., Şekercioğlu, G. & Bakır, S. (2014). Okuduğunu Anlama ve Fen Başarısı Arasındaki İlişkinin Belirlenmesi, Eğitim ve Bilim, (tedmem), 39(176), 457-466.
6. Calzada, M.E. & Scariano, S.M. (2003). Contrasting total least squares with ordinary least squares part II: Examples and Comparisons, Mathematics and Computer Education, 37(2), 159-174.
7. Carr, J.R. (2012). Orthogonal regression: A Teaching perspective, International Journal of Mathematical Education in Science and Technology, 43(1), 134-143.
8. Carroll, R.J. & Ruppert D. (1996). The use and misuse of orthogonal regression in linear errors-in-variables models, The American Statistician, 50 (1), 1-6.

9. de Julián-Ortiz, J.V., Pogliani, L. & Besalú, E. (2010). Two-variable linear regression: Modeling with orthogonal least-squares analysis, Journal of Chemical Education, 87(9), 994-995.
10. Ding, G., Chu, B., Jin, Y. & Zhu, C. (2013). Comparison of orthogonal regression and least squares in measurement error modeling for prediction of material property, Nanotechnology and Material Engineering Research. Advanced Materials Research, 661, 166-170.
11. Doruk, M., Özdemir, F. & Kaplan, A. (2015). The Relationship Between Prospective Mathematics Teachers’ Conceptions on Constructing Mathematical Proof and Their Self-Efficacy Beliefs Towards Mathematics, Kastamonu Üniversitesi Kastamonu Eğitim Dergisi, 23(2), 861-874.
12. Elfessi, A. & Hoar, R.H. (2001). Simulation study of a linear relationship between two variables affected by errors, Journal of Statistical Computation and Simulation, 71(1), 29-40.
13. Fišerová, E. & Hron, K. (2010). Total least squares solution for compositional data using linear models, Journal of Applied Statistics, 37(7), 1137-1152.
14. Glaister, P. (2005). The use of orthogonal distances in generating the total least squares estimate, Mathematics and Computer Education, 39(1), 21-30.
15. Golub, G.H. & Van Loan, C.F. (1980). An Analysis of the total least squares problem, SIAM Journal on Numerical Analysis, 17(6), 883-893.
16. İlğan, A., Erdem, M., Yapar, B., Aydın. S. & Aydemir, Ş.Ş. (2012). Parents Interest and Regression Level of Primary State School Students Level Determination Exam (SBS) Score, Journal of Educational Sciences Research, 2(2), 1-17.
17. Isobe, T., Feigelson, E.D, Akritas, M.G. & Babu, G.J. (1990). Linear regression in astronomy I, The Astrophysical Journal, 364, 104-113.
18. Kane, M.T. & Mroch, A.A. (2010). Modeling group differences in OLS and Orthogonal Regression: Implications for differential validity studies, Applied Measurement in Education, 23, 215-241.
19. Kermack, K.A. & Haldane, J.B.S. (1950). Organic correlation and allometry, Biometrika Trust, 37(1), 30–41.
20. Kesicioğlu, O.S. & Güven, G. (2014). Investigation of the Correlation Between Preservice Early Childhood Teachers' Self-Efficacy Levels and Problem Solving, Empathy and Communication Skills, Turkish Studies, International Periodical For the Languages, Literature and History of Turkish or Turkic, 9(5), 1371-1383.
21. Lane, D. M. (2016). “Introduction to Linear Regression”,
22. http://onlinestatbook.com/2/regression/intro.html Accessed 21 February 2016.
23. Leng, L., Zhang, T, Kleinman, L. & Zhu, W. (2007). Ordinary Least Square Regression, Orthogonal Regression, Geometric Mean Regression and Their Applications in Aerosol Science, Journal of Physics, Conference Series 78(1), 1-5, http://iopscience.iop.org/1742-6596/78/1/012084 Accessed 17 March 2016.
24. Li, H.C. A. (1984). Generalized problem of least squares, The American Mathematical Monthly, 91(2), 135-137.
25. Ludbrook, J. (2010). Linear regression analysis for comparing two measurers or methods of measurement: But which regression?, Clinical and Experimental Pharmacology and Physiology, 37, 692-699.
26. Markovsky, I. & Van Huffel, S. (2007). Overview of total least- squares methods, Signal Processing, 87, 2283-2302.
27. Montgomery D.C., Peck E.A. & Vining G.G. (2012). Introduction to Linear Regression Analysis, 5th Edition, 1-11.
28. http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0470542810.html Accessed 11 February 2016.
29. Nievergelt, Y. (1994). Total least squares: State-of-the-art regression in numerical analysis, SIAM Review (Society for Industrial and Applied Mathematics), 36(2), 258–264.
30. Pearson, K. (1901). On lines and planes of closest fit to systems of points in space, Philosophical Magazine, 2, 559-572.
31. Scariano, S.M. & Barnet, II.W. (2003). Contrasting total least squares with ordinary least squares part I: Basic ideas and result, Mathematics and Computer Education, 37(2), 141-158.
32. Sykes, A.O. (2016). “An Introduction to Regression Analysis”. http://www.law.uchicago.edu/files/files/20.Sykes_.Regression.pdf. Accessed 5 March 2016.
33. Üredi, I. & Üredi, L. (2005). İlköğrtim 8. Sınıf Öğrencilerinin Öz-düzenleme Stratejileri ve Motivasyonel İnançlarının Matematik Başarısını Yordama Gücü, Mersin University Journal of the Faculty of Education, 1(2), 250-260.