Principal simple linear regression

Abstract

In this paper, we propose a new method called principal simple linear regression for predicting a continuous response variable, $Y$ using a single continuous predictor, $X$ by utilizing multiple regression lines. This method is based on the theory of principal points and offers several advantages over classical simple linear regression methods, such as the ability to predict central, dispersion, and distributional tendencies of $Y$ on $X$, and simultaneous estimation. We provide the main properties, inferences, and limiting behavior of the estimators. Additionally, we conduct a comprehensive simulation study to validate our theoretical results. The model is also applied to real datasets to demonstrate its effectiveness.

Keywords

• Simple linear regression
• principal points
• quantile regression

References

1. [1] A.C. Atkinson, Plots, Transformations and Regression, Oxford University Press, 1985.
2. [2] J.L. Bali and G. Boente, Principal points and elliptical distributions from the multivariate setting to the functional case, Stat. Probab. Lett. 79, 1858–1865, 2009.
3. [3] T. Bollerslev, Generalized autoregressive conditional heteroskedasticity, J. Econom. 31 (3), 307–327, 1986.
4. [4] K.G. Brown, Estimation of variance components using residuals, J. Am. Stat. Assoc. 73 (361), 141–146, 1978.
5. [5] R.F. Engle, Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation, Econometrica 50 (4), 987–1007, 1982.
6. [6] B. Flury, Estimation of principal points, J. R. Stat. Soc., C: Appl. Stat. 42 (1), 139–151, 1993.
7. [7] B. Flury, Principal points, Biometrica 77 (1), 33–41, 1990.
8. [8] H.V. Henderson and P.F. Velleman, Building multiple regression models interactively, Biometrics 37, 391-411, 1981.

9. [9] J. Kim and H. Kim, Partitioning of functional gene expression data using principal points, BMC Bioinformatics 18, 1–17, 2017.
10. [10] R. Koenker, Quantile Regression, Cambridge University Press, 2005.
11. [11] S. Matsuura and H. Kurata, Principal points of a multivariate mixture distribution, J. Multivar. Anal. 102 (2), 213–224, 2011.
12. [12] S. Matsuura, H. Kurata and T. Tarpey, Optimal estimators of principal points for minimizing expected mean squared distance, J. Stat. Plan. Inference 167, 102–122, 2015.
13. [13] T.A. Ryan, B.L. Joiner and B.F. Ryan, The Minitab Student Handbook, Duxbury Press, 1976.
14. [14] E. Stampfer and E. Stadlober, Methods for estimating principal points, Commun. Stat. B: Simul. Comput. 31 (2), 261–277, 2002.
15. [15] T. Tarpey, Estimating principal points of univariate distributions, J. Appl. Stat. 24 (5), 499–512, 1997.
16. [16] T. Tarpey, Principal points, PhD Thesis, ProQuest, 1992.
17. [17] T. Tarpey, L. Li and B. Flury, Principal points and self-consistent points of elliptical distributions, Ann. Stat. 23 (1), 103–112, 1995.
18. [18] W. Yamamoto and N. Shinozaki, On uniqueness of two principal points for univariate location mixtures, Stat. Probab. Lett. 46 (1), 33–42, 2000.
19. [19] H. Yamashita, A Study on Principal Points for a Multivariate Binary Distribution, PhD Thesis, 2015.
20. [20] H. Yamashita, S. Matsuura and H. Suzuki, Comparison of model selection methods for the estimation of principal points for a multivariate binary distribution, Total Qual. Sci. 1 (1), 22–31, 2015.
21. [21] F. Yu, Uniqueness of principal points with respect to p-order distance for a class of univariate continuous distribution, Stat. Probab. Lett. 183, 109341, 2022.
22. [22] A. Zoppe, Principal points of univariate continuous distributions, Stat. Comput. 5, 127–132, 1995.