Bayesian analysis of the beta regression model subject to linear inequality restrictions with application

Abstract

Recent studies in machine learning are based on models in which parameters or state variables are restricted by a restricted boundedness. These restrictions are based on prior information to ensure the validity of scientific theories or structural consistency based on physical phenomena. The valuable information contained in the restrictions must be considered during the estimation process to improve the accuracy of the estimation. Many researchers have focused on linear regression models subject to linear inequality restrictions, but generalized linear models have received little attention. In this paper, the parameters of beta Bayesian regression models subjected to linear inequality restrictions are estimated. The proposed Bayesian restricted estimator, which is demonstrated by simulated studies, outperforms ordinary estimators. Even in the presence of multicollinearity, it outperforms the ridge estimator in terms of the standard deviation and the mean squared error. The results confirm that the proposed Bayesian restricted estimator makes sparsity in parameter estimating without using the regularization penalty. Finally, a real data set is analyzed by the new proposed Bayesian estimation method.

Keywords

• Bayesian inference
• Beta regression model
• Linear inequality restrictions
• Link function
• Restricted estimator

References

1. [1] M. R. Abonazel and I. M. Taha, Beta ridge regression estimators: simulation and application, Commun. Stat. Simul. Comput. 52(9), 4280–4292, 2021.
2. [2] M. R. Abonazel, Z. Y. Algamal, F. A. Awwad and I. M.Taha, A new two-parameter estimator for beta regression model: Method, simulation, and application. Front. Appl. Math. Stat. 7, 780322, 2022.
3. [3] S. Aktas and H. Unlu, Beta regression for the indicator values of well-being index for provinces in Turkey, J. Eng. Technol. Appl. Sci. 2(2), 101–111, 2017.
4. [4] Z. Y. Algamal, M. R. Abonazel and A. F. Lukman, Modified jackknife ridge estimator for beta regression model with application to chemical data. Int. J. Math. Stat. Comput. 1, 15–24, 2023.
5. [5] Z. Y. Algamal and M. R. Abonazel, Developing a Liu-type estimator in beta regression model. Concurrency Comput. Pract. Exp. 34(5), e6685, 2021.
6. [6] R. Arabi Belaghi, Y. Asar, and R. Larsson, Improved shrinkage estimators in the beta regression model with application in econometric and educational data, Stat. Pap. 64, 1891–1912, 2023.
7. [7] D. G. Bails and L. C. Peppers, Business fluctuations, Englewood Cliffs: Prentice-Hall, 1982.
8. [8] Y. Cong, B. Chen, and M. Zhou, Fast simulation of hyperplane-truncated multivariate normal distributions, Bayesian Anal. 12(4), 1017–1037, 2017.

9. [9] F. Cribari-Neto and A. Zeileis, Beta regression in R, J. Stat. Softw. 34(2), 1–24, 2010.
10. [10] W. W. Davis, Bayesian analysis of the linear model subject to linear inequality constraints, J. Am. Stat. Assoc. 73, 573–579, 1987.
11. [11] L. A. Escobar and B. Skarpness, Mean square error and efficiency of the least squares estimator over interval constraints, Commun. Stat. Theory Methods 16, 397–406, 1987.
12. [12] L. A. Escobar and B. Skarpness, The bias of the least squares estimator over interval constraints, Econ. Lett., 20 331–335, 1986.
13. [13] A. Erkoç, E. Ertan, Z. Y. Algamal and K. U. Akay, The beta Liu-type estimator: simulation and application, Hacet. J. Math. Stat., 52(3), 828–840, 2023.
14. [14] S. Ferrari and F. Cribari-Neto, Beta regression for modelling rates and proportions, J. Appl. Stat. 31(7), 799–815, 2004.
15. [15] J. Geweke, Exact inference in the inequality constrained normal linear regression model, J. Appl. Econom. 1, 127–141, 1986.
16. [16] J. Geweke, Bayesian inference for linear models subject to linear inequality constraints, New York: Springer, 1996.
17. [17] R. Ghosal and S. K. Ghosh, Bayesian inference for generalized linear model with linear inequality constraints, Comput. Stat. Data Anal. 166, 107335, 2022.
18. [18] G. G. Judge and T. Takayama, Inequality restrictions in regression analysis, J. Am. Stat. Assoc. 61, 166–181, 1966.
19. [19] P. Karlsson, K. Månsson and B. M. G. Kibria, A Liu estimator for the beta regression model and its application to chemical data, J. Chemom. 34(6), e3300, 2020.
20. [20] Y. Koike and Y. Tanoue, Oracle inequalities for sign constrained generalized linear models, Econ. Stat. 11, 145–157, 2019.
21. [21] S. Lan, B. Zhou and B. Shahbaba, Spherical Hamiltonian Monte Carlo for constrained target distributions, Proc. Int. Conf. Mach. Learn. 32, 629–637, 2014.
22. [22] Y. Li and S. K. Ghosh, Efficient sampling methods for truncated multivariate normal and Student-t distributions subject to linear inequality constraints, J. Stat. Theory Pract. 9, 712–732, 2014.
23. [23] M. C. Lovell and E. Prescott, Multiple regression with inequality constraints: Pretesting bias, hypothesis testing and efficiency, J. Am. Stat. Assoc. 65, 913–925, 1970.
24. [24] T. F. R. Ma, S. K. Ghosh and Y. Li, Sampling from truncated multivariate normal and t distributions, version 1.0.2, 2018. Available at: https://cran.r-project.org/ web/packages/tmvmixnorm
25. [25] S. Mahmood, N. Seyala and Z. Y. Algamal, Adjusted R2-type measures for beta regression model. Electron. J. Appl. Stat. Anal. 13(2), 350–357, 2020.
26. [26] D. Manolakis and G. Shaw, Detection algorithms for hyperspectral imaging applications, IEEE Signal Process. Mag. 19, 29–43, 2002.
27. [27] N. Meinshausen, Sign-constrained least squares estimation, Electron. J. Stat. 7, 1607–1631, 2013.
28. [28] B. Neelon and D. B. Dunson, Bayesian isotonic regression and trend analysis, Biometrics 60, 398–406, 2004.
29. [29] K. Ohtani, The MSE of the least squares estimator over an interval constraint, Econ. Lett. 25, 351–354, 1987.
30. [30] A. Pakman and L. Paninski, Exact Hamiltonian Monte Carlo for truncated multivariate Gaussians, J. Comput. Graph. Stat. 23(2), 518–542, 2014.
31. [31] R. S. Pindyck and D. L. Rubinfeld, Econometric Models and Economic Forecasts, 2nd ed., McGraw-Hill, New York, 1981.
32. [32] M. Qasim, K. Månsson, and B. M. G. Kibria, On some beta ridge regression estimators: method, simulation and application, J. Stat. Comput. Simul. 91, 1699–1712, 2021.
33. [33] G. Rodriguez-Yam, R. A. Davis, and L. L. Scharf, Efficient Gibbs sampling of truncated multivariate normal with application to constrained linear regression, Technical Report, Colorado State University, unpublished manuscript, 2004.
34. [34] S. Seifollahi and H. Bevrani, James-Stein type estimators in beta regression model: simulation and application, Hacet. J. Math. Stat. 52(4), 1046–1065, 2023.
35. [35] S. Seifollahi, K. Kamary, and H. Bevrani, Bayesian estimation approach for linear regression models with linear inequality restrictions, arXiv:2112.02950, 2021.
36. [36] S. Seifollahi, H. Bevrani, and K. Kamary, Inequality restricted estimator for Gamma regression: Bayesian approach as a solution to the multicollinearity, Commun. Stat. Theory Methods 53(23), 8297–8311, 2023.
37. [37] S. Seifollahi, H. Bevrani and O. Albalawi, Reducing bias in beta regression models using Jackknifed Liu-type estimators: applications to chemical data. J Math. 6694880, 2024.
38. [38] M. Slawski and M. Hein, Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization, Electron. J. Stat. 7, 3004–3056, 2013.
39. [39] M. Smithson and J. Verkuilen, A better lemon squeezer? Maximum-likelihood regression with beta-distributed dependent variables, Psychol. Methods 11(1), 54–71, 2006.
40. [40] S. D. Veiga and A. Marrel, Gaussian process regression with linear inequality constraints, Reliab. Eng. Syst. Saf. 195, 106732, 2020.
41. [41] J. Wang and S. K. Ghosh, Shape restricted nonparametric regression with Bernstein polynomials, Comput. Stat. Data Anal. 55, 2729–2741, 2012.
42. [42] J. Zhu, R. Santerre, and X. W. Chang, A Bayesian method for linear, inequality constrained adjustment and its application to GPS positioning, J. Geod. 78, 528–534, 2005.