A heteroscedastic regression model with the generalized normal distribution

Abstract

In regression analysis, joint modeling mean and dispersion is an essential tool in absence of the variance homogeneity. Moreover, it is known in the literature that the generalized normal (GN) distribution has some features that provide flexibility in modeling thanks to its shape parameter. This paper proposes a joint location and scale model of the GN distribution for modeling location and scale in the presence of heteroscedasticity. We provide maximum like-lihood (ML) estimators for the parameters of the proposed model. We also give an estimation procedure to estimate all parameters simultaneously. For the application, some simulation study scenarios and a real-life example are carried out to prove the estimation performance of the proposed model.

Keywords

• GN
• Joint Location And Scale Model
• Laplace Distribution
• ML
• Normal Distribution

References

1. REFERENCES
2. [1] Briassouli A, Tsakalides P, Stouraitis A. Hidden messages in heavy-tails: DCT-domain watermark detection using alpha-stable models. IEEE Trans Multimedia 2005;7:700–715. [CrossRef]
3. [2] Kokkinakis K, Nandi AK. Exponent parameter estimation for generalized Gaussian probability density functions with application to speech modeling. Signal Process 2005;85:1852–1858. [CrossRef]
4. [3] Sharifi K, Leon-Garcia A. Estimation of shape parameter for generalized Gaussian distributions in subband decompositions of video. IEEE Trans Circuits Syst Video Technol 1995;5:52–56. [CrossRef]
5. [4] Choi S, Cichocki A, Amari S-I. Flexible Independent Component Analysis. J VLSI Signal Process Syst Signal Image Video Technol 2000;26:25–38. [CrossRef]
6. [5] Wu H-C, Principe JC. Minimum entropy algorithms for source separation. 1998 Midwest Symposium on Circuits and Systems (Cat. No. 98CB36268), 1998. p. 242–245.
7. [6] Varanasi MK, Aazhang B. Parametric generalized Gaussian density estimation. J Acoust Soc Am 1989;86:1404–1415. [CrossRef]
8. [7] Doğru FZ, Arslan O. Joint modelling of the location, scale and skewness parameters of the skew laplace normal distribution. Iran J Sci Technol Trans A Sci 2019;43:1249– 1257. [CrossRef]

9. [8] Doğru FZ, Yu K, Arslan O. Heteroscedastic and heavy-tailed regression with mixtures of skew Laplace normal distributions. J Stat Comput Simul 2019;89:32133240. [CrossRef]
10. [9] Park RE. Estimation with heteroscedastic error terms. Econometrica 1966;34:888. [CrossRef]
11. [10] Harvey AC. Estimating regression models with multiplicative heteroscedasticity. Econometrica 1976:461–465. [CrossRef]
12. [11] Aitkin M. Modelling variance heterogeneity in normal regression using GLIM. J R Stat Soc Ser C Appl Stat 1987;36:332–339. [CrossRef]
13. [12] Verbyla AP. Modelling variance heterogeneity: residual maximum likelihood and diagnostics. Journal of the Royal Statistical Society: Series B (Methodological) 1993;55:493–508. [CrossRef]
14. [13] Engel J, Huele AF. A generalized linear modeling approach to robust design. Technometrics 1996;38:365–373. [CrossRef]
15. [14] Taylor J, Verbyla A. Joint modelling of location and scale parameters of the t distribution. Stat Modelling 2004;4:91–112. [CrossRef]
16. [15] Lin T-I, Wang Y-J. A robust approach to joint modeling of mean and scale covariance for longitudinal data. J Stat Plan Inference 2009;139:3013–3026. [CrossRef]
17. [16] Lin T-I, Wang W-L. Bayesian inference in joint modelling of location and scale parameters of the t distribution for longitudinal data. J Stat Plan Inference 2011;141:1543– 1553. [CrossRef]
18. [17] Li H, Wu L. Joint modelling of location and scale parameters of the skew-normal distribution. Applied Mathematics-A Journal of Chinese Universities 2014;29:265–272. [CrossRef]
19. [18] Wu L, Li H. Variable selection for joint mean and dispersion models of the inverse Gaussian distribution. Metrika 2012;75:795–808. [CrossRef]
20. [19] Zhang Z. Variable selection for joint mean and dispersion models of the lognormal distribution. Hacettepe J Math Stat 2012;41:307–320.
21. [20] Azzalini A. A class of distributions which includes the normal ones. Scand J Stat 1985;12:171–178.
22. [21] Azzalini A. Further results on a class of distributions which includes the normal ones. Statistica 1986;46:199–208.
23. [22] Wu L-C, Zhang Z-Z, Xu D-K. Variable selection in joint location and scale models of the skew-normal distribution. J Stat Comput Simul 2013;83:1266–1278. [CrossRef]
24. [23] Wu L-C. Variable selection in joint location and scale models of the skew-t-normal distribution. Commun Stat Simul Comput 2014;43:615–630. [CrossRef]
25. [24] Zhao W, Zhang R. Variable selection of varying dispersion student-t regression models. J Syst Sci Complex 2015;28:961–977. [CrossRef]
26. [25] Li H, Wu L, Ma T. Variable selection in joint location, scale and skewness models of the skew-normal distribution. J Syst Sci Complex 2017;30:694–709. [CrossRef]
27. [26] Wu L, Tian G-L, Zhang Y-Q, Ma T. Variable selection in joint location, scale and skewness models with a skew-t-normal distribution. Stat Interface 2017;10:217–227. [CrossRef]
28. [27] Subbotin MT. On the law of frequency of error. Matematicheskii Sbornik 1923;31:296–301.
29. [28] Nadarajah S. A generalized normal distribution. J Appl Stat 2005;32:685–694. [CrossRef]
30. [29] Varanasi MK, Aazhang B. Parametric generalized Gaussian density estimation. J Acoust Soc Am 1989;86:1404–1415. [CrossRef]
31. [30] Roenko AA, Lukin V V, Djurović I, Simeunović M. Estimation of parameters for generalized Gaussian distribution. 2014 6th International Symposium on Communications, Control and Signal Processing (ISCCSP), IEEE; 2014. p. 376–379. [CrossRef]
32. [31] Eskin EN. Joint modelling of the location and scale parameters of the generalized normal distribution (master’s thesis). Giresun: Giresun University; 2022.
33. [32] Dominguez-Molina JA, González-Farías G, Rodríguez-Dagnino RM, Monterrey IC. A practical procedure to estimate the shape parameter in the generalized Gaussian distribution. Technique Report I-01-18_eng Pdf, Available through Http://Www Cimat Mx/Reportes/Enlinea/I-01-18_eng Pdf 2003;1. [ 33] Butler RJ, McDonald JB, Nelson RD, White SB. Robust and partially adaptive estimation of regression models. Rev Econ Stat 1990;72:321–327. [CrossRef]
34. [34] Jones MC. A skew t distribution. Probability and Statistical Models with Applications. New York: Taylor & Francis Group; 2001. p. 269–278. [CrossRef]
35. [35] Azzalini A, Capitanio A. Distributions generated by perturbation of symmetry with emphasis on a multivariate skew t-distribution. J R Stat Soc Series B Stat Methodol 2003;65:367–389. [CrossRef] [ 36] Breusch TS, Pagan AR. A simple test for heteroscedasticity and random coefficient variation. Econometrica 1979;47:12871294. [CrossRef]
36. [37] Akaike H. Information Theory and an Extension of the Maximum Likelihood Principle. In: Parzen E, Tanabe K, Kitagawa G, editors. Selected Papers of Hirotugu Akaike, New York, NY: Springer New York; 1998, p. 199–213. [CrossRef]
37. [38] Schwarz G. Estimating the Dimension of a Model. The Annals of Statistics 1978;6:461–464. [CrossRef]
38. [39] Bai Z.-D, Krishnaiah PR, Zhao L.-C. On rates of convergence of efficient detection criteria in signal processing with white noise. IEEE Trans Inf Theory 1989;35:380–388. [CrossRef]
39. [40] Bozdogan H. Choosing the Number of Component Clusters in the Mixture-Model Using a New Informational Complexity Criterion of the Inverse-Fisher Information
40. Matrix. In: Opitz O, Lausen B, Klar R, editors. Information and Classification, Berlin, Heidelberg: Springer Berlin Heidelberg; 1993, p. 40–54. [CrossRef]
41. [41] Phillips RF. Least absolute deviations estimation via the EM algorithm. Stat Comput 2002;12:281–285. [CrossRef]
42. [42] Yang F. Robust mean change-point detecting through laplace linear regression using EM algorithm. J Appl Math 2014;2014:856350. [CrossRef] Dempster AP, Laird NM, Rubin DB. Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Series B Stat Methodol 1977;39:1–22. [CrossRef