Robust Bayesian Regression Analysis Using Ramsay-Novick Distributed Errors with Student-t Prior

Abstract

This paper investigates bayesian treatment of regression modelling with Ramsay - Novick (RN)  distribution  specifically  developed for robust  inferential procedures. It falls into the category of the so-called heavy-tailed distributions generally accepted as outlier resistant densities. RN is obtained by coverting the usual form of a non-robust density to a robust likelihood through  the  modification of its unbounded influence function. The  resulting  distributional form  is  quite  complicated  which  is  the  reason  for  its limited  applications   in  bayesian  analyses of real problems. With the help of innovative Markov Chain Monte Carlo (MCMC)  methods  and  softwares  currently  available,  here   we   first suggested  a  random  number  generator for  RN  distribution.  Then,  we developed  a  robust bayesian modelling with RN distributed errors and Student-t prior. The  prior  with  heavy-tailed  properties  is  here  chosen to  provide   a   built-in protection  against   the   misspecification   of   conflicting  expert  knowledge  (i.e. prior robustness). This is particularly useful to avoid accusations of too much subjective bias in the prior specification.  A  simulation  study conducted for performance assessment  and   a  real-data  application  on   the   famously   known  "stack loss"  data  demonstrated   that  robust  bayesian  estimates  with  RN likelihood and  heavy-tailed  prior are robust against outliers in all directions and inaccurately specified priors.

Keywords

• Robust bayesian regression,Ramsay-Novick,heavy-tailed distribution,Student-t prior,prior robustness

References

1. Abramowitz, M. and Stegun, I.A. (Eds), Handbook of Mathematical Functions with Formulas, Graphs and Mathematical Tables, 9th printing, New York: Dover 260 1972.
2. Aitkin, M. and Wilson, G.T., Mixture models, outliers and the EM algorithm, Techno (1980), 22, 325-331.
3. Albert, J., Delampady, M. and Polasek, W., A class of distributions for robustness studies, Journal of Statistical Planning and Inference (1991), 28, 291-304.
4. Andrews, D. R. and Mallows, C. L., Scale mixtures of normal distributions, J. R. Statist. Soc. B (1974),36, 99-102.
5. Antoch, J. and Jureckova, J., Trimmed least squares estimator resistant to leverage points, CSQ (1985), 4, 329-339.
6. Arfken, G., The Incomplete Gamma Function and Related Functions, Mathematical Methods for Physicists, 3rd ed. Orlanto, FL. Academic Press (1985), 565-572.
7. Atkinson, A. C., Two graphical displays for outlying and influential observations in regression, Bimetrika (1981), 68(1), 13-20.
8. Berger, J.O., Robust Bayesian analysis: sensitivity to the prior, J. Statist. Plann. Inference, (1990),25, 303-328.

9. Berger, J.O., An overview of robust Bayesian analysis, Technical Report (1994), 5-124.
10. Berger, J.O.,Rios INSUA, D. and Ruggeri, F., Bayesian robustness. In Robust Bayesian Analysis (D.Rios Insua and F.Ruggeri, eds) New York: Springer-Verlag 2000.
11. Birkes, D. and Dodge, Y., Alternative Methods of Regression, J. Wiley New York, 177-179, 1993.
12. Box, G. E. P. and Tiao, G. C., A further look at robustness via Bayes's theorem, Biometrika (1962), 49, 419-432.
13. Brownlee, K. A., Statistical Theory and Methodology in Science and Engineering, J. Wiley New York, 491-500, 1960.
14. Chaturvedi, A., Robust Bayesian analysis of the linear regression model, Journal of Statistical Planning and Inference (1996), 50(2), 175-186.
15. Chen, Y. and Fournier, D., Impacts of atypical data on Bayesian inference and robust Bayesian approach in fisheries, Can. J. Fish. Aquat. Sci. (1999), 56, 1525-1533.
16. Choy, S. T. B. and Smith, A. F. M., Hierarchical models with scale mixtures of normal distributions, TEST (1997), 6(1), 205-221.
17. Choy, S. T. B. and Walker, S. G., The extended exponential power distribution and Bayesian robustness, Statistics & Probability Letters (2003), 65, 227-232.
18. Denby, L. and Mallows, C. L., Two diagnostic displays for robust regression analysis, Techno (1977), 19, 1-13.
19. Dodge, Y., The guinea pig of multiple regression, Robust Statistics, Data Analysis and Computer Intensive Methods (Lecture Notes in Statistics), Springer-Verlag New York, 109, 91-118, 1996.
20. Gelman, A., Carlin, J. B., Stern, H. S. and Rubin, D. B., Bayesian data analysis (2nd ed.) Boca Raton: Chapman and Hall / CRC 2004.
21. Gilks, W. R., Richardson, S. and Spiegelhalter, D. J., Markov Chain Monte Carlo In Practice, Chapman & Hall / CRC 1996.
22. Hastings, W. K., Monte Carlo Sampling Methods Using Markov Chains and Their Applications, Biometrika (1970), 57(1), 97-109.
23. Hettmansperger, T. p. and Mc Kean, J. W., A robust alternative based on ranks to least squares in analyzing linear models, Techno (1977), 19, 275-284. Lange, K. L., Little, R. J. A. and Taylor, J. M. G., Robust statistical modelling using the t distribution, Journal of the American Statistical Association (1989), 84, 881-896.
24. Lange, K. and Sinsheimer, J. S., Normal/Independent Distributions and Their Applications in Robust Regression, Journal of Computational and Graphical Statistics (1993), 2, 175-198.
25. Liu, C., Bayesian Robust Multivariate Linear Regression with Incomplete Data, Journal of the American Statistical Association (1996), 91(435), 1219-1227.
26. Mendoza, M. and Pena, E. G., Some thoughts on the Bayesian robustness of location-scale models, Chilean Journal of Statistics (2010), 1(1), 35-58.
27. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M. N., Teller, A. H. and Teller, E., Equations of State Calculations by Fast Computing Machines, Journal of Chemical Physics (1953), 21, 1087-1092.
28. O'Hagan, A. and Pericchi, L., Bayesian heavy-tailed models and conflict resolution: A review, Brazilian Journal of Probability and Statistics (2012), 26 (4), 372-401.
29. Osborne, M. R., Finite Algorithms in Optimization and Data Analysis, J. Wiley New York, 267-270, 1985.
30. Passarin, K., Robust Bayesian Estimation, Department of Economics of the University of Insubria, Varese, Italy. 2004.
31. Preece, D. A., Illustrative examples: illustrative of what?, Statistician (1986), 35, 33. Ramsay, J. O. and Novick, M. R., PLU robust Bayesian decision theory: point estimation, Journal of the American Statistical Association (1980), 75 (372), 901-907.
32. R Development Core Team R (http://www.R-project.org.), A language and environment for statistical computing, R Foundation for Statistical Computing, Vienna, Austria, ISBN 3-000051-07-0, 2013.
33. Rey, W. j. j., M-Estimators in Robust Regression, a case study, M. B. L. E. Researc. Lab. Brussels 1977.
34. Roberts, G. O. and Rosenthal, J. S., Optimal Scaling for Various Metropolis-Hastings Algorithms, Statist. Science, (2001), 16, 351-367.
35. Ruppert, D. and Carroll, R. J., Trimmed least squares estimation in the linear model, J. Americ. Statist. Assoc. (1980), 75, 828-838.
36. Shi, J., Chen, K. and Song, W., Robust errors-in-variables linear regression via Laplace distribution, Statistics and Probability Letters (2014), 84, 113-120.
37. Vallejos, C. A. and Steel, M. F. J., On posterior propriety for the Student-t linear regression model under Jeffreys priors, arXiv: 1311.1454v2 [stat. ME] 2013.
38. Wall, H. S. Analytic Theory of Continued Fractions, New York: Chelsea, 1948.
39. West, M., Outlier Models and Prior Distributions in Bayesian Linear Regression, Journal Royal Statistical Society (1984), 46(3), 431-439.