An Application of the Bayesian Model Selection By Using Bayes Factor, Bayesian Informatıon Criterion And Deviance Information Criterion

Abstract

In statistical modelling studies, due to the advanced technology and methodological developments, it is possible to construct alternative models assumed to generate the data. Therefore, the process of choosing “the best model” among available competing models appears to be one of the crucial steps that has to be included in the modelling process. In this study, Bayes factor, which is a preferred Bayesian approach to the solution of statistical model selection problem, is introduced. For the cases when analytical computation of Bayes factor is not possible, in addition to Bayesian Information Criterion (BIC), Carlin and Chib method based on Markov Chain Monte Carlo (MCMC) simulation is explained. Besides, a frequently used criteria in the recent years of model selection applications, namely Deviance Information Criterion (DIC), which has a completely different working principle than Bayes factor, is described in detail. Two models appeared in the literature as a result of an application of quantal modelling, which is an example of a semi-parametric modelling, are compared by means of Bayes factor, BIC and DIC.

Keywords

• Bayes factor
• Carlin and Chib method
• DIC
• MCMC
• BIC

Bayes Faktörü, Bayesci Bilgi Ölçütü ve Sapma Bilgi Ölçütü Kullanımıyla Bayesci Model Seçiminin Bir Uygulaması

Abstract

İstatistiksel modelleme çalışmalarında, artan ileri teknoloji ve metodolojik gelişmeler sayesinde veriyi ürettiği varsayılan alternatif modeller oluşturabilmek mümkün olmaktadır. Dolayısıyla, mevcut rakip modeller arasından “en iyi” olanı seçme işlemi, modelleme sürecine dahil edilmesi gereken önemli aşamalardan biri olarak ortaya çıkmaktadır. Bu çalışmada, istatistiksel model seçimi probleminin Bayesci yaklaşımla çözümünde tercih edilen Bayes faktörü tanıtılmış, analitik olarak hesaplanmasının mümkün olmadığı durumlarda kullanılabilen Bayesci Bilgi Ölçütü (BIC) yanı sıra Markov Zincir Monte Carlo (MCMC) simülasyonuna dayalı Carlin ve Chib yöntemi açıklanmıştır. Ayrıca Bayes faktöründen tamamen farklı prensipte çalışan ve son yıllarda model seçimi uygulamalarında sıklıkla kullanılan Sapma Bilgi Ölçütü (DIC) ayrıntılı olarak anlatılmıştır. Bir yarı-parametrik modelleme örneği olan kuantal modellemenin, literatürdeki bir uygulaması sonucu ortaya çıkan alternatif iki model Bayes faktörü, BIC ve DIC kullanılarak kıyaslanmıştır.

Keywords

• Bayes faktörü
• Carlin ve Chib yöntemi
• DIC
• MCMC
• BIC

References

1. Acar, E., 2000. Extensions of Quantal Problems. PhD. Thesis. University of Sheffield Department of Probability and Statistics, UK. (in English).
2. Araujo, M. I., Pereira, B.B., 2007. A Comparison of Bayes factors for Separated Models: Some Simulation Results. Communications in Statistics, Simulation and Computation, 36, 297-309.
3. Asseburg, C., 2007. An Introduction to Using WinBUGS for Cost-Effectiveness Analyses in Health Economics Centre for Health Economics, University of York, UK.
4. Box, G. E. P., 1979. Robustness in the Strategy of Scientific Model Building. In R.L.Launer & G. N. Wilkinson, (Eds.) Robustness in Statistics New York: Academic Press, 201-236.
5. Burnham, K. P. 2004. Multimodel Inference: Understanding AIC and BIC in Model Selection. Colorado Cooperative F&W Research Unit, Coloroda State University, Amsterdam Workshop on model selection, USA.
6. Carlin, B., Chib, S., 1995. Bayesian Model Choice via Markov Chain Monte Carlo Methods. J. Royal Statist. Society Series B, 57(3), 473-484.
7. Celeux, G., Forbes, F., Robert, C. P., Titterington, D. M., 2006. Deviance Information Criteria for Missing Data Models. Bayesian Analysis, 4, 651-674.
8. Da Silva, S. A., Melo, L. L. M., Ehlers, R., 2004. Spatial Analysis of Incidence Rates: A Bayesian Approach. Biostatistics, 1-17.

9. Dempster, A.P., 1974. The Direct Use of Likelihood for Significance Testing in Proceedings of Conference on Foundational Questions in Statistical Inference. University of Aarhus, 335-352.
10. Freeman, P.R., 1976. A Bayesian Analysis of the Megalithic Yard. J.R. Statist.Soc.A, 139, 20-55.
11. Gelfand, A.E., Dey, D.K.,Chang, H., 1992. Model Determination Using Predictive Distributions with Implementation via Sampling-based Methods (with discussion). In Bayesian Statistics, Oxford University Press, 4, 147-167.
12. Gelfand, A.E., Dey, D.K., 1994. Bayesian Model Choice: Asymptotics and Exact Calculations. Journal Royal Statistics Soc.B., 56.
13. Gilks, W.R., Richardson, S., Spiegelhalter, D.J., 1996. Markov Chain Monte Carlo in Practice. Chapman&Hall / CRC, London.
14. Gelman, A., Carlin, J.B., Stern, H.S., Rubin, D.B., 2004. Bayesian Data Analysis. Second Edition, Chapman and Hall / CRC, Boca Raton, FL.
15. Hall, B., 2012. Bayesian Inference. Statisticat, http://www.statisticat.com/laplacesdemon.html
16. Han, C., Carlin, B.P., 2001. MCMC Methods for Computing Bayes Factors: A Comparative Review. Journal of the American Statistical Association, 96, 1122-1132.
17. Jeffreys, H., 1961. Theory of Probability. Oxford University Press, Oxford, U.K.
18. Kass, R.E., Raftery, A.E., 1995. Bayes Factors. Journal of the American Statistical Association, 90, 773-795.
19. Kass, R.E., Wasserman, L., 1995. A Reference Bayesian Test for Nested Hypotheses and Its Relationship to the Schwarz Criterion. Journal of the American Statistical Association, 90(431), 928-934.
20. Kendall, D.G., 1974. Hunting Quanta. Phil. Trans. R. Soc. A, 276, 231-266.
21. Martino, S., 2007. Recent Methods for Bayesian Model Comparison. Department of Mathematical Science, NTNU.
22. Mccullogh, R.J., 2006. An Introduction to the DIC Index. University of Maryland.
23. Raftery, A.E.,1986. Choosing Modeles for Cross-Classifications. American Sociological Review, 51, 145-146.
24. Raftery, A. E., 1995. Bayesian Model Selection in Social Research. Sociological Methodology, Marsden, P. V. Cambridge, Mass., Blackwells, 111-196.
25. Rosenkranz, S. L., Raftery, A. E., 1994. Covariate Selection in Hierarchical Models of Hospital Admission Counts: A Bayes Factor Approach No.268 Department of Statistics, GN 22 University of Washington Seattle, Washington, 98195 USA.
26. Schwarz, G., 1978. Estimating the Dimension of a Model. Annals of Statistics, 6(2), 461-464.
27. Sinharay, S., Stern, H. S., 2002. On the Sensitivity of Bayes Factors to the Prior Distributions. The American Statistician, 56(3), 196-201.
28. Spiegelhalter, D. J., Bull, K., 1997. Tutorial in Biostatistics Survival Analysis in Observational Studies. Statistics in Medicine, 16(9), 1041-1074.
29. Spiegelhalter, D. J., Best, N. G., Carlin, B. P., Van der Linde, A., 2002. Bayesian Measures of Model Complexity and Fit. Journal of the Royal Statistical Society, Series B, Methodological, 64(4), 583-616.
30. Spiegelhalter, D. J., 2006a. Two Brief Topics on Modelling with WinBUGS. MRC Biostatistics Unit, Cambridge.
31. Spiegelhalter, D. J., 2006b. Some DIC Slides. MRC Biostatistics Unit, Cambridge.
32. Thom, A., 1955. A Statistical Examination of the Megalithic Sites in Britain. J. R. Statist. Soc. A., 118, 275-295.
33. Thom, A., 1967. Megalithic Sites in Britain. Clarendon Press, Oxford.
34. Ucal, M.Ş.,2006. Ekonometrik Model Seçim Kriterleri Üzerine Bir İnceleme. C.Ü.İktisadi ve İdari Bilimler Dergisi, 7(2), 41-57.
35. Ward, E. J., 2008. A Review and Comparison of Four Commonly Used Bayesian and Maximum Likelihood Selection Tools. Ecological Modelling, 211, 1-10.
36. Wilberg, M. J., Bence, J. R., 2008. Performance of Deviance Information Criterion Model Selection in Statistical Catch-at-age Analysis. Fisheries Research, 93, 212-221.