Araştırma Makalesi
PDF Zotero Mendeley EndNote BibTex Kaynak Göster

Covariate Balance as a Quality Indicator for Propensity Score Analysis

Yıl 2021, Cilt 12, Sayı 4, 374 - 387, 29.12.2021
https://doi.org/10.21031/epod.993571

Öz

Propensity score analysis, such as propensity score matching and propensity score weighting, is becoming increasingly popular in educational research. When a propensity score analysis is conducted, examining the covariate balance is considered to be crucial to justify the quality of the analysis results. However, it has been pointed out that solely considering how covariates balance after matching may not be enough for justifying the quality of the propensity score analysis results. Suitable covariate balance may still yield biased estimates of treatment effects. The current study aimed to systematically demonstrate this problem by a series of simulation studies. As a result, it was revealed that a good covariate balance on the mean and/or the variance does not guarantee reduced bias on an estimated treatment effect. It was also found that estimation of the treatment effect can be unbiased to some degree, even with a lack of balance under specific conditions. 

Kaynakça

  • Austin, P. C. (2009). Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Statistics in Medicine, 28(25), 3083–3107. https://doi.org/10.1002/sim.3697
  • Belitser, S. V., Martens, E. P., Pestman, W. R., Groenwold, R. H. H., Boer, A. de, & Klungel, E. H. (2011). Measuring balance and model selection in propensity score methods. Pharmacoepidemiology and Drug Safety, 20(11), 1115–1129. https://doi.org/10.1002/pds.2188
  • Bhattacharya, J., & Vogt, W. B. (2007). Do instrumental variables belong in propensity scores? Cambridge, MA: National Bureau of Economic Research (NBER) Working Paper Series No. 343.
  • Brookhart, M. A., Schneeweiss, S., Rothman, K. J., Glynn, R. J., Avorn, J., & Sturmer, T. (2006). Variable selection for propensity score models. American Journal of Epidemiology, 163(12), 1149–1156. https://doi.org/10.1093/aje/kwj149
  • Cannas, M., & Arpino, B. (2019). A comparison of machine learning algorithms and covariate balance measures for propensity score matching and weighting. Biometrical Journal, 61(4), 1049–1072. https://doi.org/10.1002/bimj.201800132
  • Fong, C., Ratkovic, M., & Imai, K. (2019). CBPS: Covariate balancing propensity score. Retrieved from https://CRAN.R-project.org/package=CBPS
  • Greifer, N. (2019). Cobalt: Covariate balance tables and plots. Retrieved from https://CRAN.R-project.org/package=cobalt
  • Guo, S., & Fraser, M. W. (2015). Propensity score analysis: Statistical methods and applications (ed. 2). Thousand Oaks, CA: Sage.
  • Hirano, K., Imbens, G. W., & Ridder, G. (2003). Efficient estimation of average treatment effects using the estimated propensity score. Econometrica, 71(4), 1161–1189. https://doi.org/10.1111/1468-0262.00442
  • Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2008). MatchIt: Nonparametric preprocessing for parametric causal inference. Journal of Statistical Software, 42(8), 1–28. https://doi.org/10.18637/jss.v042.i08
  • Hong, H., Aaby, D. A., Siddique, J., & Stuart, E. A. (2018). Propensity score-based estimators with multiple error-prone covariates. American Journal of Epidemiology, 188, 222–230. https://doi.org/10.1093/aje/kwy210
  • Imai, K., & Ratkovic, M. (2014). Covariate balancing propensity score. Journal of the Royal Statistical Society, 76, 243–263.
  • Kainz, K., Greifer, N., Givens, A., Swietek, K., Lombardi, B. M., Zietz, S., & Kohn, J. L. (2017). Improving causal inference: Recommendations for covariate selection and balance in propensity score methods. Journal of the Society for Social Work and Research, 8, 2334–2351. https://doi.org/10.1086/sim.3782
  • Lee, B. K., Lessler, J., & Stuart, E. A. (2010). Improving propensity score weighting using machine learning. Statistics in Medicine, 29, 337–346. https://doi.org/10.1002/sim.3782
  • McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods, 9(4), 403–425. https://doi.org/10.1037/1082-989X.9.4.403
  • Myers, J. A., Rassen, J. A., Gagne, J. J., Huybrechts, K. F., Schneeweiss, S., Rothman, K. J., … Glynn, R. J. (2011). Effects of adjusting for instrumental variables on bias and precision of effect estimates. American Journal of Epidemiology, 174(11), 1213–1222. https://doi.org/10.1093/aje/kwr364
  • Patrick, A. R., Schneeweiss, S., Brookhart, M. A., Glynn, R. J., Rothman, K. J., Avorn, J., & Sturmer, T. (2011). The implications of propensity score variable selection strategies in pharmacoepidemiology: An empirical illustration. Pharmacoepidemiology and Drug Safety, 20(6), 551–559. https://doi.org/10.1002/pds.2098
  • R Core Team. (2018). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/
  • Ridgeway, G., McCaffrey, D., Morral, A., Griffin, B. A., & Burgette, L. (2017). Twang: Toolkit for weighting and analysis of nonequivalent groups. Retrieved from https://CRAN.R-project.org/package=twang
  • Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55. https://doi.org/10.1093/biomet/70.1.41
  • Rubin, D. B. (2001). Using propensity scores to help design observational studies: application to the tobacco litigation. Health Services and Outcomes Research Methodology, 2(3), 169-188. https://doi.org/10.1023/A:1020363010465
  • Rubin, D. B. (2007). The design versus the analysis of observational studies for causal effects: Parallels with the design of randomized trials. Statistics in Medicine, 26(1), 20–36. http://dx.doi .org/10.1002/sim.2739
  • Setoguchi, S., S amd Schneeweiss, Brookhart, M. A., Glynn, R. J., & Cook, E. F. (2008). Evaluation uses of data mining techniques in propensity score estimation: A simulation study. Pharmacoepidemiology & Drug Safety, 17(6), 546–555. https://doi.org/10.1002/pds.1555
  • Steiner, P. M., Cook, T. D., Shadish, W. R., & Clark, M. H. (2010). The importance of covariate selection in controlling for selectin bias in observational studies. Psychological Methods, 15(3), 250–267. https://doi.org/10.1037/a0018719
  • Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25(1), 1–21. https://doi.org/10.1214/09-STS313
  • Stuart, E. A, Lee, B. K., & Leacy, F. P. (2013). Prognostic score–based balance measures for propensity score methods in comparative effectiveness research. Journal of Clinical Epidemiology, 66(8), S84–S90. https://doi.org/10.1016/j.jclinepi.2013.01.013
  • Westreich, D., Lessler, J., & Funk, M. J. (2010). Propensity score estimation: Machine learning and classification methods as alternatives to logistic regression. Journal of Clinical Epidemiology, 63(8), 826–833. https://doi.org/10.1016/j.jclinepi.2009.11.020
  • What Works Clearinghouse, Institute of Education Sciences, U.S. Department of Education. (2017). What works clearinghouse: Procedures and standards handbook (version 4.0). Retrieved from http://whatworks.ed.gov

Yıl 2021, Cilt 12, Sayı 4, 374 - 387, 29.12.2021
https://doi.org/10.21031/epod.993571

Öz

Kaynakça

  • Austin, P. C. (2009). Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Statistics in Medicine, 28(25), 3083–3107. https://doi.org/10.1002/sim.3697
  • Belitser, S. V., Martens, E. P., Pestman, W. R., Groenwold, R. H. H., Boer, A. de, & Klungel, E. H. (2011). Measuring balance and model selection in propensity score methods. Pharmacoepidemiology and Drug Safety, 20(11), 1115–1129. https://doi.org/10.1002/pds.2188
  • Bhattacharya, J., & Vogt, W. B. (2007). Do instrumental variables belong in propensity scores? Cambridge, MA: National Bureau of Economic Research (NBER) Working Paper Series No. 343.
  • Brookhart, M. A., Schneeweiss, S., Rothman, K. J., Glynn, R. J., Avorn, J., & Sturmer, T. (2006). Variable selection for propensity score models. American Journal of Epidemiology, 163(12), 1149–1156. https://doi.org/10.1093/aje/kwj149
  • Cannas, M., & Arpino, B. (2019). A comparison of machine learning algorithms and covariate balance measures for propensity score matching and weighting. Biometrical Journal, 61(4), 1049–1072. https://doi.org/10.1002/bimj.201800132
  • Fong, C., Ratkovic, M., & Imai, K. (2019). CBPS: Covariate balancing propensity score. Retrieved from https://CRAN.R-project.org/package=CBPS
  • Greifer, N. (2019). Cobalt: Covariate balance tables and plots. Retrieved from https://CRAN.R-project.org/package=cobalt
  • Guo, S., & Fraser, M. W. (2015). Propensity score analysis: Statistical methods and applications (ed. 2). Thousand Oaks, CA: Sage.
  • Hirano, K., Imbens, G. W., & Ridder, G. (2003). Efficient estimation of average treatment effects using the estimated propensity score. Econometrica, 71(4), 1161–1189. https://doi.org/10.1111/1468-0262.00442
  • Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2008). MatchIt: Nonparametric preprocessing for parametric causal inference. Journal of Statistical Software, 42(8), 1–28. https://doi.org/10.18637/jss.v042.i08
  • Hong, H., Aaby, D. A., Siddique, J., & Stuart, E. A. (2018). Propensity score-based estimators with multiple error-prone covariates. American Journal of Epidemiology, 188, 222–230. https://doi.org/10.1093/aje/kwy210
  • Imai, K., & Ratkovic, M. (2014). Covariate balancing propensity score. Journal of the Royal Statistical Society, 76, 243–263.
  • Kainz, K., Greifer, N., Givens, A., Swietek, K., Lombardi, B. M., Zietz, S., & Kohn, J. L. (2017). Improving causal inference: Recommendations for covariate selection and balance in propensity score methods. Journal of the Society for Social Work and Research, 8, 2334–2351. https://doi.org/10.1086/sim.3782
  • Lee, B. K., Lessler, J., & Stuart, E. A. (2010). Improving propensity score weighting using machine learning. Statistics in Medicine, 29, 337–346. https://doi.org/10.1002/sim.3782
  • McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods, 9(4), 403–425. https://doi.org/10.1037/1082-989X.9.4.403
  • Myers, J. A., Rassen, J. A., Gagne, J. J., Huybrechts, K. F., Schneeweiss, S., Rothman, K. J., … Glynn, R. J. (2011). Effects of adjusting for instrumental variables on bias and precision of effect estimates. American Journal of Epidemiology, 174(11), 1213–1222. https://doi.org/10.1093/aje/kwr364
  • Patrick, A. R., Schneeweiss, S., Brookhart, M. A., Glynn, R. J., Rothman, K. J., Avorn, J., & Sturmer, T. (2011). The implications of propensity score variable selection strategies in pharmacoepidemiology: An empirical illustration. Pharmacoepidemiology and Drug Safety, 20(6), 551–559. https://doi.org/10.1002/pds.2098
  • R Core Team. (2018). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/
  • Ridgeway, G., McCaffrey, D., Morral, A., Griffin, B. A., & Burgette, L. (2017). Twang: Toolkit for weighting and analysis of nonequivalent groups. Retrieved from https://CRAN.R-project.org/package=twang
  • Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55. https://doi.org/10.1093/biomet/70.1.41
  • Rubin, D. B. (2001). Using propensity scores to help design observational studies: application to the tobacco litigation. Health Services and Outcomes Research Methodology, 2(3), 169-188. https://doi.org/10.1023/A:1020363010465
  • Rubin, D. B. (2007). The design versus the analysis of observational studies for causal effects: Parallels with the design of randomized trials. Statistics in Medicine, 26(1), 20–36. http://dx.doi .org/10.1002/sim.2739
  • Setoguchi, S., S amd Schneeweiss, Brookhart, M. A., Glynn, R. J., & Cook, E. F. (2008). Evaluation uses of data mining techniques in propensity score estimation: A simulation study. Pharmacoepidemiology & Drug Safety, 17(6), 546–555. https://doi.org/10.1002/pds.1555
  • Steiner, P. M., Cook, T. D., Shadish, W. R., & Clark, M. H. (2010). The importance of covariate selection in controlling for selectin bias in observational studies. Psychological Methods, 15(3), 250–267. https://doi.org/10.1037/a0018719
  • Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25(1), 1–21. https://doi.org/10.1214/09-STS313
  • Stuart, E. A, Lee, B. K., & Leacy, F. P. (2013). Prognostic score–based balance measures for propensity score methods in comparative effectiveness research. Journal of Clinical Epidemiology, 66(8), S84–S90. https://doi.org/10.1016/j.jclinepi.2013.01.013
  • Westreich, D., Lessler, J., & Funk, M. J. (2010). Propensity score estimation: Machine learning and classification methods as alternatives to logistic regression. Journal of Clinical Epidemiology, 63(8), 826–833. https://doi.org/10.1016/j.jclinepi.2009.11.020
  • What Works Clearinghouse, Institute of Education Sciences, U.S. Department of Education. (2017). What works clearinghouse: Procedures and standards handbook (version 4.0). Retrieved from http://whatworks.ed.gov

Ayrıntılar

Birincil Dil İngilizce
Konular Sosyal
Bölüm Makaleler
Yazarlar

Yusuf KARA (Sorumlu Yazar)
Southern Methodist University
0000-0003-0691-0630
United States


Akihito KAMATA Bu kişi benim
SOUTHERN METHODIST UNIVERSITY
0000-0001-9570-1464
United States


Elisa GALLEGOS Bu kişi benim
SOUTHERN METHODIST UNIVERSITY
0000-0002-7863-536X
United States


Chalie PATARAPİCHAYATHAM Bu kişi benim
SOUTHERN METHODIST UNIVERSITY
0000-0003-4946-2786
United States


Cornelis J. POTGİETER Bu kişi benim
TEXAS CHRISTIAN UNIVERSITY
0000-0002-1995-6817
United States

Yayımlanma Tarihi 29 Aralık 2021
Yayınlandığı Sayı Yıl 2021, Cilt 12, Sayı 4

Kaynak Göster

Bibtex @araştırma makalesi { epod993571, journal = {Journal of Measurement and Evaluation in Education and Psychology}, issn = {1309-6575}, eissn = {1309-6575}, address = {}, publisher = {Eğitimde ve Psikolojide Ölçme ve Değerlendirme Derneği}, year = {2021}, volume = {12}, pages = {374 - 387}, doi = {10.21031/epod.993571}, title = {Covariate Balance as a Quality Indicator for Propensity Score Analysis}, key = {cite}, author = {Kara, Yusuf and Kamata, Akihito and Gallegos, Elisa and Patarapichayatham, Chalie and Potgieter, Cornelis J.} }
APA Kara, Y. , Kamata, A. , Gallegos, E. , Patarapichayatham, C. & Potgieter, C. J. (2021). Covariate Balance as a Quality Indicator for Propensity Score Analysis . Journal of Measurement and Evaluation in Education and Psychology , 12 (4) , 374-387 . DOI: 10.21031/epod.993571
MLA Kara, Y. , Kamata, A. , Gallegos, E. , Patarapichayatham, C. , Potgieter, C. J. "Covariate Balance as a Quality Indicator for Propensity Score Analysis" . Journal of Measurement and Evaluation in Education and Psychology 12 (2021 ): 374-387 <https://dergipark.org.tr/tr/pub/epod/issue/67388/993571>
Chicago Kara, Y. , Kamata, A. , Gallegos, E. , Patarapichayatham, C. , Potgieter, C. J. "Covariate Balance as a Quality Indicator for Propensity Score Analysis". Journal of Measurement and Evaluation in Education and Psychology 12 (2021 ): 374-387
RIS TY - JOUR T1 - Covariate Balance as a Quality Indicator for Propensity Score Analysis AU - Yusuf Kara , Akihito Kamata , Elisa Gallegos , Chalie Patarapichayatham , Cornelis J. Potgieter Y1 - 2021 PY - 2021 N1 - doi: 10.21031/epod.993571 DO - 10.21031/epod.993571 T2 - Journal of Measurement and Evaluation in Education and Psychology JF - Journal JO - JOR SP - 374 EP - 387 VL - 12 IS - 4 SN - 1309-6575-1309-6575 M3 - doi: 10.21031/epod.993571 UR - https://doi.org/10.21031/epod.993571 Y2 - 2021 ER -
EndNote %0 Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi Covariate Balance as a Quality Indicator for Propensity Score Analysis %A Yusuf Kara , Akihito Kamata , Elisa Gallegos , Chalie Patarapichayatham , Cornelis J. Potgieter %T Covariate Balance as a Quality Indicator for Propensity Score Analysis %D 2021 %J Journal of Measurement and Evaluation in Education and Psychology %P 1309-6575-1309-6575 %V 12 %N 4 %R doi: 10.21031/epod.993571 %U 10.21031/epod.993571
ISNAD Kara, Yusuf , Kamata, Akihito , Gallegos, Elisa , Patarapichayatham, Chalie , Potgieter, Cornelis J. . "Covariate Balance as a Quality Indicator for Propensity Score Analysis". Journal of Measurement and Evaluation in Education and Psychology 12 / 4 (Aralık 2021): 374-387 . https://doi.org/10.21031/epod.993571
AMA Kara Y. , Kamata A. , Gallegos E. , Patarapichayatham C. , Potgieter C. J. Covariate Balance as a Quality Indicator for Propensity Score Analysis. JMEEP. 2021; 12(4): 374-387.
Vancouver Kara Y. , Kamata A. , Gallegos E. , Patarapichayatham C. , Potgieter C. J. Covariate Balance as a Quality Indicator for Propensity Score Analysis. Journal of Measurement and Evaluation in Education and Psychology. 2021; 12(4): 374-387.
IEEE Y. Kara , A. Kamata , E. Gallegos , C. Patarapichayatham ve C. J. Potgieter , "Covariate Balance as a Quality Indicator for Propensity Score Analysis", Journal of Measurement and Evaluation in Education and Psychology, c. 12, sayı. 4, ss. 374-387, Ara. 2021, doi:10.21031/epod.993571