On the Interaction between Econometrics and Machine Learning

Year 2022, Volume: 11 Issue: 2, 107 - 149, 30.09.2022

Hüseyin Taştan

Abstract

Three recent trends in economics stand out: (1) the rise of empiricism, (2) general acceptance of the causal inference framework in econometric analysis, and (3) increasing adoption of the machine learning approach and its greater interaction with econometrics. This study aims to discuss the evolution of econometrics over these main trends and to understand the nature of the interaction between econometrics and machine learning. In its relatively short history, econometrics has made important breakthroughs and has also experienced methodological and paradigmatic shifts. More recently, the main purpose of econometric analysis is to develop methods for unbiased/consistent and efficient estimation of causal economic relations. On the other hand, (supervised) machine learning aims to develop algorithms for solving estimation/prediction and classification problems. The machine learning approach generally provides more successful predictions since it can exploit the bias-variance trade-off optimally compared to the econometric approach where unbiased/consistent and asymptotically efficient estimation is the principal aim. It can be said that the interaction between econometrics and machine learning is shaped by the phenomenal predictive success of machine learning algorithms. This ongoing interaction has resulted in the development of new econometric methods for causal inference and the improvement of the existing ones.

Keywords

Econometrics, machine learning, causal inference, econometric modelling.

Ekonometri ve Makine Öğrenmesi Etkileşimi Üzerine

Abstract

İktisatta son dönemde üç eğilim göze çarpmaktadır: (1) ampirizmin yükselişi, (2) ekonometrik analizde deney bazlı nedensel çıkarım çerçevesinin genel kabul görmesi, ve (3) makine öğrenmesi yaklaşımının yaygınlaşması ve ekonometri ile daha fazla etkileşimi. Bu çalışma bu ana eğilimler üzerinden ekonometrinin gelişim sürecini tartışmayı ve makine öğrenmesi ile ekonometri arasındaki etkileşimin doğasını anlamayı amaçlamaktadır. Ekonometri göreli kısa tarihi içinde önemli atılımlar gerçekleştirmiş ve yöntemsel/paradigmatik kırılmalar yaşamıştır. Günümüzde ekonometrik analizin temel amacı nedensel ekonomik ilişkilerin sapmasız/tutarlı ve etkin tahminine ilişkin yöntemler geliştirmektir. Makine öğrenmesinde (gözetimli) ise amaç kestirim/öngörü ve sınıflandırma problemlerinin çözümüne yönelik algoritmalar geliştirilmesidir. Ekonometrik analizdeki gibi sapmasız/tutarlı ve asimptotik etkin tahmin yapabilmek geri planda olduğu için makine öğrenmesi problemleri daha başarılı kestirim modelleri verebilmektedir. Ekonometri ile makine öğrenmesi etkileşiminin özellikle bu olağanüstü kestirim başarısı üzerinden şekillendiği söylenebilir. Devam eden bu etkileşim, nedensel çıkarım için yeni ekonometrik yöntemlerin geliştirilmesi ve mevcut olanların iyileştirilmesi ile sonuçlanmıştır.

Keywords

Ekonometri, makine öğrenmesi, nedensel çıkarım, ekonometrik modelleme.

References

• Abadie, A. (2021). “Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects”, Journal of Economic Literature, 59 (2), 391-425.
• Acemoglu, D., Johnson, S., ve Robinson, J.A. (2001). “The Colonial Origins of Comparative Development: An Empirical Investigation”. American Economic Review, 91 (5): 1369-1401.
• Alpaydın, E. (2018). Yapay Öğrenme, 4. Baskı (Ethem Alpaydın, Introduction to Machine Learning, 2. baskıdan çeviri), Boğaziçi Üniversitesi Yayınevi, İstanbul.
• Angrist, J. D. ve Pischke, J.-S. (2009). Mostly harmless econometrics: An Empiricist's Companion, Princeton University Press.
• Angrist, J. D. ve Pischke, J.-S. (2010). “The credibility revolution in empirical economics: How better research design is taking the con out of econometrics”, Journal of Economic Perspectives, 24(2), 3-30.
• Angrist, J., Azoulay, P., Ellison, G., Hill, R., ve Feng Lu, S. (2017). “Economic Research Evolves: Fields and Styles.” American Economic Review, 107 (5): 293-97.
• Arlot, S., ve Celisse, A. (2010). “A survey of cross-validation procedures for model selection”, Statistics Surveys, 4, 40–79.
• Assenmacher, K. (2017). “Bridging the gap between structural VAR and DSGE models”, in DSGE Models in the Conduct of Policy: Use as Intended, ed. Gürkaynak, R. and C. Tille. VoxEU.
• Athey, S. (2018). The Impact of Machine Learning on Economics, Agrawal A., Gans, J. ve Goldfarb, A. (Der.), The Economics of Artificial Intelligence: An Agenda. içinde (s. 507-547). University of Chicago Press. URL: http://www.nber.org/chapters/c14009
• Athey, S. ve Imbens, G. (2016), “Recursive partitioning for heterogeneous causal effects”, Proceedings of the National Academy of Sciences, 113(27), 7353-7360, URL: https://www.pnas.org/doi/10.1073/pnas.1510489113 Athey, S. ve G. W. Imbens. (2017), “The State of Applied Econometrics: Causality and Policy Evaluation”, Journal of Economic Perspectives, 31 (2), 3-32.
• Athey, S. ve Imbens, G. W. (2019), “Machine learning methods that economists should know about”, Annual Review of Economics, 11(1), 685-725.
• Athey, S., Tibshirani, J. & Wager, S. (2019), “Generalized random forests”, The Annals of Statistics, 47(2), 1148-1178.
• Athey, S., Bayati, M., Doudchenko, N., Imbens G., ve Khosravi K. (2021). “Matrix Completion Methods for Causal Panel Data Models”, Journal of the American Statistical Association (https://doi.org/10.1080/01621459.2021.1891924).
• Babii, A., Ghysels, E., ve Striaukas, J. (2022), “Machine Learning Time Series Regressions With an Application to Nowcasting”, Journal of Business & Economic Statistics, 40(3), 1094-1106.
• Bai, J. ve Ng, S. (2009), “Boosting diffusion indices”, Journal of Applied Econometrics, 24(4), 607-629.
• Belloni, A., Chernozukov, V. ve Hansen, C. (2014). “Inference on Treatment Effects after Selection among High-Dimensional Controls”. The Review of Economic Studies, 81(2 (287)), 608-650.
• Bennett J., & Lanning S. (2007). The Netflix prize. In Proceedings of KDD Cup and Workshop 2007, p. 35. New York: ACM.
• Bergmeir, C., Hyndman, R.J. ve Koo, B. (2018), “A note on the validity of cross-validation for evaluating autoregressive time series prediction”, Computational Statistics & Data Analysis, 120, 70-83.
• Blanchard, O., (2017), “Do DSGE models have a future?”, in DSGE Models in the Conduct of Policy: Use as Intended, ed. Gürkaynak, R. and C. Tille. VoxEU.
• Breiman, L., (1996), “Bagging Predictors”, Machine Learning, 24, 123-140.
• Breiman, L., (2001a), “Random Forests”, Machine Learning, 45, 5-32.
• Breiman, L., (2001b), “Statistical Modeling: The Two Cultures”, Statistical Science, 16(3), 199-215.
• Breiman, L., J. H. Friedman, R. A. Olshen, ve C. J. Stone (1984). Classification and Regression Trees. Wadsworth, CRC Press reprint.
• Brodersen, K.H., Gallusser, F., Koehler, J., Remy, N., ve Scott, S.L. (2015), “Inferring causal impact using Bayesian structural time-series models”, Annals of Applied Statistics, 9 (1), 247 - 274.
• Callaway, B., ve Sant’Anna, P.H.C. (2021), “Difference-in-differences with multiple time periods”, Journal of Econometrics, 225, 200-230.
• Card, D., ve Krueger, A. (1994), “Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania”, American Economic Review, 84(4), 772-793.
• Chernozhukov, V., Hansen, C. ve Spindler, M. (2016), “hdm: High-Dimensional Metrics”, The R Journal, 8(2), 185-199. https://doi.org/10.32614/RJ-2016-040
• Chernozhukov, V., D. Chetverikov, M. Demirer, E. Duflo, C. Hansen, W. Newey, J. Robins, (2018), “Double/debiased Machine Learning for Treatment and Structural parameters”, The Econometrics Journal, 21(1), C1-C68.
• Dauphin, Jean-François, Kamil Dybczak, Morgan Maneely, Marzie Taheri Sanjani, Nujin Suphaphiphat, Yifei Wang, ve Hanqi Zhang, (2022), “Nowcasting GDP: A Scalable Approach Using DFM, Machine Learning and Novel Data, Applied to European Economies”, International Monetary Fund Working Paper Series, WP/22/52.
• Doğruel, A. Suut, ve F. Doğruel (2015), “İktisatta Yayın Yapma Telaşı (Publication Panic in Economics)”, Ekonomi-tek, 4(3), 69-88.
• Einav, L. ve J. Levin, (2014). “The Data Revolution and Economic Analysis”, Innovation Policy and the Economy, 14, 1-24.
• Epstein, Roy J., (1987), A History of Econometrics, Elsevier North Holland.
• Friedman, J. H., Hastie, T. ve Tibshirani, R., (2000), “Additive logistic regression: a statistical view of boosting, Annals of Statistics, 28(2), 337–407.
• Friedman, J.H., (2001), “Greedy function approximation: A gradient boosting machine”, Annals of Statistics, 29(5), 1189-1232.
• Frisch, R., (1970), “From Utopian Theory to Practical Applications: The Case of Econometrics”, Lecture to the memory of Alfred Nobel, June 17, 1970, https://www.nobelprize.org/uploads/2018/06/frisch-lecture-1.pdf
• Greene, W., (2018), Econometric Analysis, 8th ed., Pearson, New York, NY.
• Gürkaynak, R. S., ve C. Tille, (2017), DSGE Models in the Conduct of Policy: Use as Intended, A EoxEU.org Book, London: CEPR Press. http://voxeu.org/content/dsge-models-conduct-policy-use-intended.
• Haavelmo, Trygve (1944), “The Probability Approach in Econometrics”, Econometrica, 12, Supplement.
• Hamermesh, Daniel S. (2013). “Six Decades of Top Economics Publishing: Who and How?” Journal of Economic Literature, 51 (1), 162-72.
• Hastie, T., R. Tibshirani, ve J. Friedman, (2009), The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed., Springer.
• Hendry, D. F. (1980), “Econometrics-alchemy or science?”, Economica, 47(188), 387-406.
• Huntington-Klein, N., Arenas, A., Beam, E., Bertoni, M., Bloem, J. R., Burli, P., Chen, N., Grieco, P., Ekpe, G., Pugatch, T., Saavedra, M. & Stopnitzky, Y. (2021), “The influence of hidden researcher decisions in applied microeconomics”, Economic Inquiry, 59(3), 944-960.
• James, G., Witten, D., Hastie, T. ve Tibshirani, R. (2021), An Introduction to Statistical Learning, 2 edn, Springer, New York, NY.
• Keynes, J.M., (1939), “Professor Tinbergen’s Method”, The Economic Journal, 49 (195), 558-577.
• Kleinberg, J. J. Ludwig, S. Mullainathan, ve Z. Obermeyer, (2015), “Prediction Policy Problems”, American Economic Review, Papers and Proceedings, 105(5): 491-495. (http://dx.doi.org/10.1257/aer.p20151023)
• Kleinberg, J., H. Lakkaraju, J. Leskovec, J., Ludwig, J., ve S. Mullainathan, (2018a), “Human decisions and machine predictions”, Quarterly Journal of Economics, 133(1), 237-293.
• Kleinberg, J., J. Ludwig, S. Mullainathan, ve A. Rambachan, (2018b), “Algorithmic fairness”, American Economic Review, Papers and Proceedings, 108: 22-27.
• Leamer, E. E. (1983), “Let's take the con out of econometrics”, American Economic Review, 73(1), 31-43.
• Lucas, R. E., (1976), “Econometric Policy Evaluation: A Critique”, CarnegieRochester Conference Series on Public Policy, 1, 19–46
• Mann, H.B. ve A. Wald, (1944), “On the Statistical Treatment of Linear Stochastic Difference Equations”, Econometrica, 11 (3/4), 173-220.
• Masini, R. P., Medeiros, M. C., & Mendes, E. F. (2022), “Machine learning advances for time series forecasting”, Journal of Economic Surveys, 1- 36. https://doi.org/10.1111/joes.12429
• Medeiros, Marcelo C., Gabriel F. R. Vasconcelos, Álvaro Veiga & Eduardo Zilberman (2021), “Forecasting Inflation in a Data-Rich Environment: The Benefits of Machine Learning Methods”, Journal of Business & Economic Statistics, DOI: 10.1080/07350015.2019.1637745
• Molnar, C., (2022), Interpretable Machine Learning: A Guide For Making Black Box Models Explainable, ISBN-13 : ‎ 979-8411463330, https://christophm.github.io/interpretable-ml-book/
• Morgan, M. (1990). The History of Econometric Ideas, (Historical Perspectives on Modern Economics). Cambridge: Cambridge University Press.
• Mullainathan, S. ve Spiess, J. (2017), “Machine learning: An applied econometric approach”, Journal of Economic Perspectives, 31(2), 87-106.
• Phillips, P. C. B. ve Shi, Z. (2021), “Boosting: Why you can use the HP filter”, International Economic Review, 62(2), 521-570.
• Qin, Duo, (1993), The Formation of Econometrics: A Historical Perspective, Oxford University Press.
• Qin, Duo, (2013), A History of Econometrics: The Reformation from the 1970s, Oxford University Press.
• Rambachan, A., J. Kleinberg, J. Ludwig, ve S. Mullainathan, (2020), “An economic perspective on algorithmic fairness”, American Economic Review, Papers and Proceedings, 110, 91-95.
• Richardson, A., Thomas van Florenstein Mulder, ve Tuğrul Vehbi, (2021), Nowcasting GDP using machine-learning algorithms: A real-time assessment, International Journal of Forecasting, 37(2), 941-948.
• Rothman, D., (2022), Hands-On Explainable AI (XAI) with Python, Packt Publishing.
• Schapire RE, ve Freund Y. (2012). Boosting: Foundations and Algorithms. Cambridge,MA: MIT Press
• Stone, M., (1974), Cross-validatory choice and assessment of statistical predictions. J. R. Stat. Soc. Ser. B Stat. Methodol. 36 (2), 111–147.
• Tinbergen, J., (1939), Statistical Testing of Business Cycle Theories: vol. I, A Method and Its Application to Investment Activity, The League of Nations, Geneva.
• Uygur, E., (2006), “Ekonometrinin gelişimi: iktisadın bilim olma çabası”, Türkiye Ekonomi Kurumu Tartışma Metni, 2006/8.
• Varian, H.R. 2014. Big Data: New Tricks for Econometrics. Journal of Economic Perspectives 28 (2): 3–28.
• Wager, S. ve Athey, S. (2018), “Estimation and inference of heterogeneous treatment effects using random forests”, Journal of the American Statistical Association, 113(523), 1228-1242.
• Zou, H. and Hastie, T. (2005), Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67: 301-320.