REINFORCEMENT LEARNING IN INDIVIDUAL PENSION SYSTEM: THE CASE OF TURKEY

Year 2023, , 111 - 112, 15.01.2024

Yunis Dede Saadettin Haluk Çitçi , Halit Yanıkkaya

https://doi.org/10.17261/Pressacademia.2023.1873

Abstract

Purpose- How individuals make saving/investment decisions and they are subject to learning processes are important questions in economics. Behavioral economics and finance literature tell us that individuals can deviate from bayesian decisions and the personal experiences can be effective in decision making. “Reinforcement learning” provides a framework for individual investors who are weighing on strategies successful and avoid strategies unsuccessful in which they experience. In this study, we investigate the effect of past experiences on individuals' retirement savings/investment decisions and whether individual investors are reinforcement learner. For this purpose, we examine individual contracts in the annual micro panel dataset obtained from the Pension Monitoring Center in Individual Pension System in Turkey.
Methodology- Essentially, we assume that individuals' retirement saving/investment decisions are influenced by returns and variances (represents the risk level) of their avaliable portfolio as well as their time horizon, spending habits, retirement goals, risk tolerance and demographic characteristics. In this context, we estimate a linear model by including returns and variances in order to investigate how much sensitive individual investors are to returns and variances of their portfolio. Moreover, we add lagged returns and variances to our econometric setup to examine whether they are reinforcement learner. After that, we conduct a before-after analysis by looking at the dataset from 3-year window to analyze the impact of the 2013 state subsidy reform on reinforcement learning of individual investors.
Findings- Similar to individuals' age, gender and education level, portfolio returns and variances also have a statistically significant effect on the contributions paid. Increases in portfolio returns affect the contributions paid positively, while increases in portfolio variance affect it negatively. As an indicator of reinforcement learning, respectively, lagged returns and variances have a significant positive and negative effect like the same year returns and variances of individual investors. According to this result, individual investors weigh on successful strategies and avoid unsuccessful strategies they have experienced. Increases in variances and lagged variances of individuals' portfolios have a larger negative effect compared to returns. Additionally, looking at the 3-year window, we report that the reinforcement learning of individual investors has strengthened after the 2013 state subsidy reform.
Conclusion- We show that individual investors in IPS in Turkey exhibit reinforcement learning when making retirement savings/investment decisions. High return or low variance obtained in previous periods causes individuals to pay more contributions paid in the next period. This result reveals that individuals benefit from their past experiences when making logical and optimized decisions based on their avaliable knowledge and expectations. The 25% state subsidy in 2013 caused individual portfolio returns to increase and variances to decrease. With these effects, we report that reinforcement learning has become stronger after 2013.

Keywords

Reinforcement learning, decision-making, pension savings, personal finance, retirement policies

References

• Açıkgöz, E., Uygurtürk, H., & Korkmaz, T. (2015). Analysis of factors affecting growth of pension mutual funds in Turkey. International Journal of Economics and Financial Issues, 5(2), 427-433.
• Camerer, C., & Hua Ho, T. (1999). Experience‐weighted attraction learning in normal form games. Econometrica, 67(4), 827-874.
• Charness, G., & Levin, D. (2003). Bayesian updating vs. reinforcement and affect: A laboratory study. Levine’s Bibliography 666156000000000180, UCLA Department of Economics.
• Chetty, R., Friedman, J. N., Leth-Petersen, S., Nielsen, T. H., & Olsen, T. (2014). Active vs. passive decisions and crowd-out in retirement savings accounts: Evidence from Denmark. The Quarterly Journal of Economics, 129(3), 1141-1219.
• Choi, J. J., Laibson, D., Madrian, B. C., & Metrick, A. (2009). Reinforcement learning and savings behavior. The Journal of Finance, 64(6), 2515-2534.
• Erev, I., & Roth, A. E. (1998). Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria. American Economic Review, 848-881.
• Fama, E. F. (1970). Efficient capital markets: A review of theory and empirical work. The Journal of Finance, 25(2), 383-417.
• Feng, L., & Seasholes, M. S. (2005). Do investor sophistication and trading experience eliminate behavioral biases in financial markets?. Review of Finance, 9(3), 305-351.
• Kaustia, M., & Knüpfer, S. (2008). Do investors overweight personal experience? Evidence from IPO subscriptions. The Journal of Finance, 63(6), 2679-2702.
• Malmendier, U., & Nagel, S. (2007). Depression Babies: Do Macroeconomic Experiences Affect Risk-Taking. manuscript.
• Muller, L. A., Moore Jr, J. H., & Elliott, K. R. (2009). Who ıs likely to opt out of an automatic enrollment plan? Who ıs likely to stay in a study of 401 (k) participation choices. Benefits Quarterly, 25(1), 47-61.
• Peker, I. (2016). Analysing turkish ındividual pension system fees and returns. Journal of Economics Finance and Accounting, 3(1), 41-52.
• Roth, A. E., & Erev, I. (1995). Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term. Games and Economic Behavior, 8(1), 164-212.
• Seru, A., Shumway, T., & Stoffman, N. (2010). Learning by trading. The Review of Financial Studies, 23(2), 705-739.
• SBB. (2019). On Birinci Kalkınma Planı (2019-2023).
• Slembeck, T. (1999). A behavioral approach to learning in economics-towards an economic theory of contingent learning (No. 9905001). University Library of Munich, Germany.
• Sutton, R. S., & Barto, A. G. (1981). Toward a modern theory of adaptive networks: expectation and prediction. Psychological Review, 88(2), 135-142