High Efficient Online Double Q Learning Method Based Distributed MPPT in PV Systems under Varying Meteorological Conditions

Abstract

In photovoltaic systems, which are one of the renewable energy production tools, continuous production of power at maximum levels and with the highest possible efficiency is one of the most important objectives. For this purpose, different maximum power point tracking (MPPT) algorithms are used in photovoltaics (PV) module connections. In the array and string structures, the power generation of PV panels decreases due to mismatch reasons such as partial shading, temperature variations, irregular panel surface pollution, aging, and production differences of PV panels. Since solar panels are made up of series-connected solar cells, if some of the cells produce low current, it reduces the current of all the series-connected cells, causing the entire panel to operate at lower performance. The study proposes tabular online double Q learning with unique reinitialization algorithm for maximization of extraction power from PV panels, achieving fast convergence to the maximum power point and maintaining in transient and/or steady-state conditions of irradiation. Simulation results show that the proposed DQL algorithm achieved the highest power extraction in all five irradiance scenarios, delivering 439 W, 575 W, 778 W, 577 W, and 325 W. Under uniform irradiance, DQL reached a tracking efficiency of 99.9%, outperforming the P&O (89.2%) and INC (96.3%) algorithms. The proposed method was compared with algorithms such as particle perturb & observe (PO) and incremental conductance (INC) under rapidly changing meteorological irradiation conditions, and the success and validity of the proposed method were demonstrated with graphs.

Keywords

• MPPT
• Q Learning
• PV Systems

References

1. [1]. Karmakar, S, Singh, B. 2024. Multi-MPPT 72-Pulse VSC Based High-Power Grid Interfaced Solar PV Plant With Distributed DC-Coupled Battery Energy Storage. IEEE Transactions on Energy Conversion; 39(1): 37–48.
2. [2]. Constantinou, S, Konstantinidis, A, Zeinalipour-Yazti, D. 2023. Green Planning Systems for Self-Consumption of Renewable Energy. IEEE Internet Computing; 27(1): 34–42.
3. [3]. Reisi, AR, Moradi, MH, Jamasb, S. 2013. Classification and comparison of maximum power point tracking techniques for photovoltaic system: A review. Renewable and Sustainable Energy Reviews; 19: 433–443.
4. [4]. Dileep, G, Singh, SN. 2017. An improved particle swarm optimization based maximum power point tracking algorithm for PV system operating under partial shading conditions. Solar Energy; 158: 1006–1015.
5. [5]. Siddique, MAB, Zhao, D, Rehman, AU, Ouahada, K, Hamam, H. 2024. An adapted model predictive control MPPT for validation of optimum GMPP tracking under partial shading conditions. Scientific Reports; 14(1): 1–30.
6. [6]. Motamedi, M. A Modified Perturb and Observe Maximum Power Point Tracker with Proportional Integral Controller for Solar Photovoltaic Applications. In 15th Smart Grid Conference, Shiraz, Iran, 2025, pp 1–5.
7. [7]. Tamilamuthan, R. Design and Implementation of Solar PV MPPT Battery Charging System with Cuk Converter and Perturbed and Observe Algorithm for EV. In IEEE 6th Global Conference on Advanced Technology, Bangalore, India, 2025, pp 1–7.
8. [8]. Jabbar, RI, Mekhilef, S, Mubin, M, Mohammed, KK. 2023. A Modified Perturb and Observe MPPT for a Fast and Accurate Tracking of MPP Under Varying Weather Conditions. IEEE Access; 11: 76166–76176.

9. [9]. Ge, Z, Li, X, Xu, F, Wu, H, Wang, R, Ding, S. 2024. An Improved Distributed Maximum Power Point Tracking Technique in Photovoltaic Systems Based on Reinforcement Learning Algorithm. IEEE Journal of Emerging and Selected Topics in Industrial Electronics; 5(1): 167–178.
10. [10]. Nwachukwu, SE, Folly, KA, Awodele, KO. 2025. A Comparative Study Between Soft Actor-Critic (SAC) and Deep Deterministic Policy Gradient (DDPG) Algorithms for Solar PV MPPT Control Under Partial Shading Conditions. IEEE Access; 13: 71738–71754.
11. [11]. Dash, SK, Nema, S, Nema, RK, Verma, D. 2015. A comprehensive assessment of maximum power point tracking techniques under uniform and non-uniform irradiance and its impact on photovoltaic systems: A review. Journal of Renewable and Sustainable Energy; 7(6): 063113.
12. [12]. Li, H, Yang, D, Su, W, Lu, J, Yu, X. 2019. An Overall Distribution Particle Swarm Optimization MPPT Algorithm for Photovoltaic System under Partial Shading. IEEE Transactions on Industrial Electronics; 66(1): 265–275.
13. [13]. Muriithi, G, Chowdhury, S. 2021. Optimal energy management of a grid-tied solar PV-battery microgrid: A reinforcement learning approach. Energies; 14(9): 1-24.
14. [14]. Rizzo, SA, Scelba, G. 2015. ANN based MPPT method for rapidly variable shading conditions. Applied Energy; 145: 124–132.
15. [15]. Kofinas, P, Dounis, AI, Papadakis, G, Assimakopoulos, MN. 2015. An Intelligent MPPT controller based on direct neural control for partially shaded PV system. Energy and Buildings; 90: 51–64.
16. [16]. El-Helw, HM, Magdy, A, Marei, MI. 2017. A Hybrid Maximum Power Point Tracking Technique for Partially Shaded Photovoltaic Arrays. IEEE Access; 5: 11900–11908.
17. [17]. Zhang, Y, McLoone, S, Cao, W, Qiu, F, Gerada, C. 2017. Power Loss and Thermal Analysis of a MW High-Speed Permanent Magnet Synchronous Machine. IEEE Transactions on Energy Conversion; 32(4): 1468–1478.
18. [18]. Khan, K, Rashid, S, Mansoor, M, Khan, A, Raza, H, Zafar, M. H, Akhtar, N. 2023. Data-driven green energy extraction: Machine learning-based MPPT control with efficient fault detection method for the hybrid PV-TEG system. Energy Reports; 9: 3604–3623.
19. [19]. Shi, Y, Rong, Z. 2022. Analysis of Q-Learning Like Algorithms Through Evolutionary Game Dynamics. IEEE Transactions on Circuits and Systems II: Express Briefs; 69(5): 2463–2467.
20. [20]. Chou, KY, Yang, ST, Yang, CS, Chen, YP. Maximum Power Point Tracking of Photovoltaic System Based on Reinforcement Learning. In IEEE International Conference on Consumer Electronics - Taiwan, Yilan, Taiwan, 2019, pp 1-22.
21. [21]. Shen, Y, Ma, Y, Kang, H, Sun, X, Chen, Q. 2024. Learning and propagation: Evolutionary dynamics in spatial public goods games through combined Q-learning and Fermi rule. Chaos, Solitons and Fractals; 187: 115377.
22. [22]. Panwar, NL, Kaushik, SC, Kothari, S. 2011. Role of renewable energy sources in environmental protection: A review. Renewable and Sustainable Energy Reviews; 15(3): 1513–1524.
23. [23]. Kasper, M, Bortis, D, Kolar, JW. 2014. Classification and comparative evaluation of PV panel-integrated DC-DC converter concepts. IEEE Transactions on Power Electronics; 29(5): 2511–2526.
24. [24]. Sebin Davis, K, Jegathesan, V. Flyback PPC Converter with Artificial Neural Network for Distributed MPPT Application. In 8th International Conference on Advanced Computing and Communication Systems, Coimbatore, India, 2022, pp 1141–1145.
25. [25]. Elmelegi, A, Aly, M, Ahmed, EM, Alhaider, MM. An Efficient Low-Cost Distributed MPPT Method for Energy Harvesting in Grid-Tied Three-Phase PV Power Optimizers. In 21st International Middle East Power Systems Conference, Cairo, Egypt, 2019, pp 1042–1047.
26. [26]. Olalla, C, Clement, D, Rodriguez, M, Maksimovic, D. 2013. Architectures and control of submodule integrated dc-dc converters for photovoltaic applications. IEEE Transactions on Power Electronics; 28(6): 2980–2997.
27. [27]. Bratcu, AI, Munteanu, I, Bacha, S, Picault, D, Raison, B. 2011. Cascaded DC-DC converter photovoltaic systems: Power optimization issues. IEEE Transactions on Industrial Electronics; 58(2): 403–411.
28. [28]. Fu, C, Zhang, L. 2022. A novel method based on tuna swarm algorithm under complex partial shading conditions in PV system. Solar Energy; 248: 28–40.
29. [29]. Ayop, R, Tan, CW. 2018. Design of boost converter based on maximum power point resistance for photovoltaic applications. Solar Energy; 160: 322–335.
30. [30]. Almahamid, F, Grolinger, K. Reinforcement Learning Algorithms: An Overview and Classification. In Canadian Conference on Electrical and Computer Engineering, On, Canada, 2021, pp 1-7.
31. [31]. Caruana, R, Niculescu-Mizil, A. An empirical comparison of supervised learning algorithms. In Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, USA, 2006, pp 161–168.
32. [32]. Hao, B, Du, H, Yan, Z. 2023. A path planning approach for unmanned surface vehicles based on dynamic and fast Q-learning. Ocean Engineering; 270: 113632.
33. [33]. Van Hasselt, H, Guez, A, Silver, D. Deep Reinforcement Learning with Double Q-Learning. In Proceedings of the 30th AAAI Conference on Artificial Intelligence, Phoenix, USA, 2016, pp 2094–2100.
34. [34]. Patil, M, Tambolkar, P, Midlam-Mohler, S. 2025. Optimizing Traffic Routes With Enhanced Double Q-Learning. IET Intelligent Transport Systems; 19: e70002.
35. [35]. Prauzek, M, Konecny, J, Paterova, T. 2023. An Analysis of Double Q-Learning-Based Energy Management Strategies for TEG-Powered IoT Devices. IEEE Internet of Things Journal; 10(21): 18919–18929.