Weather-Aware Deep Reinforcement Learning for Predictive Modeling of Household Energy Dynamics

Enes Bajrami; Petre Lameski

doi:10.26650/d3ai.1741550

Weather-Aware Deep Reinforcement Learning for Predictive Modeling of Household Energy Dynamics

Abstract

This study proposes a weather-aware deep reinforcement learning (DRL) framework for predictive modelling of household energy dynamics. Using a 14-month high-resolution dataset from a residence in Northeast Mexico, the framework integrates detailed meteorological attributes and next-day forecasts to enhance prediction accuracy. Four DRL algorithms were implemented and evaluated for their performance in forecasting household energy consumption: Proximal Policy Optimisation (PPO), Soft Actor-Critic (SAC), Deep Deterministic Policy Gradient (DDPG), and Asynchronous Advantage Actor-Critic (A3C). Exploratory data analysis revealed significant seasonal trends and variability in energy usage patterns. Results show that DDPG and SAC outperform PPO and A3C, achieving the lowest root mean square error (RMSE) and mean absolute error (MAE), with DDPG recording 0.0011 RMSE and 0.0009 MAE. The framework was tested on moderately equipped hardware, demonstrating the practical feasibility of DRL-based energy forecasting systems. This work contributes original visualisations and comparative insights, advancing smart energy management solutions.

Keywords

References

Meenal R., Binu D., Ramya K., Michael P. A., Vinoth Kumar K., Rajasekaran E., and Sangeetha B. 2022. Weather forecasting for renewable energy systems: A review. Archives of Computational Methods in Engineering 29, (2022), 2875–2891. doi: 10.1007/s11831-021-09695-3 google scholar
Hao Zhang, Ming Zhang, Rui Yi, Yaping Liu, Qian-Hua Wen, and Xuan Meng. 2024. Growing importance of micrometeorology in the new power system: Review, analysis, and case study. Energies 17, (2024), 1365. https://doi.org/10.3390/en17061365 google scholar
George Papaefthymiou and Ken Dragoon. 2016. Towards 100% renewable energy systems: Uncapping power system flexibility. Energy Policy 92, (2016), 69–82. DOI: 10.1016/j.enpol.2016.01.025 google scholar
Martina K. Linnenluecke, Annika Griffiths, and Monika Winn. 2022. Extreme weather events and the critical importance of anticipatory adaptation and organisational resilience in responding to impacts. Business Strategy and the Environment (2022), 17–32. DOI: 10.1002/bse.708 google scholar
James W. Day and Charles Hall. 2016. Global climate change: A warmer and more unpredictable future. In America’s Most Sustainable Cities and Regions: Surviving the 21st Century Megatrends. Springer, 137–166. doi: 10.1007/978-1-4939-3243-6 google scholar
Peter Bauer, Alan Thorpe, and Gilbert Brunet. 2015. The quiet revolution of numerical weather prediction. Nature (2015), 47–55. doi: 10.1038/nature14956 google scholar
Anthony Lupo, William Kininmonth, J. Scott Armstrong and Kesten Green. 2013. Global climate models and their limitations. In Climate Change Reconsidered II: Physical Science. google scholar
Xiaoli Ren, Xiaoyong Li, Kaijun Ren, Junqiang Song, Zichen Xu, Kefeng Deng, Xiang Wang, Deep Learning-Based Weather Prediction: A Survey, Big Data Research, Volume 23, 2021, 100178, ISSN 2214-5796, https://doi.org/10.1016/j.bdr.2020.100178. google scholar

Markus Reichstein, Gustau Camps-Valls, Bjorn Stevens, Martin Jung, Joachim Denzler, Nuno Carvalhais, and Prabhat. 2019. Deep learning and process understanding for data-driven Earth system science. Nature (2019), 195–204. doi: 10.1038/s41586-019-0912-1 google scholar
Lam et al. 2023. Learning skilful medium-range global weather forecasting. Science 382, (2023), 1416–1421. https://doi.org/10.1126/science.adi2336 google scholar
Sunil S., Aparna B., Anik S., Tusar K. H., Biswajeet P., and Abdullah A. 2024. Deep learning algorithms-based landslide vulnerability modelling in highly landslide-prone areas of Tamil Nadu, India. Geoscience Journal 28, (2024), 1013–1038. doi: 10.1007/s12303-024-0044-y google scholar
Kaifeng Bi, Lixin Xie, Hongyi Zhang, Xin Chen, Xiaotian Gu, and Qi Tian. 2023. Accurate medium-range global weather forecasting using 3D neural networks. Nature 619, (2023), 533–538. doi: 10.1038/s41586-023-06185-3 google scholar
Jesse Willard, Xiaowei Jia, Shaobo Xu, Michael Steinbach, and Vipin Kumar. 2022. Integrating scientific knowledge with machine learning for engineering and environmental systems. ACM Computing Surveys (2022), 37–66. https://doi.org/10.1145/3514228 google scholar
David Hansen, Daniel C. Maddix, Soheil Alizadeh, Gaurav Gupta, and Michael W. Mahoney. 2024. Learning physical models that can respect conservation laws. Physica D: Nonlinear Phenomena 457, (2024), 133952. google scholar
https://doi.org/10.1016/j.physd.2023.133952 google scholar
David Lazer, Ryan Kennedy, Gary King, and Alessandro Vespignani. 2014. The parable of Google Flu: Traps in big data analysis. Science (2014), 1203–1205. https://doi.org/10.1126/science.1248506 google scholar
Brian E. Rose. 2018. CLIMLAB: A Python toolkit for interactive, process-oriented climate modelling. Journal of Open Source Software (2018). doi: 10.21105/joss.00659 google scholar
Ronald J. Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8, 3 (1992), 229–256. doi: 10.1007/BF00992696 google scholar
Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015. Continuous control with deep reinforcement learning. arXiv:1509.02971 [cs. LG]. https://arxiv.org/abs/1509.02971 google scholar
Paulo Vitor B. Ramos et al. 2023. Residential energy consumption forecasting using deep learning models. Applied Energy 350, (2023), 121705. doi: 10.1016/j.apenergy.2023.121705 google scholar
Eoin McKenna, Alan Peplinski et al. 2024. A machine learning framework to estimate residential electricity demand based on smart metre electricity, climate, building characteristics, and socioeconomic datasets. Applied Energy 357, (2024), 122413. 10.1016/j.apenergy.2023.122413 google scholar
Das et al. 2024. Machine learning-based energy forecasting for residential communities. In Proceedings of the IEEE 9th International Conference for Convergence in Technology (I2CT), 1–4. https://doi.org/10.1109/I2CT61223.2024.10544342 google scholar
H. M. Shakeel, S. Iram, R. Hill, H. M. Athar Farid, A. Sheikh-Akbari, and F. Saleem. 2025. A machine learning-based intelligent framework for predicting energy efficiency in next-generation residential buildings. Buildings 8, 5 (2025), 1275. doi: 10.3390/buildings15081275 google scholar
Enes Bajrami, Andrea Kulakov, Eftim Zdravevski, and Petre Lameski. 2025. A comparative analysis of PPO and SAC algorithms for energy optimisation with country-level energy consumption insights. IFAC Journal of Systems and Control 34, (2025), 100344. https://doi.org/10.1016/j.ifacsc.2025.100344 google scholar
Andrea Franzoso, Gabriele Fambri, and Marco Badami. 2025. Deep reinforcement learning as a tool for the analysis and optimisation of energy flows in multi-energy systems. Energy Conversion and Management 341, (2025), 120095. google scholar
https://doi.org/10.1016/j.enconman.2025.120095 google scholar
Qi Shuai, Yin Yin, Shuang Huang, and Cheng Chen. 2025. Deep reinforcement learning-based real-time energy management for an integrated electric–thermal energy system. Sustainability 17, 2 (2025), 407. google scholar
https://doi.org/10.3390/su17020407 google scholar
Pauline Béraud, Margaux Rioux, Michel Babany, Philippe de La Chevasnerie, Damien Theis, Giacomo Teodori, Chloé Pinguet, Romane Rigaud, and François Leclerc. 2024. An explainable machine learning approach for energy forecasting at the household level. arXiv:2410.14416 [cs. LG]. https://arxiv.org/abs/2410.14416 google scholar
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal Policy Optimisation Algorithms. arXiv:1707.06347 [cs. LG]. https://arxiv.org/abs/1707.06347 google scholar
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal Policy Optimisation Algorithms. arXiv:1707.06347 [cs. LG]. https://arxiv.org/abs/1707.06347 google scholar
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. 2018. Soft actor-critical: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic actor. arXiv:1801.01290 [cs. LG]. https://arxiv.org/abs/1801.01290 google scholar
Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2019. Continuous control with deep reinforcement learning. arXiv:1509.02971v6 [cs. LG]. https://arxiv.org/abs/1509.02971 google scholar
Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. arXiv:1602.01783 [cs. LG]. https://arxiv.org/abs/1602.01783 google scholar
Baldemar Aguirre-Fraire, Jessica Beltrán, and Valeria Soto. 2024. Household energy consumption enriched with weather data in the northeast of Mexico. Dataset. https://doi.org/10.17632/tvhygj8rgg.1. google scholar
Siyu Wang et al. 2024. Comparison of deep reinforcement learning-based energy management strategies for fuel cell vehicles considering economics, durability, and adaptability. Energy 307, (2024), 132771. google scholar
https://doi.org/10.1016/j.energy.2024.132771google scholar
Cristian Vidal-Silva et al. 2025. A comparative study of a deep reinforcement learning solution and alternative deep learning models for wildfire prediction. Applied Sciences 15, 7 (2025), 3990. doi: 10.3390/app15073990 google scholar
Tao et al. 2020. Study deep reinforcement learning techniques for building energy consumption forecasting. Energy and Buildings (2020), 109675. doi: 10.1016/j.enbuild.2019.109675 google scholar

Details

Primary Language

English

Subjects

Artificial Intelligence (Other)

Journal Section

Research Article

Authors

Enes Bajrami ^*
0009-0005-7960-3959
Macedonia

Petre Lameski
0000-0002-5336-1796
Macedonia

Publication Date

January 30, 2026

Submission Date

July 13, 2025

Acceptance Date

December 30, 2025

Published in Issue

Year 2026 Volume: 2 Number: 1

DOI

https://doi.org/10.26650/d3ai.1741550

IZ

https://izlik.org/JA33MX94SA

Cite

RIS / Bibtex

APA

Bajrami, E., & Lameski, P. (2026). Weather-Aware Deep Reinforcement Learning for Predictive Modeling of Household Energy Dynamics. Journal of Data Analytics and Artificial Intelligence Applications, 2(1), 17-44. https://doi.org/10.26650/d3ai.1741550

AMA

1.Bajrami E, Lameski P. Weather-Aware Deep Reinforcement Learning for Predictive Modeling of Household Energy Dynamics. Journal of Data Analytics and Artificial Intelligence Applications. 2026;2(1):17-44. doi:10.26650/d3ai.1741550

Chicago

Bajrami, Enes, and Petre Lameski. 2026. “Weather-Aware Deep Reinforcement Learning for Predictive Modeling of Household Energy Dynamics”. Journal of Data Analytics and Artificial Intelligence Applications 2 (1): 17-44. https://doi.org/10.26650/d3ai.1741550.

EndNote

Bajrami E, Lameski P (January 1, 2026) Weather-Aware Deep Reinforcement Learning for Predictive Modeling of Household Energy Dynamics. Journal of Data Analytics and Artificial Intelligence Applications 2 1 17–44.

IEEE

[1]E. Bajrami and P. Lameski, “Weather-Aware Deep Reinforcement Learning for Predictive Modeling of Household Energy Dynamics”, Journal of Data Analytics and Artificial Intelligence Applications, vol. 2, no. 1, pp. 17–44, Jan. 2026, doi: 10.26650/d3ai.1741550.

ISNAD

Bajrami, Enes - Lameski, Petre. “Weather-Aware Deep Reinforcement Learning for Predictive Modeling of Household Energy Dynamics”. Journal of Data Analytics and Artificial Intelligence Applications 2/1 (January 1, 2026): 17-44. https://doi.org/10.26650/d3ai.1741550.

JAMA

1.Bajrami E, Lameski P. Weather-Aware Deep Reinforcement Learning for Predictive Modeling of Household Energy Dynamics. Journal of Data Analytics and Artificial Intelligence Applications. 2026;2:17–44.

MLA

Bajrami, Enes, and Petre Lameski. “Weather-Aware Deep Reinforcement Learning for Predictive Modeling of Household Energy Dynamics”. Journal of Data Analytics and Artificial Intelligence Applications, vol. 2, no. 1, Jan. 2026, pp. 17-44, doi:10.26650/d3ai.1741550.

Vancouver

1.Enes Bajrami, Petre Lameski. Weather-Aware Deep Reinforcement Learning for Predictive Modeling of Household Energy Dynamics. Journal of Data Analytics and Artificial Intelligence Applications. 2026 Jan. 1;2(1):17-44. doi:10.26650/d3ai.1741550