Weather-Aware Deep Reinforcement Learning for Predictive Modeling of Household Energy Dynamics

Year 2026, Volume: 2 Issue: 1, 17 - 44, 30.01.2026

Enes Bajrami , Petre Lameski

https://doi.org/10.26650/d3ai.1741550

https://izlik.org/JA33MX94SA

Abstract

This study proposes a weather-aware deep reinforcement learning (DRL) framework for predictive modelling of household energy dynamics. Using a 14-month high-resolution dataset from a residence in Northeast Mexico, the framework integrates detailed meteorological attributes and next-day forecasts to enhance prediction accuracy. Four DRL algorithms were implemented and evaluated for their performance in forecasting household energy consumption: Proximal Policy Optimisation (PPO), Soft Actor-Critic (SAC), Deep Deterministic Policy Gradient (DDPG), and Asynchronous Advantage Actor-Critic (A3C). Exploratory data analysis revealed significant seasonal trends and variability in energy usage patterns. Results show that DDPG and SAC outperform PPO and A3C, achieving the lowest root mean square error (RMSE) and mean absolute error (MAE), with DDPG recording 0.0011 RMSE and 0.0009 MAE. The framework was tested on moderately equipped hardware, demonstrating the practical feasibility of DRL-based energy forecasting systems. This work contributes original visualisations and comparative insights, advancing smart energy management solutions.

Keywords

Deep Reinforcement Learning , Household Energy Forecasting , Weather Data Integration , Predictive Modelling , Smart Energy Systems

References

• Meenal R., Binu D., Ramya K., Michael P. A., Vinoth Kumar K., Rajasekaran E., and Sangeetha B. 2022. Weather forecasting for renewable energy systems: A review. Archives of Computational Methods in Engineering 29, (2022), 2875–2891. doi: 10.1007/s11831-021-09695-3 google scholar
• Hao Zhang, Ming Zhang, Rui Yi, Yaping Liu, Qian-Hua Wen, and Xuan Meng. 2024. Growing importance of micrometeorology in the new power system: Review, analysis, and case study. Energies 17, (2024), 1365. https://doi.org/10.3390/en17061365 google scholar
• George Papaefthymiou and Ken Dragoon. 2016. Towards 100% renewable energy systems: Uncapping power system flexibility. Energy Policy 92, (2016), 69–82. DOI: 10.1016/j.enpol.2016.01.025 google scholar
• Martina K. Linnenluecke, Annika Griffiths, and Monika Winn. 2022. Extreme weather events and the critical importance of anticipatory adaptation and organisational resilience in responding to impacts. Business Strategy and the Environment (2022), 17–32. DOI: 10.1002/bse.708 google scholar
• James W. Day and Charles Hall. 2016. Global climate change: A warmer and more unpredictable future. In America’s Most Sustainable Cities and Regions: Surviving the 21st Century Megatrends. Springer, 137–166. doi: 10.1007/978-1-4939-3243-6 google scholar
• Peter Bauer, Alan Thorpe, and Gilbert Brunet. 2015. The quiet revolution of numerical weather prediction. Nature (2015), 47–55. doi: 10.1038/nature14956 google scholar
• Anthony Lupo, William Kininmonth, J. Scott Armstrong and Kesten Green. 2013. Global climate models and their limitations. In Climate Change Reconsidered II: Physical Science. google scholar
• Xiaoli Ren, Xiaoyong Li, Kaijun Ren, Junqiang Song, Zichen Xu, Kefeng Deng, Xiang Wang, Deep Learning-Based Weather Prediction: A Survey, Big Data Research, Volume 23, 2021, 100178, ISSN 2214-5796, https://doi.org/10.1016/j.bdr.2020.100178. google scholar
• Markus Reichstein, Gustau Camps-Valls, Bjorn Stevens, Martin Jung, Joachim Denzler, Nuno Carvalhais, and Prabhat. 2019. Deep learning and process understanding for data-driven Earth system science. Nature (2019), 195–204. doi: 10.1038/s41586-019-0912-1 google scholar
• Lam et al. 2023. Learning skilful medium-range global weather forecasting. Science 382, (2023), 1416–1421. https://doi.org/10.1126/science.adi2336 google scholar
• Sunil S., Aparna B., Anik S., Tusar K. H., Biswajeet P., and Abdullah A. 2024. Deep learning algorithms-based landslide vulnerability modelling in highly landslide-prone areas of Tamil Nadu, India. Geoscience Journal 28, (2024), 1013–1038. doi: 10.1007/s12303-024-0044-y google scholar
• Kaifeng Bi, Lixin Xie, Hongyi Zhang, Xin Chen, Xiaotian Gu, and Qi Tian. 2023. Accurate medium-range global weather forecasting using 3D neural networks. Nature 619, (2023), 533–538. doi: 10.1038/s41586-023-06185-3 google scholar
• Jesse Willard, Xiaowei Jia, Shaobo Xu, Michael Steinbach, and Vipin Kumar. 2022. Integrating scientific knowledge with machine learning for engineering and environmental systems. ACM Computing Surveys (2022), 37–66. https://doi.org/10.1145/3514228 google scholar
• David Hansen, Daniel C. Maddix, Soheil Alizadeh, Gaurav Gupta, and Michael W. Mahoney. 2024. Learning physical models that can respect conservation laws. Physica D: Nonlinear Phenomena 457, (2024), 133952. google scholar
• https://doi.org/10.1016/j.physd.2023.133952 google scholar
• David Lazer, Ryan Kennedy, Gary King, and Alessandro Vespignani. 2014. The parable of Google Flu: Traps in big data analysis. Science (2014), 1203–1205. https://doi.org/10.1126/science.1248506 google scholar
• Brian E. Rose. 2018. CLIMLAB: A Python toolkit for interactive, process-oriented climate modelling. Journal of Open Source Software (2018). doi: 10.21105/joss.00659 google scholar
• Ronald J. Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8, 3 (1992), 229–256. doi: 10.1007/BF00992696 google scholar
• Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015. Continuous control with deep reinforcement learning. arXiv:1509.02971 [cs. LG]. https://arxiv.org/abs/1509.02971 google scholar
• Paulo Vitor B. Ramos et al. 2023. Residential energy consumption forecasting using deep learning models. Applied Energy 350, (2023), 121705. doi: 10.1016/j.apenergy.2023.121705 google scholar
• Eoin McKenna, Alan Peplinski et al. 2024. A machine learning framework to estimate residential electricity demand based on smart metre electricity, climate, building characteristics, and socioeconomic datasets. Applied Energy 357, (2024), 122413. 10.1016/j.apenergy.2023.122413 google scholar
• Das et al. 2024. Machine learning-based energy forecasting for residential communities. In Proceedings of the IEEE 9th International Conference for Convergence in Technology (I2CT), 1–4. https://doi.org/10.1109/I2CT61223.2024.10544342 google scholar
• H. M. Shakeel, S. Iram, R. Hill, H. M. Athar Farid, A. Sheikh-Akbari, and F. Saleem. 2025. A machine learning-based intelligent framework for predicting energy efficiency in next-generation residential buildings. Buildings 8, 5 (2025), 1275. doi: 10.3390/buildings15081275 google scholar
• Enes Bajrami, Andrea Kulakov, Eftim Zdravevski, and Petre Lameski. 2025. A comparative analysis of PPO and SAC algorithms for energy optimisation with country-level energy consumption insights. IFAC Journal of Systems and Control 34, (2025), 100344. https://doi.org/10.1016/j.ifacsc.2025.100344 google scholar
• Andrea Franzoso, Gabriele Fambri, and Marco Badami. 2025. Deep reinforcement learning as a tool for the analysis and optimisation of energy flows in multi-energy systems. Energy Conversion and Management 341, (2025), 120095. google scholar
• https://doi.org/10.1016/j.enconman.2025.120095 google scholar
• Qi Shuai, Yin Yin, Shuang Huang, and Cheng Chen. 2025. Deep reinforcement learning-based real-time energy management for an integrated electric–thermal energy system. Sustainability 17, 2 (2025), 407. google scholar
• https://doi.org/10.3390/su17020407 google scholar
• Pauline Béraud, Margaux Rioux, Michel Babany, Philippe de La Chevasnerie, Damien Theis, Giacomo Teodori, Chloé Pinguet, Romane Rigaud, and François Leclerc. 2024. An explainable machine learning approach for energy forecasting at the household level. arXiv:2410.14416 [cs. LG]. https://arxiv.org/abs/2410.14416 google scholar
• John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal Policy Optimisation Algorithms. arXiv:1707.06347 [cs. LG]. https://arxiv.org/abs/1707.06347 google scholar
• John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal Policy Optimisation Algorithms. arXiv:1707.06347 [cs. LG]. https://arxiv.org/abs/1707.06347 google scholar
• Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. 2018. Soft actor-critical: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic actor. arXiv:1801.01290 [cs. LG]. https://arxiv.org/abs/1801.01290 google scholar
• Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2019. Continuous control with deep reinforcement learning. arXiv:1509.02971v6 [cs. LG]. https://arxiv.org/abs/1509.02971 google scholar
• Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. arXiv:1602.01783 [cs. LG]. https://arxiv.org/abs/1602.01783 google scholar
• Baldemar Aguirre-Fraire, Jessica Beltrán, and Valeria Soto. 2024. Household energy consumption enriched with weather data in the northeast of Mexico. Dataset. https://doi.org/10.17632/tvhygj8rgg.1. google scholar
• Siyu Wang et al. 2024. Comparison of deep reinforcement learning-based energy management strategies for fuel cell vehicles considering economics, durability, and adaptability. Energy 307, (2024), 132771. google scholar
• https://doi.org/10.1016/j.energy.2024.132771google scholar
• Cristian Vidal-Silva et al. 2025. A comparative study of a deep reinforcement learning solution and alternative deep learning models for wildfire prediction. Applied Sciences 15, 7 (2025), 3990. doi: 10.3390/app15073990 google scholar
• Tao et al. 2020. Study deep reinforcement learning techniques for building energy consumption forecasting. Energy and Buildings (2020), 109675. doi: 10.1016/j.enbuild.2019.109675 google scholar