Flexibility Detection of Energy Consumers Using Deep Time Series Analysis and Probabilistic Clustering in Smart Meter Data

Year 2025, Volume: 13 Issue: 3, 1157 - 1172, 30.09.2025

Zühre Aydın

https://doi.org/10.29109/gujsc.1710629

Abstract

High resolution smart meter data provides valuable insights into the time-dependent behavioral patterns of electricity consumers, making it a critical input for demand-side flexibility analysis. By leveraging deep time series learning techniques to capture long term consumption dependencies, and applying probabilistic clustering algorithms, it is possible to extract flexible consumer groups that can inform the restructuring of energy systems based on demand-side dynamics. This study aims to accurately identify residential consumer segments with load shifting potential by analyzing their time series based consumption behaviors using deep learning methods. The resulting consumer clusters were evaluated in terms of their load profiles, behavioral similarities, and potential contributions to flexibility, generating data-driven outputs for distributed flexibility market planning. Unlike traditional segmentation methods commonly found in the literature, the proposed approach more effectively captures structural patterns within time series data, thereby supporting the design of targeted demand response strategies for power systems. The robustness of the segmentation was validated using the Silhouette score metric, with an average score of 0.65 indicating both high intra-cluster cohesion and clear inter cluster separation. These findings demonstrate that the proposed segmentation approach offers not only visually and intuitively distinct groups, but also quantitatively validated and flexibility-oriented consumer profiles, outperforming conventional clustering techniques.

Keywords

Transformer Based Time Series , Smart Meter Data , Energy Demand Management , Gaussian Mixture Modeling , Flexible Consumer Groups , Energy Consumer Segmentation , Demand Side Flexibility

Akıllı Sayaç Verilerinde Derin Zaman Serisi Analizi ve Olasılık Tabanlı Kümeleme ile Enerji Tüketicisi Esneklik Tespiti

Abstract

Yüksek çözünürlüklü akıllı sayaç verileri, elektrik tüketicilerinin zamana bağlı davranış örüntülerini ortaya koyarak talep tarafı esnekliği analizinde önemli bir veri kaynağı sunmaktadır. Derin zaman serisi öğrenme yöntemleri ile tüketim desenlerindeki uzun vadeli bağımlılıkların modellenmesi ve olasılık tabanlı kümeleme algoritmalarıyla esnek tüketici gruplarının ayrıştırılması, enerji sistemlerinin esnekliğe dayalı yeniden yapılandırılmasına katkı sağlamaktadır. Bu kapsamda, çalışmanın temel amacı; mesken tipi tüketicilerin zaman serisi tabanlı tüketim davranışlarını derin öğrenme yaklaşımlarıyla analiz ederek, yük kaydırma potansiyeli taşıyan grupları doğru biçimde tanımlayabilmektir. Modelleme sonucunda elde edilen tüketici kümeleri, yük profili eğilimleri, davranışsal benzerlik düzeyleri ve potansiyel esneklik katkıları açısından değerlendirilmiş; böylece dağıtık esneklik piyasalarına yönelik veri temelli planlama çıktıları üretilmiştir. Literatürde yaygın olarak kullanılan geleneksel segmentasyon yöntemlerinin aksine, önerilen yöntem zaman serilerindeki yapısal örüntüleri daha isabetli biçimde ayrıştırarak enerji sistemleri için hedefli talep tarafı stratejileri geliştirilmesini desteklemektedir. Elde edilen segmentasyon sonuçlarının güvenilirliği Silhouette skoru ile doğrulanmış, ortalama 0.65 düzeyinde elde edilen skorlar, hem küme içi tutarlılığın hem de kümeler arası ayrışmanın yüksek olduğunu göstermiştir. Bu bulgular, tüketici segmentasyonunun yalnızca görsel ve sezgisel değil, aynı zamanda nicel olarak da güçlü bir yapı sunduğunu ortaya koymakta; geliştirilen yaklaşımın klasik yöntemlere göre daha anlamlı, ayrıştırılabilir ve esneklik odaklı gruplar üretme kapasitesini kanıtlamaktadır.

Keywords

Transformatör Tabanlı Zaman Serisi , Akıllı Sayaç Verisi , Enerji Talebi Yönetimi , Gaussian Karışım Modellemesi , Esnek Tüketici Grupları , Enerji Tüketici Grupları , Talep Tarafı Esnekliği

References

• [1] Figueiredo, V., Rodrigues, F., Vale, Z., Gouveia, J. B. (2005). An electric energy consumer characterization framework based on data mining techniques. IEEE Transactions on Power Systems, 20(2), 596-602. https://doi.org/10.1109/TPWRS.2005.846234.
• [2] Chicco, G., Napoli, R., Piglione, F. (2006). Comparisons among clustering techniques for electricity customer classification. IEEE Transactions on Power Systems, 21(2), 933-940. https://doi.org/10.1109/TPWRS.2006.873122
• [3] Commission for Energy Regulation (CER). (2009-2010). CER Smart Metering Project–Electricity Customer Behaviour Trial (1st ed.). Irish Social Science Data Archive. SN: 0012-00.
• [4] Palensky, P., Dietrich, D. (2011). Demand side management: Demand response, intelligent energy systems, and smart loads. IEEE Transactions on Industrial Informatics, 7(3), 381-388. https://doi.org/10.1109/TII.2011.2158841
• [5] Quesada, C., Astigarraga, L., Merveille, C., Borges, C. E. (2024). An electricity smart meter dataset of Spanish households: insights into consumption patterns. Scientific Data, 11(1), 59.
• [6] Granja, C. Q., Hernández, C. E. B., Astigarraga, L., Merveille, C. (2023). GoiEner smart meters raw data. GoiEner smart meters raw data.https://zenodo.org/records/7362094
• [7] Albert, A., Rajagopal, R. (2013). Smart meter driven segmentation: What your consumption says about you. IEEE Transactions on Power Systems, 28(4), 4019-4030. https://doi.org/10.1109/TPWRS.2013.2266122
• [8] Beckel, C., Sadamori, L., Staake, T., Santini, S. (2014). Revealing household characteristics from smart meter data. Energy, 78, 397-410. https://doi.org/10.1016/j.energy.2014.10.025
• [9] Kwac, J., Flora, J., Rajagopal, R. (2014). Household energy consumption segmentation using hourly data. IEEE Transactions on Smart Grid, 5(1), 420-430. https://doi.org/10.1109/TSG.2013.2278477
• [10] Siano, P. (2014). Demand response and smart grids, A survey. Renewable and Sustainable Energy Reviews, 30, 461-478. https://doi.org/10.1016/j.rser.2013.10.022
• [11] Chen, X., Kang, C., O'Malley, M., Xia, Q., Bai, J. (2015). Increasing the flexibility of combined heat and power for wind power integration in China: Modeling and implications. IEEE Transactions on Power Systems, 30(4), 1848-1857. https://doi.org/10.1109/TPWRS.2014.2356796
• [12] Cominola, A., Giuliani, M., Piga, D., Castelletti, A., Rizzoli, A. E. (2015). Benefits and challenges of using smart meters for advancing residential water demand modeling and management. Environmental Modelling Software, 72, 198-214. https://doi.org/10.1016/j.envsoft.2015.07.012
• [13] McLoughlin, F., Duffy, A., Conlon, M. (2015). A clustering approach to domestic electricity load profile characterisation using smart metering data. Applied Energy, 141, 190-199. https://doi.org/10.1016/j.apenergy.2014.12.039
• [14] Parag, Y., Sovacool, B. K. (2016). Electricity market design for the prosumer era. Nature Energy, 1(4), 16032. https://doi.org/10.1038/nenergy.2016.32
• [15] Haben, S., Ward, J., Greetham, D. V., Singleton, C., Grindrod, P. (2016). A new error measure for forecasts of household-level, high resolution electrical energy consumption. International Journal of Forecasting, 32(4), 1058-1072. https://doi.org/10.1016/j.ijforecast.2016.02.005
• [16] Wang, Y., Chen, Q., Kang, C., Xia, Q. (2016). Clustering of electricity consumption behavior dynamics toward big data applications. IEEE Transactions on Smart Grid, 7(5), 2437-2447. https://doi.org/10.1109/TSG.2016.2547995
• [17] Zhou, B., Li, W., Chan, K. W., Cao, Y., Kuang, Y., Liu, X., Wang, X. (2016). Smart home energy management systems: Concept, configurations, and scheduling strategies. Renewable and Sustainable Energy Reviews, 61, 30-40. https://doi.org/10.1016/j.rser.2016.03.047
• [18] Al-Wakeel, A., Wu, J., Jenkins, N. (2017). k-means based load estimation of domestic smart meter measurements. Applied Energy, 194, 333-342. https://doi.org/10.1016/j.apenergy.2016.06.046
• [19] Shi, H., Xu, M., Li, R. (2017). Deep learning for household load forecasting—A novel pooling deep RNN. IEEE Transactions on Smart Grid, 8(5), 2421-2430. https://doi.org/10.1109/TSG.2017.2682342
• [20] Zheng, J., Xu, C., Zhang, Z., Li, X. (2017). Electric load forecasting in smart grids using long-short-term-memory based recurrent neural network. Proceedings of the 51st Annual Conference on Information Sciences and Systems (CISS). https://doi.org/10.1109/CISS.2017.7926111
• [21] Chen, Y., Xu, P., Gu, J., Schmidt, F., Li, W. (2018). Measures to improve energy demand flexibility in buildings for demand response (DR): A review. Energy and Buildings, 177, 125-139. https://doi.org/10.1016/j.enbuild.2018.08.003
• [22] Wang, F., Li, K., Liu, C., Mi, Z., Shafie-khah, M., Catalão, J. P. (2018). Synchronised pattern matching based energy time series classification for demand response applications. Applied Energy, 232, 603-614. https://doi.org/10.1016/j.apenergy.2018.09.188
• [23] Zhang, C., Wu, J., Zhou, Y., Cheng, M., Long, C. (2018). Peer-to-peer energy trading in a microgrid. Applied Energy, 220, 1-12. https://doi.org/10.1016/j.apenergy.2018.03.010
• [24] Zhang, D., Han, X., Deng, C. (2018). Review on the research and practice of deep learning and reinforcement learning in smart grids. CSEE Journal of Power and Energy Systems, 4(3), 362-370. https://doi.org/10.17775/CSEEJPES.2018.00560
• [25] Vercamer, D., Steurtewagen, B., Van den Poel, D., Vermeulen, F. (2018). A transformer-based approach for electricity consumer clustering using smart meter data. Energy, 114, 1089-1100. https://doi.org/10.1016/j.energy.2016.08.073
• [26] Zhang, W., Quan, H., Srinivasan, D. (2019). An improved quantile regression neural network for probabilistic load forecasting. IEEE Transactions on Smart Grid, 10(4), 4425-4434. https://doi.org/10.1109/TSG.2018.2867617
• [27] Kong, W., Dong, Z. Y., Jia, Y., Hill, D. J., Xu, Y., Zhang, Y. (2019). Short-term residential load forecasting based on LSTM recurrent neural network. IEEE Transactions on Smart Grid, 10(1), 841-851. https://doi.org/10.1109/TSG.2017.2753802
• [28] Li, S., Jin, X., Xuan, Y., Zhou, X., Chen, W., Wang, Y., Yan, X. (2019). Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Advances in Neural Information Processing Systems, 32.
• [29] Zhu, J., Yang, Z., Mourshed, M., Guo, Y., Zhou, Y., Chang, Y., Wu, J. (2019). Electric vehicle charging load forecasting: A comparative study of deep learning approaches. Energies, 12(14), 2692. https://doi.org/10.3390/en12142692
• [30] Irish Meteorological Service. (2021). Historical Weather Data. https://www.met.ie/climate/available-data/historical-data
• [31] Zhang, X., Grolinger, K. (2020). Time-series transformer for energy forecasting in smart buildings. Energy and Buildings, 223, 110197. https://doi.org/10.1016/j.enbuild.2020.110197
• [32] Tüfekci, P., Torriti, J. (2020). Quantifying household energy flexibility potential using smart meter data. Renewable Energy, 158, 996-1006. https://doi.org/10.1016/j.renene.2020.05.053
• [33] Gao, Y., Liu, X., Zhang, J., Yang, Z. (2021). A transformer-based deep learning approach for forecasting household electricity consumption. Applied Energy, 303, 117696. https://doi.org/10.1016/j.apenergy.2021.117696
• [34] Li, R., Li, F. (2021). Gaussian mixture model-based clustering for smart meter data analysis. Energy and AI, 3, 100051. https://doi.org/10.1016/j.egyai.2021.100051
• [35] Niu, D., Yu, M., Sun, L., Gao, T., Wang, K. (2021). Short-term multi-energy load forecasting for integrated energy systems based on CNN-Bi-GRU optimized by attention mechanism. Applied Energy, 313, 118801. https://doi.org/10.1016/j.apenergy.2022.11880
• [36] Wang, Z., Wang, Y. (2021). A hybrid deep learning model for short-term PV power forecasting. Applied Energy, 304, 117751. https://doi.org/10.1016/j.apenergy.2021.117751
• [37] Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., Zhang, W. (2021). Informer: Beyond efficient transformer for long sequence time-series forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 35(12), 11106-11115. https://doi.org/10.1609/aaai.v35i12.17325
• [38] Kudarihal, C. S., Gupta, M., Gupta, S. K. (2025). Time series analysis of AMI data and comparative energy demand forecasting using deep learning models in a smart grid scenario. Engineering Research Express, 7(1), 015387.
• [39] Pentsos, V., Tragoudas, S., Wibbenmeyer, J., Khdeer, N. (2025). A hybrid LSTM-Transformer model for power load forecasting. IEEE Transactions on Smart Grid.
• [40] Hamad, A., Yaylacı, E. K., Antar, A. P. D. R. K. (2025). Voltage Level Managements of Multilevel Inverter Based on Renewable Energy Sources and Environment Conditions. Gazi Üniversitesi Fen Bilimleri Dergisi Part C: Tasarım ve Teknoloji, 13(1), 404-416.