Anomaly Detection in Meteorological Data Using a Hierarchical Temporal Memory Model: A Study on the Case of Kazakhstan

Öz

In meteorology, which studies atmospheric events, data representing various properties such as temperature, rainfall, and wind speed are collected regularly over a certain period. Unexpected trends in the data may indicate that an abnormal situation is approaching. Therefore, time series (TS) data play an essential role in the early detection of potential meteorological risks. However, applying effective models by considering many complex parameters in performing accurate analysis and anomaly detection (AD) is an important criterion. In this study, machine learning-based AD is performed using a dataset containing meteorological data on different features collected between January 1, 2019, and June 30, 2023, for Kazakhstan, which has the ninth-largest surface area in the world. The Hierarchical Temporal Memory (HTM) model was used for AD, which can provide more accurate forecasts by modeling long-term dependencies and producing effective results in solving TS problems. Detected anomalies are reported at various levels depending on threshold values. In addition, to analyze the ADs more precisely, correlations are calculated using the Spearman model, which allows us to determine the strength and direction of the monotonic relationship between variables. The study's findings show that the HTM is an effective model for AD using TS data on meteorological features.

Anahtar Kelimeler

• Anomaly detection
• time series
• meteorological anomalies
• machine learning
• hierarchical temporal memory.

Hiyerarşik Zamansal Bellek Modeli ile Meteorolojik Verilerdeki Anomalilerin Tespiti: Kazakistan Örneği Üzerine Bir Çalışma

Öz

Atmosferik olayları inceleyen meteorolojide, sıcaklık, yağış ve rüzgar hızı gibi çeşitli özellikleri temsil eden veriler belirli bir süre boyunca düzenli olarak toplanmaktadır. Verilerdeki beklenmedik eğilimler anormal bir durumun yaklaşmakta olduğunu gösterebilmektedir. Bu nedenle, zaman serisi verileri potansiyel meteorolojik risklerin erken tespitinde önemli bir rol oynamaktadır. Ancak doğru ve güvenilir analizlerin gerçekleştirilmesinde ve anomali tespitinde karmaşık birçok parametreyi göz önünde bulundurarak etkin modelleri uygulamak önemli bir kriterdir. Bu çalışmada, dünyanın en büyük dokuzuncu yüzölçümüne sahip Kazakistan için 1 Ocak 2019 ile 30 Haziran 2023 tarihleri arasında toplanan farklı özelliklerdeki meteorolojik verileri içeren bir veri seti kullanılarak makine öğrenmesi tabanlı anomali tespiti gerçekleştirilmiştir. Anomali tespiti için uzun vadeli bağımlılıkları modelleyerek daha doğru tahminler sağlayabilen ve zaman serisi problemlerinin çözümünde etkin sonuçlar üreten Hiyerarşik Zamansal Bellek (HTM) modeli kullanılmıştır. Tespit edilen anomaliler eşik değerlerine bağlı olarak çeşitli seviyelerde raporlanmıştır. Ayrıca, anomali tespitlerini daha hassas bir şekilde analiz etmek için, değişkenler arasındaki monotonik ilişkinin gücünü ve yönünü belirlememizi sağlayan Spearman modeli kullanılarak korelasyonlar hesaplanmıştır. Çalışmanın bulguları, HTM modelinin meteorolojik özelliklere ilişkin zaman serisi verilerinin kullanıldığı AD problemlerinde etkin bir araç olduğunu göstermektedir.

Anahtar Kelimeler

• Anomali tespiti
• zaman serileri
• meteorolojik anomaliler
• makine öğrenmesi
• hiyerarşik zamansal bellek.

Kaynakça

1. Friston K and Buzsáki G, “The functional anatomy of time: what and when in the brain,” Trends in cognitive sciences, vol. 20, no. 7, pp. 500–511, 2016.
2. Choi K, Yi J., Park C., and Yoon S., “Deep learning for anomaly detection in time-series data: review, analysis, and guidelines,” IEEE Access, vol. 9, pp. 120043–120065, 2021.
3. Längkvist M., Karlsson L., and Loutfi A., “A review of unsupervised feature learning and deep learning for time-series modeling,” Pattern Recognition Letters, vol. 42, no. 1, pp. 11–24, 2014, doi: 10.1016/j.patrec.2014.01.008.
4. Idrees SM, Alam SM, and Agarwal P., “A prediction approach for stock market volatility based on time series data,” IEEE Access, vol. 7, pp. 17287–17298, 2019.
5. Box G., “Box and Jenkins: Time Series Analysis, Forecasting and Control,” in A Very British Affair: Six Britons and the Development of Time Series Analysis During the 20th Century, Springer, 2013, pp. 161–215.
6. Fu TC, “A review on time series data mining,” Engineering Applications of Artificial Intelligence, vol. 24, no. 1, pp. 164–181, 2011, doi: 10.1016/j.engappai.2010.09.007.
7. Donatelli RE, Park JA, Mathews SM, and Lee JS, “Time series analysis,” American Journal of Orthodontics and Dentofacial Orthopedics, vol. 161, no. 4. Elsevier, pp. 605–608, 2022.
8. Dietterich TG, “Machine learning for sequential data: A review,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2002, vol. 2396, pp. 15–30, doi: 10.1007/3-540-70659-3_2.

9. Agrawal R., Faloutsos C., and Swami A., “Efficient similarity search in sequence databases,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 1993, vol. 730 LNCS, pp. 69–84, doi: 10.1007/3-540-57301-1_5.
10. Abonyi J., Feil B., Nemeth S., and Arva P., “Modified Gath-Geva clustering for fuzzy segmentation of multivariate time-series,” Fuzzy Sets and Systems, vol. 149, no. 1, pp. 39–56, 2005, doi: 10.1016/j.fss.2004.07.008.
11. Keogh E. and Ratanamahatana CA, “Exact indexing of dynamic time warping,” Knowledge and information systems, vol. 7, pp. 358–386, 2005.
12. Keogh E. and Pazzani M., “An enhanced representation of time series which allows fast and accurate classification, clustering and relevance feedback,” in Kdd, 1998, vol. 98, pp. 239–243, . Available: http://www.aaai.org/Papers/KDD/1998/KDD98-041.pdf.
13. Liu G., Zhong K., Li H., Chen T., and Wang Y., “A state of art review on time series forecasting with machine learning for environmental parameters in agricultural greenhouses,” Information Processing in Agriculture, 2022.
14. UA. Bhatti et al., “Time series analysis and forecasting of air pollution particulate matter (PM 2.5): an SARIMA and factor analysis approach,” Ieee Access, vol. 9, pp. 41019–41031, 2021.
15. Saka F., Karaoğlan KM, “Detecting Anomalies in Dam Water Levels using Hierarchical Temporal Memory: A Case Study in Istanbul Province,” in 4th International Symposium of Engineering Applications on Civil Engineering and Earth Sciences 2023 (IEACES2023), 2023, pp. 139–150.
16. Thoppil NM., Vasu V., and Rao CSP., “Deep Learning Algorithms for Machinery Health Prognostics Using Time-Series Data: A Review,” Journal of Vibration Engineering and Technologies, vol. 9, no. 6, pp. 1123–1145, 2021, doi: 10.1007/s42417-021-00286-x.
17. Abanda A., Mori U., and Lozano JA., “A review on distance based time series classification,” Data Mining and Knowledge Discovery, vol. 33, no. 2, pp. 378–412, 2019, doi: 10.1007/s10618-018-0596-4.
18. Li H.and Du T., “Multivariate time-series clustering based on component relationship networks,” Expert Systems with Applications, vol. 173, p. 114649, 2021, doi: 10.1016/j.eswa.2021.114649.
19. Wu J., Zeng W., and Yan F., “Hierarchical Temporal Memory method for time-series-based anomaly detection,” Neurocomputing, vol. 273, pp. 535–546, 2018, doi: 10.1016/j.neucom.2017.08.026.
20. Li G.and Jung JJ., “Deep learning for anomaly detection in multivariate time series: Approaches, applications, and challenges,” Information Fusion, vol. 91, pp. 93–102, 2023, doi: 10.1016/j.inffus.2022.10.008.
21. Ahmed M., Mahmood AN., and Islam MR., “A survey of anomaly detection techniques in financial domain,” Future Generation Computer Systems, vol. 55, pp. 278–288, 2016.
22. Shaukat K. et al., “A Review of Time-Series Anomaly Detection Techniques: A Step to Future Perspectives,” in Advances in Intelligent Systems and Computing, 2021, vol. 1363 AISC, pp. 865–877, doi: 10.1007/978-3-030-73100-7_60.
23. Lin J., Keogh E., Fu A., and Van HH., “Approximations to magic: Finding unusual medical time series,” in Proceedings - IEEE Symposium on Computer-Based Medical Systems, 2005, pp. 329–334, doi: 10.1109/cbms.2005.34.
24. Shyu ML., Chen SC., Sarinnapakorn K., and Chang L., “A novel anomaly detection scheme based on principal component classifier,” in Proceedings of the IEEE foundations and new directions of data mining workshop, 2003, pp. 172–179.
25. Angiulli F. and Pizzuti C., “Fast outlier detection in high dimensional spaces,” in European conference on principles of data mining and knowledge discovery, 2002, pp. 15–27.
26. Hosseinzadeh M., Rahmani AM., Vo B., Bidaki M., Masdari M., and Zangakani M., “Improving security using SVM-based anomaly detection: issues and challenges,” Soft Computing, vol. 25, no. 4, pp. 3195–3223, 2021, doi: 10.1007/s00500-020-05373-x.
27. Hu M., Feng X., Ji Z., Yan K., and Zhou S., “A novel computational approach for discord search with local recurrence rates in multivariate time series,” Information Sciences, vol. 477, pp. 220–233, 2019.
28. Chandola V., Banerjee A., and Kumar V., “Anomaly detection: A survey,” ACM computing surveys (CSUR), vol. 41, no. 3, pp. 1–58, 2009.
29. Lavin A. and Ahmad S., “Evaluating real-time anomaly detection algorithms--the Numenta anomaly benchmark,” in 2015 IEEE 14th international conference on machine learning and applications (ICMLA), 2015, pp. 38–44.
30. Sgueglia A., Di Sorbo A., Visaggio CA., and Canfora G., “A systematic literature review of IoT time series anomaly detection solutions,” Future Generation Computer Systems, vol. 134, pp. 170–186, 2022.
31. Terrades OR., Berenguel A., and Gil D., “A flexible outlier detector based on a topology given by graph communities,” Big Data Research, vol. 29, p. 100332, 2022.
32. Li C., Mo L., Tang H., and Yan R., “Lifelong condition monitoring based on NB-IoT for anomaly detection of machinery equipment,” Procedia Manufacturing, vol. 49, pp. 144–149, 2020, doi: 10.1016/j.promfg.2020.07.010.
33. Kim TY. and Cho SB., “Web traffic anomaly detection using C-LSTM neural networks,” Expert Systems with Applications, vol. 106, pp. 66–76, 2018.
34. He Q., Zheng YJ., Zhang CL., and Wang HY., “MTAD-TF: Multivariate time series anomaly detection using the combination of temporal pattern and feature pattern,” Complexity, vol. 2020, pp. 1–9, 2020.
35. Audibert J., Michiardi P., Guyard F., Marti S., and Zuluaga MA., “Do deep neural networks contribute to multivariate time series anomaly detection?,” Pattern Recognition, vol. 132, p. 108945, 2022.
36. Lindemann B., Maschler B., Sahlab N., and Weyrich M., “A survey on anomaly detection for technical systems using LSTM networks,” Computers in Industry, vol. 131, p. 103498, 2021.
37. Ahmed M., Naser MA., and Hu J., “A survey of network anomaly detection techniques,” Journal of Network and Computer Applications, vol. 60, pp. 19–31, 2016, doi: 10.1016/j.jnca.2015.11.016.
38. Fernando T., Gammulle H., Denman S., Sridharan S., and Fookes C., “Deep learning for medical anomaly detection--a survey,” ACM Computing Surveys (CSUR), vol. 54, no. 7, pp. 1–37, 2021.
39. Kwon D., Kim H., Kim J., Suh SC., Kim I., and Kim KJ., “A survey of deep learning-based network anomaly detection,” Cluster Computing, vol. 22, pp. 949–961, 2019.
40. Aldweesh A., Derhab A., and Emam AZ., “Deep learning approaches for anomaly-based intrusion detection systems: A survey, taxonomy, and open issues,” Knowledge-Based Systems, vol. 189, p. 105124, 2020.
41. Rashid ANMB., Ahmed M., Sikos LF., and Haskell-Dowland P., “Anomaly detection in cybersecurity datasets via cooperative co-evolution-based feature selection,” ACM Transactions on Management Information Systems (TMIS), vol. 13, no. 3, pp. 1–39, 2022.
42. Hodge V. and Austin J., “A survey of outlier detection methodologies,” Artificial intelligence review, vol. 22, pp. 85–126, 2004.
43. Chen Y., Sui Y., and Shayilan A., “Constructing a high-performance self-training model based on support vector classifiers to detect gold mineralization-related geochemical anomalies for gold exploration targeting,” Ore Geology Reviews, p. 105265, 2022.
44. Nassif AB., Talib MA., Nasir Q., and Dakalbab FM., “Machine learning for anomaly detection: A systematic review,” Ieee Access, vol. 9, pp. 78658–78700, 2021.
45. Schmidl S., Wenig P., and Papenbrock T., “Anomaly Detection in Time Series: A Comprehensive Evaluation,” Proceedings of the VLDB Endowment, vol. 15, no. 9, pp. 1779–1797, 2022, doi: 10.14778/3538598.3538602.
46. Dong B. and Wang X., “Comparison deep learning method to traditional methods using for network intrusion detection,” in 2016 8th IEEE international conference on communication software and networks (ICCSN), 2016, pp. 581–585.
47. Kim K. and Aminanto ME., “Deep learning in intrusion detection perspective: Overview and further challenges,” in 2017 International Workshop on Big Data and Information Security (IWBIS), 2017, pp. 5–10.
48. Karatas G., Demir O., and Sahingoz OK., “Deep learning in intrusion detection systems,” in 2018 International Congress on Big Data, Deep Learning and Fighting Cyber Terrorism (IBIGDELFT), 2018, pp. 113–116.
49. Li J., Izakian H., Pedrycz W., and Jamal I., “Clustering-based anomaly detection in multivariate time series data,” Applied Soft Computing, vol. 100, p. 106919, 2021.
50. Ahmad S., Lavin A., Purdy S., and Agha Z., “Unsupervised real-time anomaly detection for streaming data,” Neurocomputing, vol. 262, pp. 134–147, 2017.
51. Ma T., Zhu Z., Wang L., Wang H., and Ma L., “Anomaly detection for hydropower turbine based on variational modal decomposition and hierarchical temporal memory,” Energy Reports, vol. 8, pp. 1546–1551, 2022, doi: 10.1016/j.egyr.2022.02.286.
52. Soares E., Costa Jr P., Costa B., and Leite D., “Ensemble of evolving data clouds and fuzzy models for weather time series prediction,” Applied Soft Computing, vol. 64, pp. 445–453, 2018.
53. Bamaqa A., Sedky M., Bosakowski T., and Bastaki BB., “Anomaly Detection Using Hierarchical Temporal Memory (HTM) in Crowd Management,” in Proceedings of the 2020 4th International Conference on Cloud and Big Data Computing, 2020, pp. 37–42, doi: 10.1145/3416921.3416940.
54. “Sample Weather Dataset in Kazakhstan | Kaggle.” https://www.kaggle.com/datasets/armanzhalgasbayev/sample-weather-dataset (accessed Sep. 12, 2023).
55. Cui Y., Surpur C., Ahmad S., and Hawkins J., “A comparative study of HTM and other neural network models for online sequence learning with streaming data,” in 2016 International Joint Conference on Neural Networks (IJCNN), 2016, pp. 1530–1538.
56. Hawkins J.and Ahmad S., “Why neurons have thousands of synapses, a theory of sequence memory in neocortex,” Frontiers in neural circuits, vol. 10, p. 23, 2016, doi: 10.3389/fncir.2016.00023.
57. Hawkins J., Ahmad S., and Dubinsky D., “Hierarchical temporal memory including HTM cortical learning algorithms,” Techical report, Numenta, Inc, Palto Alto http://www.numenta.com/htmoverview/education/HTM_CorticalLearningAlgorithms. pdf, 2010.
58. Wielgosz M. and Pietroń M., “Using Spatial Pooler of Hierarchical Temporal Memory to classify noisy videos with predefined complexity,” Neurocomputing, vol. 240, pp. 84–97, 2017, doi: 10.1016/j.neucom.2017.02.046.
59. Ahmad S. and Hawkins J., “Properties of Sparse Distributed Representations and their Application to Hierarchical Temporal Memory,” arXiv preprint arXiv:1503.07469, 2015, . Available: http://arxiv.org/abs/1503.07469.
60. Struye J. and Latré S., “Hierarchical temporal memory and recurrent neural networks for time series prediction: An empirical validation and reduction to multilayer perceptrons,” Neurocomputing, vol. 396, pp. 291–301, 2020, doi: 10.1016/j.neucom.2018.09.098.
61. Barua A., Muthirayan D., Khargonekar PP., and Al Faruque MA., “Hierarchical Temporal Memory-Based One-Pass Learning for Real-Time Anomaly Detection and Simultaneous Data Prediction in Smart Grids,” IEEE Transactions on Dependable and Secure Computing, vol. 19, no. 3, pp. 1770–1782, 2020.
62. Akila D., Pal S., Jayakarthik R., Chattopadhyay S., and Obaid AJ., “Deep Learning Enhancing Performance Using Support Vector Machine HTM Cortical Learning Algorithm,” in Journal of Physics: Conference Series, 2021, vol. 1963, no. 1, p. 12144, doi: 10.1088/1742-6596/1963/1/012144.
63. Attneave F., MB., and Hebb DO., The Organization of Behavior; A Neuropsychological Theory, vol. 63, no. 4. Psychology press, 1950.
64. Hawkins J., “(BAMI) Biological and Machine Intelligence,” Numenta, p. 64, 2019.
65. Purdy S., “Encoding data for HTM systems,” arXiv preprint arXiv:1602.05925, 2016.
66. Cui Y., Ahmad S., and Hawkins J., “The HTM spatial pooler—a neocortical algorithm for online sparse distributed coding,” Frontiers in Computational Neuroscience, vol. 11, p. 111, 2017, doi: 10.3389/fncom.2017.00111.
67. Lai YK.and Nguyen MH., “A Real-Time DDoS Attack Detection and Classification System Using Hierarchical Temporal Memory,” APSIPA Transactions on Signal and Information Processing, vol. 12, no. 2, 2023.
68. Zyarah AM and Kudithipudi D., “Neuromorphic Architecture for the Hierarchical Temporal Memory,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 3, no. 1, pp. 4–14, 2019, doi: 10.1109/TETCI.2018.2850314.
69. Taylor R., “Interpretation of the Correlation Coefficient: A Basic Review,” Journal of Diagnostic Medical Sonography, vol. 6, no. 1, pp. 35–39, 1990, doi: 10.1177/875647939000600106.
70. Warren WG., “Correlation or Regression: Bias or Precision,” Applied Statistics, vol. 20, no. 2, p. 148, 1971, doi: 10.2307/2346463.
71. Asuero AG., Sayago A., and González AG., “The correlation coefficient: An overview,” Critical Reviews in Analytical Chemistry, vol. 36, no. 1, pp. 41–59, 2006, doi: 10.1080/10408340500526766.
72. Ratner B., “The correlation coefficient: Its values range between+ 1/- 1, or do they?,” Journal of targeting, measurement and analysis for marketing, vol. 17, no. 2, pp. 139–142, 2009.
73. Spearman C., “The Proof and Measurement of Association between Two Things,” The American Journal of Psychology, vol. 15, no. 1, p. 72, 1904, doi: 10.2307/1412159.
74. Lehman A. and Rourke NO., “JMP for Basic Univariate and Multivariate Statistics A Step-by-Step Guide,” Analysis, p. 146, 2005, Accessed: Sep. 18, 2023. . Available: http://books.google.com/books?id=1nlApuloc0AC&pgis=1.
75. Eden SK., Li C., and Shepherd BE., “Nonparametric estimation of Spearman’s rank correlation with bivariate survival data,” Biometrics, vol. 78, no. 2, pp. 421–434, 2022, doi: 10.1111/biom.13453.
76. Agostino RBD., “Tests for the Normal Distribution,” in Goodness-of-Fit Techniques, Routledge, 2018, pp. 367–420.
77. Mohd RN. and Bee WY., “Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests,” Journal of Statistical Modeling and Analytics, vol. 2, no. 1, pp. 13–14, 2011.
78. Yap BW. and Sim CH., “Comparisons of various types of normality tests,” Journal of Statistical Computation and Simulation, vol. 81, no. 12, pp. 2141–2155, 2011, doi: 10.1080/00949655.2010.520163.
79. Kavak B., “Pazarlama araştirmalari tasarim ve analiz.,” Ankara. Detay Yayıncılık, 2013.
80. Gegez AE., Pazarlama araştırmacıları. Beta Basım Yayım Dağıtım AŞ., 2007.
81. Nakip M., Pazarlama Araştırmaları Teknikler ve Uygulamalar. Seçkin Yayıncılık, 2006.