Transformer (Dönüştürücü) Mimarisi Kullanarak Zaman Serilerinde Anomali Tespiti

Year 2023, Volume: 2 Issue: 1, 1 - 7, 21.06.2023

Osman Safa Çifçi

Abstract

Çok değişkenli zaman verilerinde anormallik tespiti pek çok uygulama için önem arz etmektedir. Ancak, yüksek doğrulukla ve hızlı bir şekilde anormallik tespiti tam olarak çözülebilmiş bir problem değildir. Pek çok zaman verisi setindeki veri oynaklığı da problemi güçlendirmektedir. Son yıllarda anormallik tespiti için istatistiksel yaklaşımların yanısıra derin öğrenme tabanlı yöntemler ortaya çıkmıştır. Bunların içinde 2017 yılında ortaya çıkan ve ilk olarak dil çeviri uygulamalarında kullanılan dönüştürücü (transformer) tabanlı modeller önerilmiştir. Dönüştürücü tabanlı modeller paralel olarak işlem yaparak eğitim sürecini kısaltıp çeşitli uygulamalarda yüksek performans sergilemektedir. 2022 yılında çıkan TranAD isimli model hem temel modellere göre 100 kata kadar daha hızlı eğitim sürecine sahiptir hem de performans olarak pek çok modelden daha iyidir. Bu çalışmada ilgili model daha da hızlı eğitim yapacak şekilde modifiye edilmiştir ve orjinal makalede denenmeyen halka açık başka bir çok zamanlı verisetinde denenmiştir. Ortaya çıkan model eski bir dizüstü bilgisayarda yirmi saniyenin altında eğitilip, kendisinden daha fazla veri kullanan ve daha çok kaynak gerektiren modellere benzer performans sergilemiştir. Elde edilen model çeşitli zaman serilerinde, özellikle de sanayi 4.0 uygulamlarında kendine yer bulabilir.

Keywords

Zaman serileri, Anormallik tespiti, Dönüştürücü modeli

References

• [1] H. Xu et al., “Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications,” in The Web Conference 2018 - Proceedings of the World Wide Web Conference, WWW 2018, 2018.
• [2] Y. Su, R. Liu, Y. Zhao, W. Sun, C. Niu, and D. Pei, “Robust anomaly detection for multivariate time series through stochastic recurrent neural network,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019.
• [3] C. Zhang et al., “A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data,” in 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019 and the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, 2019.
• [4] A. Sánchez-Fernández, F. J. Baldán, G. I. Sainz-Palmero, J. M. Benítez, and M. J. Fuente, “Fault detection based on time series modeling and multivariate statistical process control,” Chemom. Intell. Lab. Syst., 2018.
• [5] F. Zeng, M. Chen, C. Qian, Y. Wang, Y. Zhou, and W. Tang, “Multivariate time series anomaly detection with adversarial transformer architecture in the Internet of Things,” Futur. Gener. Comput. Syst., 2023.
• [6] M. Ma, L. Han, and C. Zhou, “BTAD: A binary transformer deep neural network model for anomaly detection in multivariate time series data,” Adv. Eng. Informatics, 2023.
• [7] A. Blázquez-García, A. Conde, U. Mori, and J. A. Lozano, “A Review on Outlier/Anomaly Detection in Time Series Data,” ACM Computing Surveys. 2021.
• [8] Y. Zhou, R. Qin, H. Xu, S. Sadiq, and Y. Yu, “A data quality control method for seafloor observatories: The application of observed time series data in the east China sea,” Sensors (Switzerland), 2018.
• [9] G. Li and J. J. Jung, “Deep learning for anomaly detection in multivariate time series: Approaches, applications, and challenges,” Information Fusion. 2023.
• [10] A. Garg, W. Zhang, J. Samaran, R. Savitha, and C. S. Foo, “An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series,” IEEE Trans. Neural Networks Learn. Syst., 2022.
• [11] X. Wang, D. Pi, X. Zhang, H. Liu, and C. Guo, “Variational transformer-based anomaly detection approach for multivariate time series,” Meas. J. Int. Meas. Confed., 2022.
• [12] Z. Tian, M. Zhuo, L. Liu, J. Chen, and S. Zhou, “Anomaly detection using spatial and temporal information in multivariate time series,” Sci. Rep., 2023.
• [13] K. Choi, J. Yi, C. Park, and S. Yoon, “Deep Learning for Anomaly Detection in Time-Series Data: Review, Analysis, and Guidelines,” IEEE Access. 2021.
• [14] M. M. Breuniq, H. P. Kriegel, R. T. Ng, and J. Sander, “LOF: Identifying density-based local outliers,” SIGMOD Rec. (ACM Spec. Interes. Gr. Manag. Data), 2000.
• [15] J. Tang, Z. Chen, A. W. C. Fu, and D. W. Cheung, “Enhancing effectiveness of Outlier detections for low Density Patterns,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2002.
• [16] B. Zong et al., “Deep autoencoding Gaussian mixture model for unsupervised anomaly detection,” in 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings, 2018.
• [17] T. Yairi, N. Takeishi, T. Oda, Y. Nakajima, N. Nishimura, and N. Takata, “A Data-Driven Health Monitoring Method for Satellite Housekeeping Data Based on Probabilistic Clustering and Dimensionality Reduction,” IEEE Trans. Aerosp. Electron. Syst., 2017.
• [18] D. M. J. Tax and R. P. W. Duin, “Support Vector Data Description,” Mach. Learn., 2004.
• [19] L. Ruff et al., “Deep one-class classification,” in 35th International Conference on Machine Learning, ICML 2018, 2018.
• [20] L. Shen, Z. Li, and J. T. Kwok, “Timeseries anomaly detection using temporal hierarchical one-class network,” in Advances in Neural Information Processing Systems, 2020.
• [21] Y. Shin et al., “ITAD: Integrative Tensor-based Anomaly Detection System for Reducing False Positives of Satellite Systems,” in International Conference on Information and Knowledge Management, Proceedings, 2020.
• [22] D. Park, Y. Hoshi, and C. C. Kemp, “A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder,” IEEE Robot. Autom. Lett., 2018.
• [23] Z. Li et al., “Multivariate Time Series Anomaly Detection and Interpretation using Hierarchical Inter-Metric and Temporal Embedding,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2021.
• [24] I. J. Goodfellow et al., “Generative adversarial nets,” in Advances in Neural Information Processing Systems, 2014.
• [25] B. Zhou, S. Liu, B. Hooi, X. Cheng, and J. Ye, “Beatgan: Anomalous rhythm detection using adversarially generated time series,” in IJCAI International Joint Conference on Artificial Intelligence, 2019.
• [26] E. Mahmoud, “Accuracy in forecasting: A survey,” J. Forecast., 1984.
• [27] K. Hundman, V. Constantinou, C. Laporte, I. Colwell, and T. Soderstrom, “Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2018.
• [28] J. Audibert, P. Michiardi, F. Guyard, S. Marti, and M. A. Zuluaga, “USAD: UnSupervised Anomaly Detection on Multivariate Time Series,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2020.
• [29] H. Zhao et al., “Multivariate time-series anomaly detection via graph attention network,” in Proceedings - IEEE International Conference on Data Mining, ICDM, 2020.
• [30] A. Vaswani et al., “Attention is all you need,” in Advances in Neural Information Processing Systems, 2017.
• [31] T. B. Brown et al., “Language models are few-shot learners,” in Advances in Neural Information Processing Systems, 2020.
• [32] C. Z. A. Huang et al., “Music transformer: Generating music with long-term structure,” in 7th International Conference on Learning Representations, ICLR 2019, 2019.
• [33] Y. Wang, R. Huang, S. Song, Z. Huang, and G. Huang, “Not All Images are Worth 16x16 Words: Dynamic Vision Transformers with Adaptive Sequence Length,” in NeurIPS, 2021.
• [34] S. Li et al., “Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting,” in Advances in Neural Information Processing Systems, 2019.
• [35] C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in 34th International Conference on Machine Learning, ICML 2017, 2017.
• [36] S. Tuli, G. Casale, and N. R. Jennings, “TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data,” Proc. VLDB Endow., vol. 15, no. 6, pp. 1201–1214, Jun. 2022.
• [37] A. Abdulaal, Z. Liu, and T. Lancewicki, “Practical Approach to Asynchronous Multivariate Time Series Anomaly Detection and Localization,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2021.
• [38] D. Li, D. Chen, B. Jin, L. Shi, J. Goh, and S. K. Ng, “MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2019.
• [39] A. Siffer, P. A. Fouque, A. Termier, and C. Largouet, “Anomaly detection in streams with extreme value theory,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017.
• [40] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Comput., 1997.