VarioGram – Zaman serileri için renkli bir zaman-graf temsili

Abstract

Bu çalışmada zaman serilerinin ağ tabanlı temsili için bir çerçeve sunulmuştur. Önerilen yöntemde öncelikle, zaman domenindeki sinyaller %50 örtüşmeli sabit genişlikli zaman pencerelerine bölünerek segmentasyon işlemi tamamlanır. Her segment, ana sinyalin mutlak maksimum genlik değerinin ve negatif karşılığının tanımladığı aralık baz alınarak normalize edilir ve normalize sinyaller 2^n seviyesine kuantize edilir. 3 farklı atlama değerinin ifade ettiği 3 kanaldan ilerleyen bu dönüşüm, kanalların katmanlar şeklinde birleştirilmesiyle düşey bir RGB görüntü temsilini oluşturur. Sinyalin her zaman penceresinden elde edilen bu düşey RGB imajlarının yan yana döşenmesinin sonucunda yatay eksenin zamanı ve düşey eksenin sinyal dalgalanmalarını temsil ettiği VarioGram olarak adlandırılan bir zaman-graf temsili elde edilmiş olur. Çevresel ses sınıflandırma problemlerinde sıklıkla kullanılan ESC-10 veri setindeki ses sinyallerinin dönüşümü ile elde edilen VarioGram temsilleri bir ResNet modeline girdi olarak verildiğinde %82.08’lik bir sınıflandırma başarısı elde edilmiş, mel-spectrogram görüntüleri ile hibritleştirilerek kullanılan VarioGram temsilleri ile bu başarı %93.33’e kadar çıkarılmıştır. Dolayısıyla VarioGram temsilleri, tek başına mel-spectrogram ile elde edilebilen en yüksek sınıflandırma başarısını küçük bir farkla iyileştirme yönünde etki yapmıştır.

Keywords

• Graf temsili
• Ses sınıflandırma
• Zaman serilerinin sınıflandırılması
• Karmaşık ağlar

VarioGram – A colorful time-graph representation for time series

Abstract

In this study, a framework for network-based representation of time series is presented. In the proposed method, initially, a segmentation procedure is completed by dividing the signals in the time domain into fixed-width time windows with 50% overlap. Each segment is normalized based on the range defined by the absolute maximum amplitude value of the main signal and its negative counterpart, and the normalized signals are quantized to 2^n levels. This transformation, proceeding through 3 channels expressed by 3 different jump values, generates a vertical RGB image representation by combining the channels in layers. As a result of tiling these vertical RGB images from each time window horizontally, a time-graph representation called VarioGram is obtained, where the horizontal axis represents time, and the vertical axis represents signal fluctuations. Feeding a ResNet model with VarioGram representations obtained by the transformation of the audio signals in the ESC-10 dataset which is frequently used in environmental sound classification problems, a classification success of 82.08% has been obtained, while this success has been 93.33% with the VarioGram representations hybridized with mel-spectrogram images. The VarioGram representations therefore acted to slightly improve the highest classification success achievable with the mel-spectrogram alone.

Keywords

• Graph representation
• Sound classification
• Time-series classification
• Complex networks

References

1. Ares, J., Lara, J. A., Lizcano, D., & Suarez, S. (2016). A soft computing framework for classifying time series based on fuzzy sets of events. Information Sciences, 330, 125–144. https://doi.org/10.1016/J.INS.2015.10.014
2. Bao, W., Yue, J., & Rao, Y. (2017). A deep learning framework for financial time series using stacked autoencoders and long-short term memory. PLoS ONE, 12.
3. Baydilli, Y. Y., Bayir, Ş., & Türker, I. (2017). A Hierarchical View of a National Stock Market as a Complex Network. Economic Computation & Economic Cybernetics Studies & Research, 51(1).
4. Canizo, M., Triguero, I., Conde, A., & Onieva, E. (2019). Multi-head CNN–RNN for multi-time series anomaly detection: An industrial case study. Neurocomputing, 363, 246–260. https://doi.org/10.1016/J.NEUCOM.2019.07.034
5. Chan, K.-P., & Fu, A. W.-C. (1999). Efficient time series matching by wavelets. Proceedings 15th International Conference on Data Engineering (Cat. No.99CB36337), 126–133. https://doi.org/10.1109/ICDE.1999.754915
6. Chen, Z., Zuo, W., Hu, Q., & Lin, L. (2015). Kernel sparse representation for time series classification. Information Sciences, 292, 15–26. https://doi.org/10.1016/J.INS.2014.08.066
7. Dafna, E., Tarasiuk, A., & Zigel, Y. (2018). Sleep staging using nocturnal sound analysis. Scientific Reports, 8(1), 13474. https://doi.org/10.1038/s41598-018-31748-0
8. Demir, S., & Türker, İ. (2021). Arithmetic success and gender-based characterization of brain connectivity across EEG bands. Biomedical Signal Processing and Control, 64, 102222. https://doi.org/10.1016/J.BSPC.2020.102222

9. Deng, W., Wang, G., & Xu, J. (2016). Piecewise two-dimensional normal cloud representation for time-series data mining. Information Sciences, 374, 32–50. https://doi.org/10.1016/J.INS.2016.09.027
10. Gharehbaghi, A., & Lindén, M. (2017). A deep machine learning method for classifying cyclic time series of biological signals using time-growing neural network. IEEE Transactions on Neural Networks and Learning Systems, 29(9), 4102–4115.
11. Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P.-A. (2019). Deep learning for time series classification: a review. Data Mining and Knowledge Discovery, 33(4), 917–963. https://doi.org/10.1007/s10618-019-00619-1
12. Kanani, P., & Padole, M. (2020). ECG Heartbeat Arrhythmia Classification Using Time-Series Augmented Signals and Deep Learning Approach. Procedia Computer Science, 171, 524–531. https://doi.org/10.1016/J.PROCS.2020.04.056
13. Lacasa, L., Luque, B., Ballesteros, F., Luque, J., & Nuño, J. C. (2008). From time series to complex networks: The visibility graph. Proceedings of the National Academy of Sciences, 105(13), 4972–4975. https://doi.org/10.1073/PNAS.0709247105
14. Lacasa, L., Nicosia, V., & Latora, V. (2015). Network structure of multivariate time series. Scientific Reports, 5(1), 15508. https://doi.org/10.1038/srep15508
15. Mushtaq, Z., Su, S. F., & Tran, Q. V. (2021). Spectral images based environmental sound classification using CNN with meaningful data augmentation. Applied Acoustics, 172, 107581. https://doi.org/10.1016/J.APACOUST.2020.107581
16. Peng, Z., Dang, J., Unoki, M., & Akagi, M. (2021). Multi-resolution modulation-filtered cochleagram feature for LSTM-based dimensional emotion recognition from speech. Neural Networks, 140, 261–273. https://doi.org/10.1016/J.NEUNET.2021.03.027
17. Piczak, K. J. (2015a). Environmental sound classification with convolutional neural networks. 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP), 1–6.
18. Piczak, K. J. (2015b). ESC: Dataset for Environmental Sound Classification. Proceedings of the 23rd ACM International Conference on Multimedia, 1015–1018. https://doi.org/10.1145/2733373.2806390
19. Pourbabaee, B., Roshtkhari, M. J., & Khorasani, K. (2018). Deep Convolutional Neural Networks and Learning ECG Features for Screening Paroxysmal Atrial Fibrillation Patients. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 48(12), 2095–2104. https://doi.org/10.1109/TSMC.2017.2705582
20. Ruiz, A. P., Flynn, M., Large, J., Middlehurst, M., & Bagnall, A. (2021). The great multivariate time series classification bake off: a review and experimental evaluation of recent algorithmic advances. Data Mining and Knowledge Discovery, 35(2), 401–449. https://doi.org/10.1007/s10618-020-00727-3
21. Salamon, J., Jacoby, C., & Bello, J. P. (2014). A Dataset and Taxonomy for Urban Sound Research. Proceedings of the 22nd ACM International Conference on Multimedia, 1041–1044. https://doi.org/10.1145/2647868.2655045
22. Sharan, R. v, & Moir, T. J. (2015). Cochleagram image feature for improved robustness in sound recognition. 2015 IEEE International Conference on Digital Signal Processing (DSP), 441–444. https://doi.org/10.1109/ICDSP.2015.7251910
23. Soares, E., Costa, P., Costa, B., & Leite, D. (2018). Ensemble of evolving data clouds and fuzzy models for weather time series prediction. Applied Soft Computing, 64, 445–453. https://doi.org/10.1016/J.ASOC.2017.12.032
24. Türker, İ., & Aksu, S. (2022). Connectogram – A graph-based time dependent representation for sounds. Applied Acoustics, 191, 108660. https://doi.org/10.1016/J.APACOUST.2022.108660
25. Türker, İ., Şehirli, E., & Demiral, E. (2016). Uncovering the differences in linguistic network dynamics of book and social media texts. SpringerPlus, 5(1), 864. https://doi.org/10.1186/s40064-016-2598-2
26. Türker, İ., & Sulak, E. E. (2018). A multilayer network analysis of hashtags in twitter via co-occurrence and semantic links. International Journal of Modern Physics B, 32(04), 1850029. https://doi.org/10.1142/S0217979218500297
27. Yin, J., Liu, Z., Jin, Z., & Yang, W. (2012). Kernel sparse representation based classification. Neurocomputing, 77(1), 120–128. https://doi.org/10.1016/J.NEUCOM.2011.08.018
28. Zhang Zhichao and Xu, S. and C. S. and Z. S. (2018). Deep Convolutional Neural Network with Mixup for Environmental Sound Classification. In C.-L. and C. X. and Z. J. and T. T. and Z. N. and Z. H. Lai Jian-Huang and Liu (Ed.), Pattern Recognition and Computer Vision (pp. 356–367). Springer International Publishing.