Fusion of Image Transformation Techniques for IoT-based Multivariate Time-Series

Abstract

The emergence of the Internet of Things (IoT) has ushered in a new era of data generation with the opportunity for data to become a key element of connected devices. This study investigates new methods to bridge the realms of multivariate time-series data and image analysis, paying special attention to Gramian Angular Summation Field (GASF), Gramian Angular Difference Field (GADF), Markov Transition Field (MTF), and Recurrence Plot (RP) transformation techniques. These techniques serve to convert raw time-series data into visual representations, laying the foundation for deeper analysis and predictive modeling. The study introduces a novel paradigm by not only employing individual image transformation techniques but also fusing them in both horizontal and square orientations. By leveraging Convolutional Neural Networks (CNNs), this study demonstrates the efficiency of innovative fused-oriented image transformation techniques in predicting complex patterns within a multivariate time-series dataset related to electricity distribution and transformer oil temperature. The experimental results indicate that the Fused-Horizontal image transformation technique, using the order RP - GADF - MTF - GASF, yields the best performance, achieving the lowest MSE of 0.01047, RMSE of 0.10235, and MAE of 0.08054. Additionally, the order RP - GADF - GASF - MTF results in the lowest MAPE of 0.21997, outperforming both Fused-Square techniques and individual methods like GASF, GADF, MTF, and RP. These findings underscore the potential of fused image transformation techniques in improving prediction accuracy, offering a significant advancement over traditional methods.

Keywords

• IoT
• Time-series data
• Image transformation
• Fusion
• Deep learning

References

1. [1] Hu, C., Sun, Z., Li, C., Zhang, Y., and Xing, C., “Survey of Time-series Data Generation in IoT”, Sensors, 23(15): 6976–6976, (2023).
2. [2] Kashani, M. H., Madanipour, M., Nikravan, M., Asghari, P., and Mahdipour, E., “A systematic review of IoT in healthcare: Applications, techniques, and trends”, Journal of Network and Computer Applications, 192: 1-41, (2021).
3. [3] Bedi, G., Venayagamoorthy, G. K., Singh, R., Brooks, R. R., and Wang, K. C., “Review of Internet of Things (IoT) in electric power and energy systems”, IEEE Internet of Things Journal, 5(2): 847-870, (2018).
4. [4] Oguz, F. E., Ekersular, M. N., Sunnetci, K. M., and Alkan, A., “Enabling Smart Agriculture: An IoT-Based Framework for Real-Time Monitoring and Analysis of Agricultural Data”, Agricultural Research, 13: 574-585, (2024).
5. [5] Khanna, A., Kaur, S., “Internet of things (IoT), applications and challenges: a comprehensive review”, Wireless Personal Communications, 114: 1687-1762, (2020).
6. [6] Estebsari, A., Rajabi, R., “Single residential load forecasting using deep learning and image encoding techniques”, Electronics, 9: 1-17, (2020).
7. [7] Ye, X., Huang, Y., Bai, Z., and Wang, Y., “A novel approach for sports injury risk prediction: based on time-series image encoding and deep learning”, Frontiers in Physiology, 14: 1-14, (2023).
8. [8] Baldini, G., Giuliani, R., and Dimc, F., “Physical layer authentication of Internet of Things wireless devices using Convolutional Neural Networks and Recurrence Plots”, Internet Technology Letters, 2: 1-6, (2018).

9. [9] Ferraro, A., Galli, A., Moscato, V., and Sperlí, G., “A novel approach for predictive maintenance combining GAF encoding strategies and deep networks”, In 2020 IEEE 6th International Conference on Dependability in Sensor, Cloud and Big Data Systems and Application (DependSys), IEEE, (2020).
10. [10] Hammoud, M., Kovalenko, E., Somov, A., Bril, E., and Baldycheva, A., “Deep learning framework for neurological diseases diagnosis through near-infrared eye video and time-series imaging algorithms”, Internet of Things, 24: 1-20, (2023).
11. [11] Wang, C.-C., Kuo, C.-H., “Detecting dyeing machine entanglement anomalies by using time-series image analysis and deep learning techniques for dyeing-finishing process”, Advanced Engineering Informatics, 55: 1-10, (2023).
12. [12] Deng, X., Ping, Z., and Sun, R., "UWB NLOS Recognition Based on Improved Convolutional Neural Network Assisted by Wavelet Analysis and Gramian Angular Field", IEEE Sensors Journal, 23(14): 16384-16392, (2023).
13. [13] Lee, H., Lee, J., “Convolutional Model with a Time-series Feature Based on RSSI Analysis with the Markov Transition Field for Enhancement of Location Recognition”, Sensors, 23(7): 1-16, (2023).
14. [14] Abidi, A., Ienco, D., Abbes, A. B., and Farah, I. R., “Combining 2D encoding and convolutional neural network to enhance land cover mapping from Satellite Image Time-series”, Engineering Applications of Artificial Intelligence, 122: 1-17, (2023).
15. [15] Yang, C. L., Yang, C. Y., Chen, Z. X., and Lo, N. W., “Multivariate time series data transformation for convolutional neural network”, In 2019 IEEE/SICE International Symposium on System Integration (SII), IEEE, (2019).
16. [16] Jiang, W., Zhang, D. Ling, L., and Lin, R., “Time-series Classification Based on Image Transformation Using Feature Fusion Strategy”, Neural Processing Letters, 54: 1-22, (2022).
17. [17] Cheng, Y., Lu, M., Gai, X., Guan, R., Zhou, S., and Xue, J., “Research on multi-signal milling tool wear prediction method based on GAF-ResNext”, Robotics and Computer-Integrated Manufacturing, 85: 1-15, (2024).
18. [18] Mitiche, I., Morison, G., Nesbitt, A., Hughes-Narborough, M., Stewart, B., and Boreham, P., “Imaging Time-series for the Classification of EMI Discharge Sources”, Sensors, 18(9): 1-17, (2018).
19. [19] Yan, J., Kan, J., and Luo, H., “Rolling Bearing Fault Diagnosis Based on Markov Transition Field and Residual Network”, Sensors, 22(10): 3936–3936, (2022).
20. [20] Chen, Y., Su, S., and Yang, H., “Convolutional Neural Network Analysis of Recurrence Plots for Anomaly Detection”, International Journal of Bifurcation and Chaos, 30(1): 201-213, (2020).
21. [21] “ETDataset/ETT-small at main · zhouhaoyi/ETDataset,” GitHub. https://github.com/zhouhaoyi/ETDataset/tree/main/ETT-small. Accessed Date: 02 March 2024
22. [22] Jiang, J. R., Yen, and C. T., “Product quality prediction for wire electrical discharge machining with markov transition fields and convolutional long short-term memory neural networks”, Applied Sciences, 11(13): 1-13, (2021).
23. [23] Plevris, V., Solorzano, G., Bakas, N., and Seghier, M. B., “Investigation of performance metrics in regression analysis and machine learning-based prediction models”, 8th European Congress on Computational Methods in Applied Sciences and Engineering, (2022).