Comparison of Multi-Label Learning Approaches for Pie Chart Image Classification Using Deep Learning

Year 2021, Volume: 1 Issue: 2, 116 - 124, 30.12.2021

Derya Bırant , Cem Kösemen

Abstract

A pie chart is a powerful and circular information graphic used to display numerical proportions to the whole. However, the properties of pie charts cannot be directly noticed by machines since they are usually in an image format. To make a pie chart classifiable by machines, this paper proposes a novel solution using deep learning methods. This study is original in that it automatically and jointly classifies charts in terms of two respects: shape (pie or donut) and dimension (2D or 3D). This is the first study that compares two multi-label learning approaches to classify pie charts: binary-class-based convolutional neural networks (BCNN) and multi-class- based convolutional neural networks (MCNN). The experimental results showed that the BCNN model achieved 86% accuracy, while the MCNN model reached 85% accuracy on real-world pie chart data.

Keywords

Convolutional neural networks, deep learning, image classification, machine learning, multi-label learning, pie charts

References

• [1] F. Bajic and J. Job, “Chart classification using Siamese CNN,” Journal of Imaging, vol. 7, no. 11, pp. 1-18, 2021.
• [2] P. Mishra, S. Kumar, and M. K. Chaube, “ChartFuse: a novel fusion method for chart classification using heterogeneous microstructures,”, Multimedia Tools and Applications, vol. 80, no. 7, pp. 10417-10439, 2021.
• [3] J. Fu, B. Zhu, W. Cui, S. Ge, Y. Wang, H. Zhang, H. Huang, Y. Tang, D. Zhang, and X. Ma. “Chartem: reviving chart images with data embedding,” IEEE Transactions on Visualization and Computer Graphics, vol. 27, no. 2, pp. 337-346, 2021.
• [4] C-S. Cheng, Y. Ho, and T-C. Chiu, “End-to-End control chart pattern classification using a 1D convolutional neural network and transfer learning,” Processes, vol. 9, no. 9, pp. 1-26, 2021.
• [5] Y. Zheng, Y-W. Si, and R. Wong, “Feature extraction for chart pattern classification in financial time series,” Knowledge and Information Systems, vol. 63, no. 7, pp. 1807-1848, 2021.
• [6] C. Sohn, H. Choi, K. Kim, J. Park, and J. Noh, “Line chart understanding with convolutional neural network,” Electronics, vol. 10, no. 6, pp. 1-17, 2021.
• [7] F. Zhou, Y. Zhao, W. Chen, Y. Tan, Y. Xu, Y. Chen, C. Liu, and Y. Zhao, “Reverse-engineering bar charts using neural networks,” Journal of Visualization, vol. 24, no. 2, pp. 419-435, 2021.
• [8] R. Ünlü, “A robust data simulation technique to improve early detection performance of a classifier in control chart pattern recognition systems,” Information Sciences, vol. 548, pp. 18-36, 2021.
• [9] M. Zaman, and A. Hassan, “Fuzzy heuristics and decision tree for classification of statistical feature-based control chart patterns,” Symmetry, vol. 13, no. 1, pp. 1-12, 2021.
• [10] M. Siper, K. Makinen, and R. Kanan, “TABot - a distributed deep learning framework for classifying price chart images,” Advanced Computing, vol. 1367, pp. 465-473, 2021.
• [11] T. Araujo, P. Chagas, J. Alves, C. Santos, B. Santos, and B. Meiguins, “A real-world approach on the problem of chart recognition using classification, detection and perspective correction,” Sensors, vol. 20, no. 16, pp. 1-21, 2020.
• [12] S. Birogul, G. Temür, and U. Kose, “YOLO object recognition algorithm and buy-sell decision model over 2D candlestick charts,” IEEE Access, vol. 8, pp. 91894-91915, 2020.
• [13] F. Bajic, J. Job, and K. Nenadic, “Data visualization classification using simple convolutional neural network model,” International Journal of Electrical and Computer Engineering Systems, vol. 11, no. 1, pp. 43-51, 2020.
• [14] F. Bajic, J. Job, and K. Nenadic, “Chart classification using simplified VGG model,” In International Conference on Systems Signals and Image Processing, Osijek, Croatia, 2019, pp. 229-233.
• [15] W. Dai, M. Wang, Z. Niu, and J. Zhang, “Chart decoder: generating textual and numeric information from chart images automatically,” Journal of Visual Languages & Computing, vol. 48, pp. 101-109, 2018.
• [16] J. Poco and J. Heer, “Reverse-engineering visualizations: recovering visual encodings from chart images,” Computer Graphics, vol. 36, no. 3, pp. 353-363, 2017.
• [17] B. Tang, X. Liu, J. Lei, M. Song, D. Tao, S. Sun, and F. Dong, “DeepChart: combining deep convolutional networks and deep belief networks in chart classification,” Signal Process, vol. 124, pp. 156-161, 2016.
• [18] J. Shtok, S. Harary, O. Azulai, A. R. Goldfarb, A. Arbelle, and L. Karlinsky, “CHARTER: heatmap-based multi-type chart data extraction,” In Document Intelligence Workshop at KDD, 2021, pp. 1-5.
• [19] M-L. Zhang and Z-H. Zhou, “A review on multi-label learning algorithms,” IEEE Transactions on Knowledge and Data Engineering, vol. 26, no. 8, pp. 1819-1837, 2014.
• [20] S. H. S. Basha, S. R. Dubey, V. Pulabaigari, and S. Mukherjee, “Impact of fully connected layers on performance of convolutional neural networks for image classification,” Neurocomputing, vol. 378, pp. 112-119, 2020.