Classification of UML Diagrams from Images in Software Engineering Using Deep Learning

Abstract

Software design is a fundamental phase in software engineering, where Unified Modeling Language (UML) diagrams play a critical role in representing system structures and behavior for software engineers. However, UML diagrams are often available only as images, which limits their automated processing and integration into tools such as search engines, software documentation systems, and accessibility solutions for visually impaired users. In this study, we propose a deep learning approach for automatically classifying UML diagrams from images. A lightweight convolutional neural network (CNN) model is designed with a minimal number of layers and reduced trainable parameters, providing an efficient solution without requiring data augmentation or transfer learning. The model is trained to accurately classify six UML diagram types: Activity, Component, Class, Sequence, Deployment, and Use-Case. Experimental evaluations on a UML image dataset demonstrated a high classification accuracy of 94.61%, highlighting its strong predictive capability despite its compact design.

Keywords

• Artificial Intelligence
• Classification
• Deep Learning
• Software Engineering
• Unified Modeling Language (UML) Diagrams.

References

1. [1]. Rumpe, B. Modeling with UML: Language, Concepts, Methods; Springer International Publishing: Cham, Switzerland, 2016; pp 281.
2. [2]. Hernandez, JCS, Sanchez-García, AJ, Alonso-Ramírez, O. 2021. Features Used to Classify UML Diagrams from Images: a Systematic Literature Review. Research in Computing Science; 150(11): 197–205.
3. [3]. Polischuk, V. 2025. Automatic recognition of UML diagrams in images: Approaches, trends, and challenges. Technologies and Engineering; 26(1): 23-35.
4. [4]. Nedelcu, IG, Mocanu, SA, Saru, D, Caramihai, SI, Ionita, AD. Performance of Neural Networks for Recognizing Images of UML Class, Sequence and State Diagrams. In: Graf, S, Markos, A. (eds) Generative Systems and Intelligent Tutoring Systems. Lecture Notes in Computer Science, vol 15724, Springer, Cham, Switzerland, 2026, pp 104–113.
5. [5]. Lakehal, A, Alti, A, Annane, B, Imene, B, Afaf, SA., Meriem, M. Multi-classification and Multi-labeling of UML Diagrams Using Transfer Learning and CNN. In: Hatti, M. (ed) Smart Computing and Control Renewable Energy Systems. Lecture Notes in Networks and Systems, vol 1238, Springer, Cham, Switzerland, 2025, pp 573–583.
6. [6]. Shehzadi, N, Ferzund, J, Fatima, R, Riaz, A. 2025. Automatic Complexity Analysis of UML Class Diagrams Using Visual Question Answering (VQA) Techniques. Software; 4: 1-29.
7. [7]. Kour, M, Hains, G, Saber, M. 2025. Diagram Analysis for Visually Impaired Readers. Hyper Articles en Ligne (HAL) Science; 2025: pp. 1-9.
8. [8]. Nedelcu, I-G, Ionita, AD. 2024. Evaluating the Conformity to Types of Unified Modeling Language Diagrams with Feature-Based Neural Networks. Applied Sciences; 14(20): 1-22.

9. [9]. Torcal, J, Moreno, V, Llorens, J, Granados, A. 2024. Creating and Validating a Ground Truth Dataset of Unified Modeling Language Diagrams Using Deep Learning Techniques. Applied Sciences; 14(23): 1-12.
10. [10]. Yuan, Z, Ma, Z. 2023. Supervised Classification of UML Class Diagrams Based on F-KNB. International Journal of Software Engineering and Knowledge Engineering; 33(8): 1169-1210.
11. [11]. Bnouni Rhim, N, Cheballah, S, Ben Mabrouk, M. Cross Synergetic Mobilenet-VGG16 for UML Multiclass Diagrams Classification. In: Abraham, A., Bajaj, A., Gandhi, N., Madureira, A.M., Kahraman, C. (eds) Lecture Notes in Networks and Systems, vol 649, Springer, Cham, Switzerland, 2023, pp 24-30.
12. [12]. Nedelcu, I-G, Caramihai, SI, Mocanu, S, Ionita, AD. 2023. UML Diagrams Classification with MIMO and SISO Neural Networks. University Politehnica of Bucharest Scientific Bulletin Series C; 85 (4): 89-100.
13. [13]. Chen F, Zhang L, Lian X, Niu N. 2022. Automatically recognizing the semantic elements from UML class diagram images. The Journal of Systems & Software; 193: 1–19.
14. [14]. Bergström G, Hujainah F, Ho-Quang T, Jolak R, Rukmono SA, Nurwidyantoro A, Chaudron MRV. 2022. Evaluating the layout quality of UML class diagrams using machine learning. The Journal of Systems & Software; 192: 1–20.
15. [15]. Gosala B, Chowdhuri SR, Singh J, Gupta M, Mishra A. 2021. Automatic classification of UML class diagrams using deep learning technique: Convolutional neural network. Applied Sciences; 11(9): 1–14.
16. [16]. Shcherban S, Liang P, Li Z, Yang C. 2021. Multiclass classification of UML diagrams from images using deep learning. International Journal of Software Engineering and Knowledge Engineering; 31(11): 1683–1698.
17. [17]. Shcherban S, Liang P, Li Z, Yang C. Multiclass classification of four types of UML diagrams from images using deep learning. Proceedings of the 33rd International Conference on Software Engineering and Knowledge Engineering, Pittsburgh, USA, 1-10 July 2021, pp 1-6.
18. [18]. Tavares JF, Costa YMG, Colanzi TE. Classification of UML Diagrams to Support Software Engineering Education, Proceedings of 36th IEEE/ACM International Conference on Automated Software Engineering Workshops, Melbourne, Australia, 15-19 November 2021, pp 102-107.
19. [19]. Moreno V, Genova G, Alejandres M, Fraga A. 2020. Automatic classification of web images as UML static diagrams using machine learning techniques. Applied Sciences; 10(7): 1–24.
20. [20]. Best N, Ott J, Linstead EJ. 2020. Exploring the efficacy of transfer learning in mining image-based software artifacts. Journal of Big Data; 7(59): 1–10.