Deep Learning Based Offline Handwritten Signature Recognition

Year 2024, , 871 - 884, 26.09.2024

Bahar Çiftçi , Ramazan Tekin

https://doi.org/10.17798/bitlisfen.1527670

Abstract

In our digitalized world, the need for reliable authentication methods is steadily increasing. Biometric authentication methods are divided into two main categories: physiological and behavioral. While physiological biometrics include features such as face, iris, and fingerprint, behavioral biometrics encompass dynamics such as gait, speech, and signature. Most of these methods require specialized equipment, whereas signatures can be easily obtained without additional tools, making them ideal for verifying the legality of documents. Although manual signature recognition is effective, it is resource-intensive, slow, and susceptible to errors. With advancements in technology, the need to automate the signature recognition process to enhance accuracy and efficiency has become increasingly important. Based on this need, in this study, five different DL techniques (GoogLeNet, MobileNet-V3 Large, Inception-V3, ResNet50 and EfficientNet-B0) are used to classify signature images with detailed analyses. DL methods have outperformed traditional techniques by leveraging the power of CNNs to automatically learn and extract complex features from signature data. The dataset used consists of a total of 12,600 images belonging to 420 individuals, each contributing 30 original signatures. The dataset is divided into training, validation, and test sets in different proportions to analyze classification performance. The pre-trained DL models were fine-tuned to optimize their parameters for the signature dataset. The results demonstrate that DL models achieve high accuracy in signature classification, with the GoogLeNet and Inception-V3 models reaching an accuracy of 98.77% at a 20% test rate. The study also highlights the impact of different test rates on model performance.

Keywords

Signature Recognition, Image Classification, Deep Learning, CNN Architectures

Ethical Statement

The study is complied with research and publication ethics

References

• [1] A. K. Jain, K. Nandakumar, and A. Ross, “50 years of biometric research: Accomplishments, challenges, and opportunities,” Pattern Recognit Lett, vol. 79, pp. 80–105, Aug. 2016, doi: 10.1016/J.PATREC.2015.12.013.
• [2] M. A. Ferrer, J. F. Vargas, A. Morales, and A. Ordóñez, “Robustness of offline signature verification based on gray level features,” IEEE Transactions on Information Forensics and Security, vol. 7, no. 3, pp. 966–977, 2012, doi: 10.1109/TIFS.2012.2190281.
• [3] N. S. Kamel, S. Sayeed, and G. A. Ellis, “Glove-Based Approach to Online Signature Verification,” IEEE Trans Pattern Anal Mach Intell, vol. 30, no. 6, pp. 1109–1113, Jun. 2008, doi: 10.1109/TPAMI.2008.32.
• [4] Z. Wang, M. Muhammat, N. Yadikar, A. Aysa, and K. Ubul, “Advances in Offline Handwritten Signature Recognition Research: A Review,” IEEE Access, vol. 11, pp. 120222–120236, 2023, doi: 10.1109/ACCESS.2023.3326471.
• [5] A. Rıza Yılmaz et al., “İmza Tanıma Uygulaması için Çok Katmanlı Algılayıcıların Diferansiyel Gelişim Algoritması ile Eğitimi Training Multilayer Perceptron Using Differential Evolution Algorithm for Signature Recognition Application,” 2013.
• [6] R. Sabourin and Drouhard Jean-Pierre, “Offline signature verification using directional pdf and neural networks,” International Conference on Pattern Recognition, 1994.
• [7] M. A. Ismail and S. Gad, “Off-line arabic signature recognition and verification,” Pattern Recognit, vol. 33, no. 10, pp. 1727–1740, Oct. 2000, doi: 10.1016/S0031-3203(99)00047-3.
• [8] M. R. Deore and S. M. Handore, “Offline signature recognition: Artificial neural network approach,” in 2015 International Conference on Communications and Signal Processing (ICCSP), 2015, vol. 2, pp. 1708–1712.
• [9] NN. Calik, O. C. Kurban, A. R. Yilmaz, L. D. Ata, and T. Yildirim, “Signature recognition application based on deep learning,” in 2017 25th Signal Processing and Communications Applications Conference (SIU), 2017, vol. 20, pp. 1–4.
• [10] MM. K. Kalera, S. Srihari, and A. Xu, “Offline signature verification and identification using distance statistics,” Intern. J. Pattern Recognit. Artif. Intell., vol. 18, no. 07, pp. 1339–1360, 2004.
• [11] D. Gumusbas and T. Yildirim, “Offline signature identification and verification using capsule network,” in 2019 IEEE International Symposium on INnovations in Intelligent SysTems and Applications (INISTA), 2019.
• [12] D. Gumusbas and T. Yildirim, “Offline signature identification and verification based on capsule representations,” Cybernetics and Information Technologies, vol. 20, no. 5, pp. 60–67, Dec. 2020, doi: 10.2478/CAIT-2020-0040.
• [13] OO. Tarek and A. Atia, “Forensic handwritten signature identification using deep learning,” in 2022 IEEE 9th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (SETIT), 2022, vol. 39, pp. 185–190.
• [14] A. Jain, S. K. Singh, and K. Pratap Singh, “Multi-task learning using GNet features and SVM classifier for signature identification,” IET Biom, vol. 10, no. 2, pp. 117–126, Mar. 2021, doi: 10.1049/BME2.12007.
• [15] A. Foroozandeh, A. Askari Hemmat, and H. Rabbani, “Offline handwritten signature verification and recognition based on deep transfer learning,” in 2020 International Conference on Machine Vision and Image Processing (MVIP), 2020.
• [16] F. Özyurt, J. Majidpour, T. A. Rashid, and C. Koç, “Offline Handwriting Signature Verification: A Transfer Learning and Feature Selection Approach,” Traitement du Signal vol.40, no.6, 2024.
• [17] S. Pokharel, S. Giri, and S. Shakya, “Deep Learning Based Handwritten Signature Recognition,” NCE Journal of Scince and Engineering, vol.1, no.1, pp. 21-24, 2020.
• [18] C. Szegedy et al., “Going deeper with convolutions,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
• [19] A. Howard et al., “Searching for MobileNetV3,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
• [20] C.C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
• [21] K.K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
• [22] M. Tan and Q. V. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” 36th International Conference on Machine Learning, ICML 2019, vol. 2019-June, pp. 10691–10700, May 2019, Accessed: Jul. 25, 2024. [Online]. Available: https://arxiv.org/abs/1905.11946v5
• [23] OO. El Melhaoui, S. Said, A. Benlghazi, and S. Elouaham, “Improved signature recognition system based on statistical features and fuzzy logic,” e-Prime - Advances in Electrical Engineering, Electronics and Energy, vol. 8, no. 100505, p. 100505, 2024.
• [24] M. Jampour, S. Abbaasi, and M. Javidi, “CapsNet regularization and its conjugation with ResNet for signature identification,” Pattern Recognit, vol. 120, p. 107851, Dec. 2021, doi: 10.1016/J.PATCOG.2021.107851.
• [25] K. Kancharla, V. Kamble, and M. Kapoor, “Handwritten signature recognition: A convolutional neural network approach,” in 2018 International Conference on Advanced Computation and Telecommunication (ICACAT), 2018, vol. 2709, pp. 1–5.
• [26] F. Noor, A. E. Mohamed, F. A. S. Ahmed, and S. K. Taha, “Offline handwritten signature recognition using convolutional neural network approach,” in 2020 International Conference on Computing, Networking, Telecommunications & Engineering Sciences Applications (CoNTESA), 2020, pp. 51–57.
• [27] S. Mshir and M. Kaya, “Signature Recognition Using Machine Learning,” in 2020 8th International Symposium on Digital Forensics and Security (ISDFS), 2020.
• [28] E. K. D. Kette, D. R. Sina, and B. S. Djahi, “Digital image processing: Offline handwritten signature identification using local binary pattern and rotational invariance local binary pattern with learning vector quantization,” J. Phys. Conf. Ser., vol. 2017, no. 1, p. 012011, 2021.
• [29] YY. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.
• [30] S. Armand, M. Blumenstein, and V. Muthukkumarasamy, “Off-line signature verification using the enhanced modified direction feature and neural-based classification,” in The 2006 IEEE International Joint Conference on Neural Network Proceedings, 2006, pp. 684–691.
• [31] A.A. A. M. Abushariah, T. S. Gunawan, J. Chebil, and M. A. M. Abushariah, “Automatic person identification system using handwritten signatures,” in 2012 International Conference on Computer and Communication Engineering (ICCCE), 2012, pp. 560–565.
• [32] M. V. Arısoy, “Signature verification using siamese neural network one-shot learning,” International Journal of Engineering and Innovative Research, vol. 3, no. 3, pp. 248–260, Sep. 2021, doi: 10.47933/IJEIR.972796.