SIGNATURE VERIFICATION USING SIAMESE NEURAL NETWORK ONE-SHOT LEARNING

Abstract

With the acceleration of digitalization in all areas of our lives, the need for biometric verification methods is increasing. The fact that biometric data is unique and biometric verification is stronger against phishing attacks compared to password-based authentication methods, has increased its preference rate. Signature verification, which is one of the biometric verification types, plays an important role in many areas such as banking systems, administrative and judicial applications. There are 2 types of signature verification, online and offline, for identifying the identity of the person and detecting signature forgery. Online signature verification is carried out during signing and temporal dynamic data are available regarding the person's signature. Offline verification is applied by scanning the image after signing, and this verification is limited to spatial data. Therefore, the offline signature verification process is considered a more challenging task. In this study, offline signature verification, independent of the writer, based on One-Shot Learning, was performed using Siamese Neural Network. Due to the fact that the Deep Convolution Neural Network requires a large amount of labeled data for image classification, real and fake signature distinction has been achieved by using the One-Shot Learning method, which can perform a successful classification by using less numbers of signature images. As a result of the experiments conducted on signature datasets, using the Siamese architecture, the proposed approach achieved percentage accuracy of 93.23, 92.11, 89.78, 91.35 verification in 4NSigComp2012, SigComp2011, 4NSigComp2010 and BHsig260 respectively.

Keywords

• Offline Signature Verification
• Siamese Neural Network
• One-Shot Learning.
• Deep Learning
• Machine Learning

Siamese Sinir Ağı One-Shot Öğrenmeyi Kullanarak İmza Doğrulama

Abstract

Dijitalleşmenin hayatımızın her alanında hızlanmasıyla birlikte biyometrik doğrulama yöntemlerine olan ihtiyaç da artmaktadır. Biyometrik verilerin benzersiz olması ve biyometrik doğrulamanın e-dolandırıcılık saldırılarına karşı parola tabanlı doğrulama yöntemlerine göre daha güçlü olması tercih oranını artırmıştır. Biyometrik doğrulama türlerinden olan imza doğrulama, bankacılık sistemleri, idari ve adli uygulamalar gibi birçok alanda önemli rol oynamaktadır. Kişinin kimliğini ve imza sahteciliğini tespit etmek için çevrimiçi ve çevrimdışı olmak üzere 2 tür imza doğrulaması vardır. İmzalama sırasında çevrimiçi imza doğrulaması yapılır ve kişinin imzasına ilişkin zamansal dinamik veriler mevcuttur. Çevrimdışı doğrulama, imzalandıktan sonra görüntü taranarak uygulanır ve bu doğrulama mekansal verilerle sınırlıdır. Bu nedenle, çevrimdışı imza doğrulama süreci daha zorlu bir görev olarak kabul edilir. Bu çalışmada, Siyam Sinir Ağı kullanılarak yazardan bağımsız, One-Shot Learning tabanlı çevrimdışı imza doğrulaması yapılmıştır. Derin Evrişim Sinir Ağı'nın görüntü sınıflandırması için büyük miktarda etiketli veri gerektirmesi nedeniyle, daha az sayıda imza kullanarak başarılı bir sınıflandırma yapabilen One-Shot Learning yöntemi kullanılarak gerçek ve sahte imza ayrımı sağlanmıştır. Siyam mimarisi kullanılarak imza veri setleri üzerinde yapılan deneyler sonucunda, sırasıyla 4NSigComp2012, SigComp2011, 4NSigComp2010 ve BHsig260'da 93.23, 92.11, 89.78, 91.35 doğrulama yüzdesi doğruluğu elde etmiştir.

Keywords

• Çevrimdışı İmza Doğrulama
• Siamese Sinir Ağı
• One-shot Öğrenme
• Makine Öğrenmesi
• Derin Öğrenme

References

1. 1. Gokul, N. B., & Sankaran, S. (2020). Identity Based Security Framework For Smart Cities. In 2020 IEEE International Conference on Advanced Networks and Telecommunications Systems (ANTS) (pp. 1-4). IEEE. 2. Ullah, Z., Al-Turjman, F., Mostarda, L., & Gagliardi, R. (2020). Applications of artificial intelligence and machine learning in smart cities. Computer Communications, 154, 313-323.
2. 3. Ghosh, S., Ghosh, S., Kumar, P., Scheme, E., Roy, P.P. (2021). A novel spatio-temporal Si amese network for 3D signature recognition. Pattern Recognition Letters. 144, 13-20.
3. 4. Jain, S., Khanna, M., Singh, A. (2021). Comparison among different CNN Architectures for Signature Forgery Detection using Siamese Neural Network. In: 2021 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS), pp. 481-486. IEEE Press, Greater Noida, India
4. 5. Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Ortega-Garcia, J. (2021). DeepSign: Deep On-Line Signature Verification. Ieee Transactions On Biometrics, Behavior, And Identity Science. 3, 229-239
5. 6. Jain, A., Singh, S.K., Singh, K.P. (2020). Handwritten signature verification using shallow convolutional neural network. Multimed Tools Appl. 79, 19993-20018.
6. 7. Jagtap A.B., Sawat D.D., Hegadi R.S., Hegadi R.S. (2019). Siamese Network for Learning Genuine and Forged Offline Signature Verification. In: Recent Trends in Image Processing and Pattern Recognition. RTIP2R 2018, pp. 131-139.
7. 8. Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Ortega-Garcia, J. (2017). Biometric Signature Verification Using Recurrent Neural Networks. In: 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), pp. 652-657. Kyoto, Japan
8. 9. Dey, S., Dutta, A., Toledo, J.I., Ghosh, S.K., Llados, J., Pal, U. (2017). SigNet: Convolutional Siamese Network for Writer Independent Offline Signature Verification. Pattern Recognition Letters. 1-7.

9. 10. Ruiz, V., Linares, I., Sanchez, A., Velez, J.F. (2020). Off-line handwritten signature verification using compositional synthetic generation of signatures and Siamese Neural Networks. Neurocomputing. 374, 30-41.
10. 11. Ghosh, R. (2021). A Recurrent Neural Network based deep learning model for offline signature verification and recognition system. Expert Systems with Applications. 168.
11. 12. Chakladar, D.D., Kumar, P., Roy, P.P., Dogra, D.P., Scheme, E., Chang, V. (2021). A multimodal-Siamese Neural Network (mSNN) for person verification using signatures and EEG. Information Fusion. 71, 17-27.
12. 13. Yapıcı, M.M., Tekerek, A., Topaloğlu, N. (2021). Deep learning-based data augmentation method and signature verification system for offline handwritten signature. Pattern Anal Applic. 24, 165-179.
13. 14. Tahir, N.M., Ausat, A.N., Bature, U.I., Abubakar, K.A., Gambo, I. (2021). Off-line Handwritten Signature Verification System: Artificial Neural Network Approach. 1, 45.57.
14. 15. Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Ortega-Garcia, J. (2018). Exploring Recurrent Neural Networks for On-Line Handwritten Signature Biometrics. in IEEE Access. 6, 5128-5138
15. 16. Hefny, A., Moustafa, M. (2019). Online Signature Verification Using Deep Learning and Feature Representation Using Legendre Polynomial Coefficients. In: n book: The International Conference on Advanced Machine Learning Technologies and Applications AMLTA, pp. 689-697.
16. 17. Okawa, M.: Time-series averaging and local stability-weighted dynamic time warping for online signature verification. Pattern Recognition. 112, (2021)
17. 18. Yılmaz, M.B., Yanıkoğlu, B. (2016). Score level fusion of classifiers in off-line signature verification. Information Fusion. 32, 109-119.
18. 19. Calik, N., Kurban, O.C., Yilmaz, A.R., Yıldırım, T., Durak, A.L. (2019). Large-scale offline signature recognition via deep neural networks and feature embedding. Neurocomputing, 359, 1-14.
19. 20. Zois, E.N., Alewijnse, L., Economou, G. (2016). Offline signature verification and quality characterization using poset-oriented grid features. Pattern Recognition. 54, 162-177.
20. 21. Manjunatha, K.S., Manjunath, S., Guru, D.S., Somashekara, M.T. (2016). Online signature verification based on writer dependent features and classifiers. Pattern Recognition Letters. 80, 129-136.
21. 22. Zois, E.N., Theodorakopoulos, I., Tsourounis, D., Economou, G. (2017). Parsimonious Coding and Verification of Offline Handwritten Signatures. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 636—645. Honolulu, HI, USA.
22. 23. Guerbai, Y., Chibani, Y., Hadjadji, B. (2015). The effective use of the one-class SVM classifier for handwritten signature verification based on writer-independent parameters. Pattern Recognit., 48, 103-113.
23. 24. Hafemann, L.G., Sabourin, R., Oliveira, L.S. (2019). Characterizing and Evaluating Adversarial Examples for Offline Handwritten Signature Verification. IEEE Transactions on Information Forensics and Security. 14, 2153-2166.
24. 25. Shah, A.S., Khan, M.A., Subhan, F., Fayaz, M., Shah, A. (2016). An offline signature verification technique using pixels intensity levels. International Journal of Signal Processing, Image Processing and Pattern Recognition. 9, 205-222.
25. 26. Jain, V., Chaudhary, G., Luthra, N., Rao, A., Walia, S. (2019). Dynamic handwritten signature and machine learning based identity verification for keyless cryptocurrency transactions. Journal of Discrete Mathematical Sciences and Cryptography. 22, 191-202.
26. 27. Wencheng, C., Xiaopeng, G., Hong, S., Limin, Z. (2018). Offline Chinese Signature Verification Based on AlexNet. In: International Conference on Advanced Hybrid Information Processing ADHIP 2017: Advanced Hybrid Information Processing, pp. 33-37.
27. 28. Rateria, A., Agarwal, S. (2018). Off-line Signature Verification through Machine Learning. In: 2018 5th IEEE Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON), Gorakhpur, India.
28. 29. Chandra, S. (2020). Verification of dynamic signature using machine learning approach. Neural Comput & Applic. 32, 11875-11895.
29. 30. Ateş, M., Önder, D.E. (2019). ‘Akıllı Şehir’ Kavramı ve Dönüşen Anlamı Bağlamında Eleştiriler. Megaron 2019. 14, 41-50.
30. 31. Sovacool, B.K., Furszyfer Del Rio, D.D. (2020). Smart home technologies in Europe: A critical review of concepts, benefits, risks and policies. Renewable and Sustainable Energy Reviews. 120.
31. 32. Afrianto, I., Heryandi, A., Finandhita, A., Atin, S. (2019). E-Document Autentification With Digital Signature For Smart City : Reference Model. In: The 2nd ASEAN Workshop on Information Science and Technology (AWIST2019), Bandung , Indonesia.
32. 33. Khare, A., Merlino, G., Longo, F., Puliafito, A., Vyas, O.P. (2020). Design of a Trustless Smart City system: The #SmartME experiment. Internet of Things. 10.
33. 34. Khan, P.W., Byun, Y., Park, N. (2020). A Data Verification System for CCTV Surveillance Cameras Using Blockchain Technology in Smart Cities. Electronics, 9, 484.
34. 35. Lalić, D., Sajić, M., Vidović, Z., Kuzmić, G., Bundalo, D., Bundalo, Z. (2021). Application Of Web Based Technologies For Implementation Of Automated Smart City Services. Automatic Control and Robotics. 20.
35. 36. Fang, L., Zhang, H., Li, M., Ge, C., Liu, L., Liu, Z. (2020). A Secure and Fine-Grained Scheme for Data Security in Industrial IoT Platforms for Smart City. IEEE Internet of Things Journal. 7, 7982-7990.
36. 37. Gong, B., Zhang, X., Cao, Y., Li, Z., Yang, J., Wang, W. (2021). A threshold group signature scheme suitable for the Internet of Things. Concurrency and Computation: Practice and Experience.
37. 38. Chen, J., Gan, W., Hu, M., Chen, C.M. (2021). On the Construction of a Post-Quantum Blockchain for Smart City. Journal of Information Security and Applications. 58.
38. 39. Gupto, P., Sinha, A., Srivastava, P.K., Perti, A., Singh, A.K. (2020). Security Implementations in IoT Using Digital Signature. Innovations in Electrical and Electronic Engineering. Lecture Notes in Electrical Engineering. 661, 523-535.
39. 40. Ferreira, C.M.S., Garrocho, C.T.B., Oliveira, R.A.R., Silva, J.S., Cavalcanti, C.F.M.D.C. (2021). IoT Registration and Authentication in Smart City Applications with Blockchain. Sensors (Basel). 21.
40. 41. Sovacool, B., Furszyfer Del Rio, D. (2020). Smart home technologies in Europe: A critical review of concepts, benefits, risks and policies. Renewable and Sustainable Energy Reviews. 120.
41. 42. Minaee, S., Abdolrashidi, A., Su, H., Bennamoun, M., Zhang, D. (2019). Biometrics Recognition Using Deep Learning: A Survey. arXiv.
42. 43. Meng, T., Jing, X., Yan, Z., Pedrycz, W. (2020). A survey on machine learning for data fusion. Information Fusion. 57, 115-129.
43. 44. Machine Learning, https://www.ibm.com/cloud/learn/machine-learning
44. 45. Ayoub, M. (2020). A review on machine learning algorithms to predict daylighting inside buildings. Solar Energy. 202, 249-275.
45. 46. Yılmaz, A., Kaya, U. (2019). Derin Öğrenme. Kodlab Yayınları, Turkey.
46. 47. Minaee, S., Boykov, Y., Porikli, F., Plaza, A., Kehtarnavaz, N., Terzopoulos, D. (2020). Image Segmentation Using Deep Learning: A Survey. arXiv.
47. 48. Voulodimos, A., Doulamis, N., Doulamis, A., Protopapadakis, E. (2018) Deep Learning for Computer Vision: A Brief Review. Comput Intell Neurosci.
48. 49. A Gentle Introduction to Generative Adversarial Networks (GANs), https://machinelearningmastery.com/what-are-generative-adversarial-networks-gans/
49. 50. Liwicki, M., Blumenstein, M., Heuvel, E., Berger, C.E.H, Stoel, R.D., Found, B., Chen, X., Malik, M.I. (2011). Sigcomp11: signature verification competition for on- and offline skilled forgeries, In: 11th Int. Conf. Document Anal Recognit.
50. 51. Hafemann, L.G., Sabourin, R., Oliveira, L.S. (2016). Analyzing features learned for offline signature verification using deep cnns. In: Pattern Recognition (ICPR), pp. 2989-2994.
51. 52. Malik, M.I. (2010). ICFHR 2010 Signature Verification Competition (4NSigComp2010).
52. 53. Liwicki, M., Malik, M.I., Alewijnse, L., Heuvel, E., Found, B. (2012). ICFHR 2012 Competition on Automatic Forensic Signature Verification (4NsigComp 2012). In: 2012 International Conference on Frontiers in Handwriting Recognition, pp. 823-828. Bari, Italy.
53. 54. Jagtap, A.B., Sawat, D.D., Hegadi, R.S. et al. (2020). Verification of genuine and forged offline signatures using Siamese Neural Network (SNN). Multimed Tools Appl. 79, 35109-35123.
54. 55. One-shot learning, https://en.wikipedia.org/wiki/One-shot_learning
55. 56. Koch, G., Zemel, R., Salakhutdinov, R. (2015). Siamese neural networks for one-shot image recognition. In: ICML deep learning workshop.