Recognition of Online Turkish Handwriting using Transfer Learning

Year 2023, , 719 - 726, 27.09.2023

Esma Fatıma Bilgin Taşdemir

https://doi.org/10.29109/gujsc.1141508

Abstract

We present a recognition system for online Turkish handwriting using transfer learning. Training deep networks requires large amounts of data. Since such a sufficiently large collection of Turkish handwriting samples is not available, So we adopt the transfer learning approach and train and optimize a CNN-BLSTM recognition system first using the standard IAM-On dataset of English handwriting. Then, we fine tune it with Turkish handwriting samples from a smaller dataset. Fine tuning increases the character recognition rate of the final system which is evaluated on 2,041 samples of isolated Turkish words from the initial value of 49% to 85%. The results show that transfer learning can be a solution to the data scarcity problem of online Turkish handwriting.

Keywords

Online Handwriting Recognition, Deep Learning, Transfer Learning, Turkish Handwriting Recognition

References

• 1. R. Plamondon and S. N. Srihari, “On-line and off-line handwriting recognition: A comprehensive survey” , IEEE Trans. Pattern Anal. Mach.Intell., vol. 22, no. 1, pp. 63–84, 2000.
• 2. B. M. Al-Helali and S. A. Mahmoud, “Arabic online handwriting recognition (AOHR): A survey”, ACM Comput. Surv., vol. 50, no. 3, pp. 33:1–33:35, 2017.
• 3. N. Tagougui, M. Kherallah, and A. M. Alimi, “Online Arabic handwriting recognition: a survey”, IJDAR, vol. 16, no. 3, pp. 209–226, 2013.
• 4. D. S. Doermann and S. Jaeger, Eds., Arabic and Chinese handwriting recognition - SACH 2006 Summit College Park, MD, USA, September 27-28, 2006 Selected Papers, ser. Lecture Notes in Computer Science, vol. 4768. Springer, 2008.
• 5. A. Priya, S. Mishra, S. Raj, S. Mandal, and S. Datta, “Online and offline character recognition: A survey”, in 2016 International Conference on Communication and Signal Processing (ICCSP), 2016, pp. 0967–0970.
• 6. M. Liwicki and H. Bunke, “Handwriting recognition of whiteboard notes”, In Proceedings of the 12th Conference of the International Graphonomics Society, 2005, pp. 118–122.
• 7. V. Carbune, P. Gonnet, T. Deselaers, H. A. Rowley, A. N. Daryin, M. Calvo, L. Wang, D. Keysers, S. Feuz, and P. Gervais, “Fast multi-language lstm-based online handwriting recognition”, Int. J. DocumentAnal. Recognit., vol. 23, no. 2, pp. 89–102, 2020.
• 8. X.-Y. Zhang, Y. Bengio, and C.-L. Liu, “Online and offline handwritten Chinese character recognition: A comprehensive study and new bench-mark”, Pattern Recognition, vol. 61, pp. 348–360, 2017.
• 9. S. Jager, S. Manke, J. Reichert, and A. Waibel, “Online handwriting recognition: the NPen++ recognizer”, IJDAR, vol. 3, no. 3, pp. 169–180, 2001.
• 10. S. Garcia-Salicetti, B. Dorizzi, P. Gallinari, and Z. Wimmer, “Maximum Mutual information training for an online neural predictive handwritten word recognition system”, IJDAR, vol. 4, no. 1, pp. 56–68, 2001.
• 11. E. Caillault and C. Viard-Gaudin, “Mixed discriminant training of hybrid ANN/HMM systems for online handwritten word recognition”, IJPRAI,vol. 21, no. 1, pp. 117–134, 2007.
• 12. T. M. T. Do and T. Arti`eres, “Maximum margin training of gaussian HMMs for handwriting recognition”, in 10th International Conference On Document Analysis and Recognition, ICDAR 2009, Barcelona, Spain, 26-29 July 2009, 2009, pp. 976–980.
• 13. J. Schenk and G. Rigoll, “Novel hybrid NN/HMM modelling techniques for on-line handwriting recognition”, in 10th International Workshop on Frontiers in Handwriting Recognition, IWFHR 2006, IAPR. , La Baule,France, Oct 2006, 2006, pp. 619––6230.
• 14. N. Gauthier, T. Artieres, P. Gallinari, and B. Dorizzi, “Strategies for combining on-line and off-line information in an on-line handwriting recognition system”, in 6th International Conference on Document Analysis and Recognition, ICDAR 2001, 10-13 September 2001, Seattle,WA, USA, 2001, pp. 412–416.
• 15. S. Marukatat, T. Artieres, P. Gallinari, and B. Dorizzi, “Sentence recognition through hybrid neuro-markovian modeling”, in 6th International Conference on Document Analysis and Recognition, ICDAR 2001, 10-13 September 2001, Seattle, WA, USA, 2001, pp. 731–737.
• 16. M. Schenkel, I. Guyon, and D. Henderson, “On-line cursive script recognition using time-delay neural networks and Hidden Markov Models”, Mach. Vis. Appl., vol. 8, no. 4, pp. 215–223, 1995.
• 17. J. Hu, S. G. Lim, and M. K. Brown, “Writer independent on-line handwriting recognition using an HMM approach”, Pattern Recognition,vol. 33, no. 1, pp. 133–147, 2000.
• 18. A. Biem, “Minimum classification error training for online handwriting recognition”, IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 7, pp.1041–1051, 2006.
• 19. M. Liwicki and H. Bunke, “IAM-OnDB - an on-line English sentence database acquired from handwritten text on a whiteboard”, in Eighth International Conference on Document Analysis and Recognition, ICDAR 2005, 29 August - 1 September 2005, Seoul, Korea, 2005, pp. 956–961.
• 20. M. Liwicki, A. Graves, H. Bunke, and J. Schmidhuber, “A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks”, in Proceedings of the 9th International Conference on Document Analysis and Recognition, ICDAR, 2007, pp.367–371.
• 21. A. Graves, S. Fern ́andez, M. Liwicki, H. Bunke, and J. Schmidhuber, “Unconstrained on-line handwriting recognition with recurrent neural networks”, in Advances in Neural Information Processing Systems 20, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, NIPS, Vancouver, British Columbia, Canada,December 3-6, 2007, 2007, pp. 577–584.
• 22. A. Graves, M. Liwicki, S. Fernandez, R. Bertolami, H. Bunke, and J. Schmidhuber, “A novel connectionist system for unconstrained handwriting recognition”, IEEE Trans. Pattern Anal. Mach. Intell., vol. 31,no. 5, pp. 855–868, 2009.
• 23. M. Liwicki, A. Graves, H. Bunke, and J. Schmidhuber, “A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks”, In Proceedings of the 9th InternationalConference on Document Analysis and Recognition, ICDAR 2007, 2007.
• 24. E. F. B. Tasdemir and B. A. Yanikoglu, “Large vocabulary recognition for online Turkish handwriting with sublexical units”, Turkish J. Electr.Eng. Comput. Sci., vol. 26, no. 5, pp. 2218–2233, 2018.
• 25. F. Biadsy, J. El-Sana, and J. Habash, “Online Arabic handwriting recognition using Hidden Markov Models”, in Tenth International Workshop on Frontiers in Handwriting Recognition, IWFHR 2007, IAPR. New York,USA, 2006, 2006.
• 26. H. A. A. Alshafy and M. E. Mustafa, “HMM based approach for online Arabic handwriting recognition”, in 14th International Conference on Intelligent Systems Design and Applications, ISDA 2014, Okinawa, Japan, November 28-30, 2014. IEEE, 2014, pp. 211–215.
• 27. T. Robinson, M. Hochberg, and S. Renals, The Use of Recurrent NeuralNetworks in Continuous Speech Recognition.Boston, MA: Springer US, 1996, pp. 233–258.
• 28. M. Husken and P. Stagge, “Recurrent neural networks for time series classification”, Neurocomputing, vol. 50, pp. 223–235, 2003.
• 29. A. Capar, K. Tasdemir, O. Kilic, and M. Gokmen, “A Turkish handprint character recognition system”, in Computer and Information Sciences -ISCIS 2003, 18th International Symposium, Antalya, Turkey, November3-5, 2003, Proceedings, 2003, pp. 447–456.
• 30. K. Kaplan, H. M. Ertunc ̧, and E. Vardar, “Handwriting character recognition by using fuzzy logic”, Fırat University Turkish Journal of Science & Technology, vol. 12, pp. 71 – 77, 2017.
• 31. S. U. Korkmaz, G. Kirçiçeği, Y. Akinci, and V. Atalay, “A character recognizer for Turkish language”, in 7th International Conference on Document Analysis and Recognition (ICDAR 2003), 2-Volume Set, 3-6 August 2003, Edinburgh, Scotland, UK, 2003, pp. 1238–1241.
• 32. B. Yanikoglu and A. Kholmatov, “Turkish handwritten text recognition: a case of agglutinative languages”, in Document Recognition And Retrieval X, 22-23 January 2003, Santa Clara, California, USA, Proceedings, 2003, pp. 227–233.
• 33. M. Şekerci, “Turkish connected and slant handwritten recognition system”, Master’s thesis, Trakya ̈Universitesi, 2007.
• 34. A. T. Kabakus and P. Erdogmus, “A novel handwritten turkish letter recognition model based on convolutional neural network”, Concurr. Comput. Pract. Exp., vol. 33, no. 21, 2021.
• 35. E. Vural, H. Erdogan, K. Oflazer, and B. A. Yanikoglu, “An online handwriting recognition system for Turkish”, in Document Recognition And Retrieval XII, DRR 2005, San Jose, California, USA, January 16-20,2005, Proceedings, 2005, pp. 56–65.
• 36. M. Liwicki and H. Bunke, “Iam-On DB - an on-line English sentence database acquired from handwritten text on a whiteboard”, in Eighth International Conference on Document Analysis and Recognition (ICDAR 2005), 29 August - 1 September 2005, Seoul, Korea. IEEE Computer Society, 2005, pp. 956–961.
• 37. V. Frinken and S. Uchida, “Deep BLSTM neural networks for unconstrained continuous handwritten text recognition”, in 13th International Conference on Document Analysis and Recognition, ICDAR 2015, Nancy, France, August 23-26, 2015. IEEE Computer Society, 2015, pp. 911–915.
• 38. K. R. Weiss, T. M. Khoshgoftaar, and D. Wang, “A survey of transfer learning”, J. Big Data, vol. 3, p. 9, 2016.
• 39. S. Niu, Y. Liu, J. Wang, and H. Song, “A decade survey of transfer learning (2010-2020)”, IEEE Trans. Artif. Intell., vol. 1, no. 2, pp. 151–166, 2020.
• 40. E. F. B. Tasdemir, “Online Turkish Handwriting Recognition Using Synthetic Data”, European Journal of Science and Technology, vol. 32, no. 5, pp. 649-656, 2021.