Recognition of Online Turkish Handwriting using Transfer Learning

Abstract

We present a recognition system for online Turkish handwriting using transfer learning. Training deep networks requires large amounts of data. Since such a sufficiently large collection of Turkish handwriting samples is not available, So we adopt the transfer learning approach and train and optimize a CNN-BLSTM recognition system first using the standard IAM-On dataset of English handwriting. Then, we fine tune it with Turkish handwriting samples from a smaller dataset. Fine tuning increases the character recognition rate of the final system which is evaluated on 2,041 samples of isolated Turkish words from the initial value of 49% to 85%. The results show that transfer learning can be a solution to the data scarcity problem of online Turkish handwriting.

Keywords

• Online Handwriting Recognition
• Deep Learning
• Transfer Learning
• Turkish Handwriting Recognition

References

1. 1. R. Plamondon and S. N. Srihari, “On-line and off-line handwriting recognition: A comprehensive survey” , IEEE Trans. Pattern Anal. Mach.Intell., vol. 22, no. 1, pp. 63–84, 2000.
2. 2. B. M. Al-Helali and S. A. Mahmoud, “Arabic online handwriting recognition (AOHR): A survey”, ACM Comput. Surv., vol. 50, no. 3, pp. 33:1–33:35, 2017.
3. 3. N. Tagougui, M. Kherallah, and A. M. Alimi, “Online Arabic handwriting recognition: a survey”, IJDAR, vol. 16, no. 3, pp. 209–226, 2013.
4. 4. D. S. Doermann and S. Jaeger, Eds., Arabic and Chinese handwriting recognition - SACH 2006 Summit College Park, MD, USA, September 27-28, 2006 Selected Papers, ser. Lecture Notes in Computer Science, vol. 4768. Springer, 2008.
5. 5. A. Priya, S. Mishra, S. Raj, S. Mandal, and S. Datta, “Online and offline character recognition: A survey”, in 2016 International Conference on Communication and Signal Processing (ICCSP), 2016, pp. 0967–0970.
6. 6. M. Liwicki and H. Bunke, “Handwriting recognition of whiteboard notes”, In Proceedings of the 12th Conference of the International Graphonomics Society, 2005, pp. 118–122.
7. 7. V. Carbune, P. Gonnet, T. Deselaers, H. A. Rowley, A. N. Daryin, M. Calvo, L. Wang, D. Keysers, S. Feuz, and P. Gervais, “Fast multi-language lstm-based online handwriting recognition”, Int. J. DocumentAnal. Recognit., vol. 23, no. 2, pp. 89–102, 2020.
8. 8. X.-Y. Zhang, Y. Bengio, and C.-L. Liu, “Online and offline handwritten Chinese character recognition: A comprehensive study and new bench-mark”, Pattern Recognition, vol. 61, pp. 348–360, 2017.

9. 9. S. Jager, S. Manke, J. Reichert, and A. Waibel, “Online handwriting recognition: the NPen++ recognizer”, IJDAR, vol. 3, no. 3, pp. 169–180, 2001.
10. 10. S. Garcia-Salicetti, B. Dorizzi, P. Gallinari, and Z. Wimmer, “Maximum Mutual information training for an online neural predictive handwritten word recognition system”, IJDAR, vol. 4, no. 1, pp. 56–68, 2001.
11. 11. E. Caillault and C. Viard-Gaudin, “Mixed discriminant training of hybrid ANN/HMM systems for online handwritten word recognition”, IJPRAI,vol. 21, no. 1, pp. 117–134, 2007.
12. 12. T. M. T. Do and T. Arti`eres, “Maximum margin training of gaussian HMMs for handwriting recognition”, in 10th International Conference On Document Analysis and Recognition, ICDAR 2009, Barcelona, Spain, 26-29 July 2009, 2009, pp. 976–980.
13. 13. J. Schenk and G. Rigoll, “Novel hybrid NN/HMM modelling techniques for on-line handwriting recognition”, in 10th International Workshop on Frontiers in Handwriting Recognition, IWFHR 2006, IAPR. , La Baule,France, Oct 2006, 2006, pp. 619––6230.
14. 14. N. Gauthier, T. Artieres, P. Gallinari, and B. Dorizzi, “Strategies for combining on-line and off-line information in an on-line handwriting recognition system”, in 6th International Conference on Document Analysis and Recognition, ICDAR 2001, 10-13 September 2001, Seattle,WA, USA, 2001, pp. 412–416.
15. 15. S. Marukatat, T. Artieres, P. Gallinari, and B. Dorizzi, “Sentence recognition through hybrid neuro-markovian modeling”, in 6th International Conference on Document Analysis and Recognition, ICDAR 2001, 10-13 September 2001, Seattle, WA, USA, 2001, pp. 731–737.
16. 16. M. Schenkel, I. Guyon, and D. Henderson, “On-line cursive script recognition using time-delay neural networks and Hidden Markov Models”, Mach. Vis. Appl., vol. 8, no. 4, pp. 215–223, 1995.
17. 17. J. Hu, S. G. Lim, and M. K. Brown, “Writer independent on-line handwriting recognition using an HMM approach”, Pattern Recognition,vol. 33, no. 1, pp. 133–147, 2000.
18. 18. A. Biem, “Minimum classification error training for online handwriting recognition”, IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 7, pp.1041–1051, 2006.
19. 19. M. Liwicki and H. Bunke, “IAM-OnDB - an on-line English sentence database acquired from handwritten text on a whiteboard”, in Eighth International Conference on Document Analysis and Recognition, ICDAR 2005, 29 August - 1 September 2005, Seoul, Korea, 2005, pp. 956–961.
20. 20. M. Liwicki, A. Graves, H. Bunke, and J. Schmidhuber, “A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks”, in Proceedings of the 9th International Conference on Document Analysis and Recognition, ICDAR, 2007, pp.367–371.
21. 21. A. Graves, S. Fern ́andez, M. Liwicki, H. Bunke, and J. Schmidhuber, “Unconstrained on-line handwriting recognition with recurrent neural networks”, in Advances in Neural Information Processing Systems 20, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, NIPS, Vancouver, British Columbia, Canada,December 3-6, 2007, 2007, pp. 577–584.
22. 22. A. Graves, M. Liwicki, S. Fernandez, R. Bertolami, H. Bunke, and J. Schmidhuber, “A novel connectionist system for unconstrained handwriting recognition”, IEEE Trans. Pattern Anal. Mach. Intell., vol. 31,no. 5, pp. 855–868, 2009.
23. 23. M. Liwicki, A. Graves, H. Bunke, and J. Schmidhuber, “A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks”, In Proceedings of the 9th InternationalConference on Document Analysis and Recognition, ICDAR 2007, 2007.
24. 24. E. F. B. Tasdemir and B. A. Yanikoglu, “Large vocabulary recognition for online Turkish handwriting with sublexical units”, Turkish J. Electr.Eng. Comput. Sci., vol. 26, no. 5, pp. 2218–2233, 2018.
25. 25. F. Biadsy, J. El-Sana, and J. Habash, “Online Arabic handwriting recognition using Hidden Markov Models”, in Tenth International Workshop on Frontiers in Handwriting Recognition, IWFHR 2007, IAPR. New York,USA, 2006, 2006.
26. 26. H. A. A. Alshafy and M. E. Mustafa, “HMM based approach for online Arabic handwriting recognition”, in 14th International Conference on Intelligent Systems Design and Applications, ISDA 2014, Okinawa, Japan, November 28-30, 2014. IEEE, 2014, pp. 211–215.
27. 27. T. Robinson, M. Hochberg, and S. Renals, The Use of Recurrent NeuralNetworks in Continuous Speech Recognition.Boston, MA: Springer US, 1996, pp. 233–258.
28. 28. M. Husken and P. Stagge, “Recurrent neural networks for time series classification”, Neurocomputing, vol. 50, pp. 223–235, 2003.
29. 29. A. Capar, K. Tasdemir, O. Kilic, and M. Gokmen, “A Turkish handprint character recognition system”, in Computer and Information Sciences -ISCIS 2003, 18th International Symposium, Antalya, Turkey, November3-5, 2003, Proceedings, 2003, pp. 447–456.
30. 30. K. Kaplan, H. M. Ertunc ̧, and E. Vardar, “Handwriting character recognition by using fuzzy logic”, Fırat University Turkish Journal of Science & Technology, vol. 12, pp. 71 – 77, 2017.
31. 31. S. U. Korkmaz, G. Kirçiçeği, Y. Akinci, and V. Atalay, “A character recognizer for Turkish language”, in 7th International Conference on Document Analysis and Recognition (ICDAR 2003), 2-Volume Set, 3-6 August 2003, Edinburgh, Scotland, UK, 2003, pp. 1238–1241.
32. 32. B. Yanikoglu and A. Kholmatov, “Turkish handwritten text recognition: a case of agglutinative languages”, in Document Recognition And Retrieval X, 22-23 January 2003, Santa Clara, California, USA, Proceedings, 2003, pp. 227–233.
33. 33. M. Şekerci, “Turkish connected and slant handwritten recognition system”, Master’s thesis, Trakya ̈Universitesi, 2007.
34. 34. A. T. Kabakus and P. Erdogmus, “A novel handwritten turkish letter recognition model based on convolutional neural network”, Concurr. Comput. Pract. Exp., vol. 33, no. 21, 2021.
35. 35. E. Vural, H. Erdogan, K. Oflazer, and B. A. Yanikoglu, “An online handwriting recognition system for Turkish”, in Document Recognition And Retrieval XII, DRR 2005, San Jose, California, USA, January 16-20,2005, Proceedings, 2005, pp. 56–65.
36. 36. M. Liwicki and H. Bunke, “Iam-On DB - an on-line English sentence database acquired from handwritten text on a whiteboard”, in Eighth International Conference on Document Analysis and Recognition (ICDAR 2005), 29 August - 1 September 2005, Seoul, Korea. IEEE Computer Society, 2005, pp. 956–961.
37. 37. V. Frinken and S. Uchida, “Deep BLSTM neural networks for unconstrained continuous handwritten text recognition”, in 13th International Conference on Document Analysis and Recognition, ICDAR 2015, Nancy, France, August 23-26, 2015. IEEE Computer Society, 2015, pp. 911–915.
38. 38. K. R. Weiss, T. M. Khoshgoftaar, and D. Wang, “A survey of transfer learning”, J. Big Data, vol. 3, p. 9, 2016.
39. 39. S. Niu, Y. Liu, J. Wang, and H. Song, “A decade survey of transfer learning (2010-2020)”, IEEE Trans. Artif. Intell., vol. 1, no. 2, pp. 151–166, 2020.
40. 40. E. F. B. Tasdemir, “Online Turkish Handwriting Recognition Using Synthetic Data”, European Journal of Science and Technology, vol. 32, no. 5, pp. 649-656, 2021.