Sentetik Veri Kullanarak Türkçe Çevrimiçi El Yazısı Tanıma

Öz

Bu çalışmada yapay veri ve öğrenme tranferi kullanan bir çevrimiçi Türkçe el yazısı tanıma sistemi sunuyoruz. Derin yapay sinir ağlarını eğitmek için çok miktarda veri gerekir. Ancak Türkçe çevrimiçi el yazısı için böylesine büyük bir veri seti bulunmamaktadır. Bu sorunu aşmak için yapay veri üreterek sistemi ön eğitime tabi tutmayı, ardından gerçek veri ile ince ayar yapmayı öneriyoruz. Büyük bir İngilizce çevrimiçi el yazısı veri setindeki ayrık karakter örneklerini kullanarak çevrimiçi el yazısı kelimeler üretiyoruz. Bu yapay veri ile ön eğitime tabi tuttuğumuz sistemi gerçek veri ile de eğiterek 2,041 kelimelik gerçek veri üzerinde test ediyoruz. Öğrenme transferi yöntemi sayesinde Türkçe kelimeler için karakter tanıma oranının %61’den %88’e yükseldiğini gözlemliyoruz. Yapay test verisinde de buna yakın bir sonuç alınması yapay verinin gerçek veriye yeterince benzediğini gösterir. Alınan sonuçlara dayanarak yapay veri kullanmanın Türkçe çevrim içi el yazısı alanında yaşanan veri yetersizliği problemine bir çözüm olabileceğini söyleyebiliriz.

Anahtar Kelimeler

• Türkçe El Yazısı Tanıma
• Çevrimiçi El Yazısı Tanıma
• Derin Öğrenme
• Sentetik Veri
• Öğrenme Transferi.

Online Turkish Handwriting Recognition Using Synthetic Data

Abstract

We present a recognition system for online Turkish handwriting trained with synthetically generated data and transfer learning. Training deep networks requires large amounts of data. However, a sufficiently large collection of Turkish handwriting samples is not available. Hence we synthesize data to do pretraining before adapting the system to target dataset by fine tuning. We generate words from isolated character collection of a large English handwriting dataset. Then, we train the system first with synthetic data and fine tune it with Turkish handwriting samples from a smaller dataset. Fine tuning increases the character recognition rate of the final system which is evaluated on 2,041 samples of isolated Turkish words from the initial value of 61% to 88%. Performance of the system on synthetic data is quite similar to that on the Turkish test data which shows that the synthetic data resembles the real data quite closely. According to these results, synthetic data generation can be a solution to the data scarcity problem of online Turkish handwriting.

Keywords

• Turkish Handwriting Recognition
• Online Handwriting Recognition
• Deep Learning
• Synthetic Data Generation
• Transfer Learning.

Kaynakça

1. Aksan, Y., Aksan, M., Koltuksuz, A., Sezer, T., Mersinli, Ü., Demirhan, U. U., Yilmazer, H., Atasoy, G., Öz, S., Yildiz, I. & Kurtoglu, Ö. (2012). Construction of the Turkish National Corpus (TNC). In N. Calzolari, K. Choukri, T. Declerck, M. U. Dogan, B. Maegaard, J. Mariani, J. Odijk & S. Piperidis (Hrsg.), Proceedings of the Eighth International Conference on Language Resources and Evaluation, LREC2012, Istanbul, Turkey, May23-25,2012 (S. 3223–3227). European Language Resources Association (ELRA).
2. Al-Helali, B. M. & Mahmoud, S. A. (2017). Arabic Online Handwriting Recognition (AOHR): A Survey. ACM Comput. Surv., 50(3), 33:1–33:35.
3. Ballard, L., Lopresti, D. P. & Monrose, F. (2007). Forgery Quality and Its Implications for Behavioral Biometric Security. IEEE Trans. Syst. Man Cybern. Part B, 37(5), 1107–1118.
4. Biem, A. (2006). Minimum classification error training for online handwriting Recognition. IEEE Trans. Pattern Anal. Mach. Intell., 28(7), 1041–1051.
5. Caillault, É. & Viard-Gaudin, C. (2007). Mixed Discriminant Training of Hybrid ANN/HMM Systems for Online Handwritten Word Recognition. IJPRAI, 21(1), 117–134.
6. Boubaker, H. , Elbaati, A., Tagougui, N., El Abed, H., Kherallah, M., Märgner, V., & Alimi, A. M. (2021). ADAB database. IEEE Dataport.
7. Li, Z., Liu, F., Yang, W., Peng, S. & Zhou, J. "A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects," in IEEE Transactions on Neural Networks and Learning Systems
8. Çapar, A., Tasdemir, K., Kilic, Ö. & Gökmen, M. (2003). A Turkish Handprint Character Recognition System. Computer and Information Sciences - ISCIS 2003, 18th International Symposium, Antalya, Turkey, November 3-5, 2003, Proceedings, 447–456.

9. Carbune, V., Gonnet, P., Deselaers, T., Rowley, H. A., Daryin, A. N., Calvo, M., Wang, L., Keysers, D., Feuz, S. & Gervais, P. (2020). Fast multi-language LSTM-based online handwriting recognition. Int. J. Document Anal. Recognit., 23(2), 89–102.
10. Do, T. M. T. & Artières, T. (2009). Maximum Margin Training of Gaussian HMMs for Handwriting Recognition. 10th International Conference on Document Analysis and Recognition, ICDAR2009, Barcelona, Spain, 26-29 July 2009, 976–980.
11. Doermann, D. S. & Jaeger, S. (Hrsg.). (2008). Arabic and Chinese Handwriting Recognition SACH2006 Summit College Park, MD, USA, September 27-28, 2006 Selected Papers (Bd. 4768). Springer.
12. Dutta, K., Krishnan, P., Mathew, M. & Jawahar, C. V. (2018). Improving CNN-RNN Hybrid Networks for Handwriting Recognition. 16th International Conference on Frontiers in Handwriting Recognition, ICFHR 2018, Niagara Falls, NY, USA, August 5-8, 2018, 80–85.
13. Elarian, Y., Abdel-Aal, R. E., Ahmad, I., Parvez, M. T. & Zidouri, A. B. C. (2014). Handwriting synthesis: classifications and techniques. Int. J. Document Anal. Recognit., 17(4), 455–469.
14. Garcia-Salicetti, S., Dorizzi, B., Gallinari, P. & Wimmer, Z. (2001). Maximum Mutual Information training for an online neural predictive handwritten word recognition system. IJDAR, 4(1), 56–68.
15. Gauthier, N., Artières, T., Gallinari, P. & Dorizzi, B. (2001). Strategies for Combining On-line and Off-line Information in an On-line Handwriting Recognition System. 6th International Conference on Document Analysis and Recognition, ICDAR 2001, 10-13 September 2001, Seattle, WA, USA, 412–416.
16. Graves, A. (2012). Supervised Sequence Labelling with Recurrent Neural Networks (Bd. 385). Springer.
17. Graves, A. (2013). Generating Sequences With Recurrent Neural Networks. CoRR, abs/1308.0850.
18. Graves, A., Fernández, S., Liwicki, M., Bunke, H. & Schmidhuber, J. (2007). Unconstrained on-line handwriting recognition with recurrent neural networks. Advances in Neural Information Processing Systems, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, NIPS, Vancouver, British Columbia, Canada, December 3-6, 2007, 577–584.
19. Graves, A., Liwicki, M., Fernandez, S., Bertolami, R., Bunke, H. & Schmidhuber, J. (2009). A novel coNNectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell., 31(5), 855–868.
20. Guerfali, W. & Plamondon, R. (1995). The Delta LogNormal theory for the generation and modeling of cursive characters. Third International Conference on Document Analysis and Recognition, ICDAR 1995, August 14 - 15, 1995, Montreal, Canada. Volume I, 495–498.
21. Guyon, I., Schomaker, L., Plamondon, R., Liberman, M. & Janet, S. (1994). UNIPEN project of on-line data exchange and recognizer benchmarks. 12th IAPR International Conference on Pattern Recognition, Conference B: Patern Recognition and Neural Networks, ICPR 1994, Jerusalem, Israel, 9-13 October, 1994, Volume 2, 29–33.
22. Haines, T. S. F., Aodha, O. M. & Brostow, G. J. (2016). My Text in Your Handwriting. ACM Trans. Graph., 35(3), 26:1–26:18.
23. Hu, J., Lim, S. G. & Brown, M. K. (2000). Writer independent on-line handwriting recognition using an HMM approach. Pattern Recognition, 33(1), 133–147.
24. Hüsken, M. & Stagge, P. (2003). Recurrent neural networks for time series classification. Neurocomputing, 50, 223–235.
25. Jäger, S., Manke, S., Reichert, J. & Waibel, A. (2001). Online handwriting recognition: the NPen++ recognizer. IJDAR, 3(3), 169–180.
26. Jawahar, C. V., Balasubramanian, A., Meshesha, M. & Namboodiri, A. M. (2009). Retrieval of online handwriting by synthesis and matching. Pattern Recognit., 42(7), 1445–1457.
27. Kabakus, A. T. & Erdogmus, P. (2021). A novel handwritten Turkish letter recognition model based on convolutional neural network. Concurr. Comput. Pract. Exp., 33(21).
28. Kaplan, K., Ertunç, H. M. & Vardar, E. (2017). Handwriting Character Recognision by using Fuzzy Logic. Fırat University Turkish Journal of Science & Technology, 12, 71–77.
29. Korkmaz, S. U., Kirçiçegi, G., Akinci, Y. & Atalay, V. (2003). A Character Recognizer for Turkish Language. 7th International Conference on Document Analysis and Recognition (ICDAR 2003), 2-Volume Set, 3-6 August 2003, Edinburgh, Scotland, UK, 1238–1241.
30. Lin, Z. & Wan, L. (2007). Style-preserving English handwriting synthesis. PatternRecognit., 40(7), 2097–2109.
31. Liwicki, M. & Bunke, H. (2005a). Handwriting recognition of whiteboard notes. In Proceedings of the 12th Conference of the International Graphonomics Society, 118–122.
32. Liwicki, M. & Bunke, H. (2005b). IAM-OnDB - an On-Line English Sentence Database Acquired from Handwritten Text on a Whiteboard. Eighth International Conference on Document Analysis and Recognition (ICDAR 2005), 29 August - 1 September 2005, Seoul, Korea, 956–961.
33. Liwicki, M., Graves, A., Bunke, H. & Schmidhuber, J. (2007a). A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks. In Proceedings of the 9th International Conference on Document Analysis and Recognition, ICDAR 2007, 367–371.
34. Marukatat, S., Artières, T., Gallinari, P. & Dorizzi, B. (2001). Sentence Recognition through Hybrid Neuro-Markovian Modeling. 6th International Conference on Document Analysis and Recognition, ICDAR 2001, 10-13 September 2001, Seattle, WA, USA, 731–737.
35. Mayr, M., Stumpf, M., Nicolaou, A., Seuret, M., Maier, A. & Christlein, V. (2020). Spatio-Temporal Handwriting Imitation. In A. Bartoli & A. Fusiello (Hrsg.), Computer Vision - ECCV 2020 Workshops - Glasgow, UK, August 23-28, 2020, Proceedings, Part V (S. 528–543). Springer.
36. Naz, S., Umar, A. I., Ahmad, R., Siddiqi, I., Ahmed, S. B., Razzak, M. I. & Shafait, F. (2017). Urdu Nastaliq recognition using convolutional-recursive deep learning. Neurocomputing, 243, 80–87.
37. Niu, S., Liu, Y., Wang, J. & Song, H. (2020). A Decade Survey of Transfer Learning (2010-2020). IEEE Trans. Artif. Intell., 1(2), 151–166.
38. Plamondon, R. (1995). A kinematic theory of rapid human movements. Biological Cybernetics, 72, 295–307.
39. Plamondon, R. & Srihari, S. N. (2000). On-line and off-line handwriting recognition: A Comprehensive Survey. IEEE Trans. Pattern Anal. Mach. Intell., 22(1), 63–84.
40. Priya, A., Mishra, S., Raj, S., Mandal, S. & Datta, S. (2016). Online and offline character recognition: A survey. 2016 International Conference on Communication and Signal Processing (ICCSP), 0967–0970.
41. Robinson, T., Hochberg, M. & Renals, S. (1996). The Use of Recurrent Neural Networks in Continuous Speech Recognition. In C.-H. Lee, F. K. Soong & K. K. Paliwal (Hrsg.), Automatic Speech and Speaker Recognition: Advanced Topics (S. 233–258). Springer US.
42. Romero, V., Rossi, A. H. T. & Vidal, E. (2012). Multimodal Interactive Handwritten Text Transcription (Bd. 80). WorldScientific.
43. Schenk, J. & Rigoll, G. (2006). Novel hybrid NN/HMM modelling techniques for on-line handwriting recognition. 10th International Workshop on Frontiers in Handwriting Recognition, IWFHR 2006, IAPR. , La Baule, France, Oct 2006, 619–6230.
44. Schenkel, M., Guyon, I. & Henderson, D. (1995). On-line cursive script recognition using Time-delay Neural Networks and Hidden Markov Models. Mach. Vis. Appl., 8(4), 215–223.
45. Şekerci, M. (2007). Turkish connected and slant handwriting recognition system. ( MSc thesis ). Trakya Üniversitesi.
46. Shewalkar, A., Nyavanandi, D. & Ludwig, S. A. (2019). Performance Evaluation of Deep Neural Networks Applied to Speech Recognition: RNN, LSTM and GRU. J. Artif. Intell. Soft Comput. Res., 9(4), 235–245.
47. Shi, B., Bai, X. & Yao, C. (2017). An End-to-End Trainable Neural Network for Image-Based Sequence Recognition and Its Application to Scene Text Recognition. IEEE Trans. Pattern Anal. Mach. Intell., 39(11), 2298–2304.
48. Singer, Y. & Tishby, N. (1994). Dynamical encoding of cursive handwriting. Biol. Cybern., 71(3), 227–237.
49. Tagougui, N., Kherallah, M. & Alimi, A. M. (2013). Online Arabic handwriting recognition: a survey. IJDAR, 16(3), 209–226.
50. Tasdemir, E. F. B. & Yanikoglu, B. A. (2018). Large vocabulary recognition for online Turkish handwriting with sublexical units. Turkish J. Electr. Eng. Comput. Sci., 26(5), 2218–2233.
51. The UNIPEN Consortium. (n. d.). The UNIPEN project [Accessed: 2017-12-26]. http://www.unipen.org/products.html
52. Türk, U., Atmaca, F., Özates, S. B., Berk, G., Bedir, S. T., Köksal, A., Basaran, B. Ö., Güngör, T. & Özgür, A. (2020). Resources for Turkish Dependency Parsing: Introducing the BOUN Treebank and the BoAT Annotation Tool. CoRR, abs/2002.10416.
53. Ul-Hasan, A., Ahmed, S. B., Rashid, S. F., Shafait, F. & Breuel, T. M. (2013). Offline Printed Urdu Nastaleeq Script Recognition with Bidirectional LSTM Networks. 12th International Conference on Document Analysis and Recognition, 1061–1065.
54. Voigtlaender, P., Doetsch, P. & Ney, H. (2016). Handwriting Recognition with Large Multidimensional Long Short-Term Memory Recurrent Neural Networks. 15th International Conference on Frontiers in Handwriting Recognition, 228–233.
55. Vural, E., Erdogan, H., Oflazer, K. & Yanikoglu, B. A. (2005). An online handwriting recognition system for Turkish. Document Recognition and Retrieval XII, DRR2005, SanJose, California, USA, January 16-20, 2005, Proceedings, 56–65.
56. Weiss, K. R., Khoshgoftaar, T. M. & Wang, D. (2016). A survey of transfer learning. J.BigData, 3, 9.
57. Yanikoglu, B. & Kholmatov, A. (2003). Turkish handwritten text recognition: a case of agglutinative languages. Document Recognition and Retrieval X, 22-23 January 2003, Santa Clara, California, USA, Proceedings, 227–233.