saucis

Sakarya University Journal of Computer and Information Sciences

2636-8129

Sakarya University

10.35377/saucis.7.87942.1544012

Software Engineering (Other)

Yazılım Mühendisliği (Diğer)

TurkishLex:Development of a Context-Aware Spell Checker for Detecting and Correcting Spelling Errors in Turkish Texts

Savci

Pinar

Arçelik A.S.

https://orcid.org/0000-0002-2498-3297

Daş

Bihter

FIRAT ÜNİVERSİTESİ

12 31 2024

7 3 404 415 09 05 2024 10 10 2024

2018

Sakarya University Journal of Computer and Information Sciences

In Turkish, correct spelling correction is crucial for effective communication and preserving the integrity of written text. The challenge lies in the complexity of Turkish morphology and spelling, which can lead to frequent and diverse spelling errors. This study presents a spelling checker adapted for Turkish by creating a new Turkish dataset. The proposed spelling checker model effectively captures both minor and major textual changes and can detect the error. Our findings show that the proposed spelling checker system provides high accuracy and reliability with 98.21% accuracy performance with the Symspell module in correcting Turkish texts. This study provides valuable information about the strengths and weaknesses of existing spelling checkers and contributes to the improvement of spelling correction tools for Turkish.

Spell checker Spelling errors Natural Language Processing (NLP) Spelling correction Turkish texts

AR-22-087-0001

Y. Chaabi and F. Ataa Allah, “Amazigh spell checker using Damerau-Levenshtein algorithm and N-gram,” Journal of King Saud University - Computer and Information Sciences, vol. 34, no. 8, Part B, pp. 6116–6124, Sep. 2022, doi: 10.1016/j.jksuci.2021.07.015.

V. J. Hodge and J. Austin, “A comparison of a novel neural spell checker and standard spell checking algorithms,” Pattern Recognition, vol. 35, no. 11, pp. 2571–2580, Nov. 2002, doi: 10.1016/S0031-3203(01)00174-1.

R. Garfinkel, E. Fernandez, and R. Gopal, “Design of an interactive spell checker: Optimizing the list of offered words,” Decision Support Systems, vol. 35, no. 3, pp. 385–397, Jun. 2003, doi: 10.1016/S0167-9236(02)00115-X.

M. Nejja and A. Yousfi, “The Context in Automatic Spell Correction,” Procedia Computer Science, vol. 73, pp. 109–114, Jan. 2015, doi: 10.1016/j.procs.2015.12.055.

K. Sarıtaş, C. A. Öz, and T. Güngör, “A comprehensive analysis of static word embeddings for Turkish,” Expert Systems with Applications, vol. 252, p. 124123, Oct. 2024, doi: 10.1016/j.eswa.2024.124123.

S. Demir and B. Topcu, “Graph-based Turkish text normalization and its impact on noisy text processing,” Engineering Science and Technology, an International Journal, vol. 35, p. 101192, Nov. 2022, doi: 10.1016/j.jestch.2022.101192.

Y. B. Kaya and A. C. Tantuğ, “Effect of tokenization granularity for Turkish large language models,” Intelligent Systems with Applications, vol. 21, p. 200335, Mar. 2024, doi: 10.1016/j.iswa.2024.200335.

Kukich K. Techniques for automatically correcting words in text. ACM computing surveys (CSUR). 1992 Dec 1;24(4):377-439.

P. T. Hacken and C. Tschichold, “Word Manager and CALL: Structured access to the lexicon as a tool for enriching learners’ vocabulary,” ReCALL, vol. 13, no. 1, pp. 121–131, May 2001, doi: 10.1017/S0958344001001112.

W. Phatthiyaphaibun, K. Chaovavanich, C. Polpanumas, A. Suriyawongkul, L. Lowphansirikul, and P. Chormai, PyThaiNLP: Thai Natural Language Processing in Python. (Jun. 2024). Python. Accessed: Aug. 27, 2024. [Online]. Available: https://github.com/PyThaiNLP/pythainlp

hunspell/hunspell. (Aug. 27, 2024). C++. hunspell. Accessed: Aug. 27, 2024. [Online]. Available: https://github.com/hunspell/hunspell

A. Lertpiya, T. Chaiwachirasak, N. Maharattanamalai, T. Lapjaturapit, T. Chalothorn, N. Tirasaroj, et al., "A preliminary study on fundamental Thai NLP tasks for user-generated Web content", Proc. Int. Joint Symp. Artif. Intell. Natural Lang. Process. (iSAI-NLP), pp. 1-8, Nov. 2018.

S. Watcharabutsarakham, "Spell checker for Thai document", Proc. IEEE Region Conf., pp. 1-4, Nov. 2005.

M. Rodphon, K. Siriboon and B. Kruatrachue, "Thai OCR error correction using token passing algorithm", Proc. IEEE Pacific Rim Conf. Commun. Comput. Signal Process., pp. 599-602, 2001.

B. Kruatrachue, K. Somguntar and K. Siriboon, "Thai OCR error correction using genetic algorithm", Proc. 1st Int. Symp. Cyber Worlds, pp. 137-141, 2002.

H. T. Ng, S. M. Wu, T. Briscoe, C. Hadiwinoto, R. H. Susanto and C. Bryant, "The CoNLL-2014 shared task on grammatical error correction", Proc. 18th Conf. Comput. Natural Lang. Learn. Shared Task, pp. 1-14, 2014.

A. Rozovskaya and D. Roth, "Grammatical error correction: Machine translation and classifiers", Proc. 54th Annu. Meeting Assoc. Comput. Linguistic, pp. 2205-2215, Aug. 2016, [online] Available: https://www.aclweb.org/anthology/P16-1208.

M. Junczys-Dowmunt and R. Grundkiewicz, "Phrase-based Machine Translation is State-of-the-Art for Automatic Grammatical Error Correction", Proc. Conf. Empirical Methods Natural Lang. Process., pp. 1546-1556, Nov. 2016, [online] Available: https://www.aclweb.org/anthology/D16-1161.

Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (pp. 5998-6008).

Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692.

Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. In Advances in Neural Information Processing Systems (pp. 2672-2680).

D. N. Mati, M. Hamiti, B. Selimi and J. Ajdari, "Building Spell-Check Dictionary for Low-Resource Language by Comparing Word Usage," 2021 44th International Convention on Information, Communication and Electronic Technology (MIPRO), Opatija, Croatia, 2021, pp. 229-236, doi: 10.23919/MIPRO52101.2021.9597183.

A. Kicsi, K. Szabó Ledenyi, and L. Vidács, “Radiologic text correction for better machine understanding,” Engineering Reports, vol. n/a, no. n/a, p. e12891, doi: 10.1002/eng2.12891.

D. Pogrebnoi, A. Funkner, and S. Kovalchuk, “RuMedSpellchecker: A new approach for advanced spelling error correction in Russian electronic health records,” Journal of Computational Science, vol. 82, p. 102393, Oct. 2024, doi: 10.1016/j.jocs.2024.102393.

E., O’Neill, R., Young, E., Thiaville, M., MacCarthy, J., Carson-Berndsen, & A. Ventresque, S-capade: Spelling correction aimed at particularly deviant errors. In Statistical Language and Speech Processing: 8th International Conference, SLSP 2020, Cardiff, UK, October 14–16, 2020, Proceedings 8 (pp. 85-96). Springer International Publishing.

U., Liyanapathirana, K., Gunasinghe, & G. Dias. Sinspell: A comprehensive spelling checker for sinhala. arXiv preprint arXiv:2107.02983, 2021.

O. Abiola, A. Abayomi-Alli, O. A. Tale, S. Misra, and O. Abayomi-Alli, “Sentiment analysis of COVID-19 tweets from selected hashtags in Nigeria using VADER and Text Blob analyser,” Journal of Electrical Systems and Inf Technol, vol. 10, no. 1, p. 5, Jan. 2023, doi: 10.1186/s43067-023-00070-9.

N. Bölücü and B. Can, "Context Based Automatic Spelling Correction for Turkish," 2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT), Istanbul, Turkey, 2019, pp. 1-4, doi: 10.1109/EBBT.2019.8742067.

Aydoğan, M., & Karci, A. (2020). Spelling Correction with the Dictionary Method for the Turkish Language Using Word Embeddings. Avrupa Bilim ve Teknoloji Dergisi, 57–63. https://doi.org/10.31590/ejosat.araconf8

O. Büyük, M. Erden and L. M. Arslan, "Context Influence on Sequence to Sequence Turkish Spelling Correction," 2019 27th Signal Processing and Communications Applications Conference (SIU), Sivas, Turkey, 2019, pp. 1-4, doi: 10.1109/SIU.2019.8806476.