A LEXICAL SIMPLIFICATION APPLICATION AND EFFECT IN INFORMATION RETRIEVAL

Year 2025, Volume: 26 Issue: 3, 363 - 378, 25.09.2025

Gökhan Göksel

https://doi.org/10.18038/estubtda.1716842

Abstract

The application of lexical simplification techniques in this study aims to reduce vocabulary mismatch between user queries and document collections. Two distinct strategies —complex word substitution (Stg1) and query expansion with simplified alternative (Stg2)— are systematically evaluated across six benchmark datasets (FiQA, ArguAna, NFCorpus, SCIDOCS, SciFact, and TREC-COVID) using nDCG@10 and MAP utilizing dense and sparse neural retrieval models. Our experiments reveal that Stg1 consistently degrades retrieval performance, with nDCG@10 and MAP declines often exceeding 0.10 and 0.03, respectively. In contrast, Stg2 preserves nearly all baseline effectiveness, nDCG@10 drops by fewer than 0.02 points, MAP by fewer than 0.01, and in some instances even surpasses the baseline. Risk-sensitive evaluation further confirms that Stg2 incurs minimal or no per-query risk and exhibits promising results, whereas Stg1 uniformly harms effectiveness. These findings indicate that appending simplified term variants alongside original query terms offers a fair and robust means of addressing vocabulary mismatch without sacrificing precision or recall. It is recommended that future IR pipelines incorporate carefully tailored query simplification models to balance user query intent and high retrieval quality.

Keywords

Lexical simplification , Information retrieval , Complex word identification , Vocabulary mismatching , Query rewriting

References

• [1] Saggion H. Automatic text simplification. Morgan & Claypool Publishers 2017.
• [2] Al-Thanyyan SS, Azmi AM. Automated text simplification: A survey. ACM Comput. Surv. 2021; 54(2): 1-36.
• [3] Kincaid JP, Fishburne Jr RP, Rogers RL, Chissom BS. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. 1975.
• [4] Shardlow M. A survey of automated text simplification. International Journal of Advanced Computer Science and Applications(IJACSA), Special Issue on Natural Language Processing 2014; 4(1): 58-70.
• [5] Grabar N, Saggion H. Evaluation of automatic text simplification: Where are we now, where should we go from here. Traitement Automatique des Langues Naturelles 2022; 1: 453–463.
• [6] Aydın A, Arslan A, Dinçer BT. A set of novel html document quality features for web information retrieval: Including applications to learning to rank for information retrieval. Expert Systems with Applications 2024; 246.
• [7] Cao G, Nie JY, Gao J, Robertson S. Selecting good expansion terms for pseudo-relevance feedback. In: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval 2008; 243–250.
• [8] Göksel G, Arslan A, Dinçer BT. A selective approach to stemming for minimizing the risk of failure in information retrieval systems. PeerJ Computer Science 2023; 9.
• [9] Krovetz R. Viewing morphology as an inference process. In: Proceedings of the 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval SIGIR’93 1993; 191–202.
• [10] Porter MF. An algorithm for suffix stripping. Readings in Information Retrieval 1997; 313–316.
• [11] Harman D. How effective is suffixing?. Journal of the American Society for Information Science 1991; 42: 7–15.
• [12] Wood V. Improving query term expansion with machine learning. Master’s thesis, University of Otago 2013.
• [13] Paetzold GH, Specia L. A survey on lexical simplification. Journal of Artificial Intelligence Research 2017; 60: 549–593.
• [14] Sheang KC, Ferres D, Saggion H. Controllable lexical simplification for English. In: Proceedings of the Workshop on Text Simplification, Accessibility, and Readability 2022; 199–206.
• [15] Smadu RA, Ion DG, Cercel DC, Pop F, Cercel MC. Investigating large language models for complex word identification in multilingual and multidomain setups. In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing 2024; 16764–16800.
• [16] Tan K, Luo K, Lan Y, Yuan Z, Shu J. An LLM-enhanced adversarial editing system for lexical simplification. In: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), ELRA and ICCL 2024; 1136–1146.
• [17] Horn C, Manduca C, Kauchak D. Learning a lexical simplifier using Wikipedia. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics 2014; 2: 458–463.
• [18] Saggion H, Stajner S, Ferres D, Sheang KC, Shardlow M, North K, Zampieri M. Findings of the TSAR-2022 shared task on multilingual lexical simplification. In: Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022) 2022; 271–283.
• [19] Baez A, Saggion H. LSLlama: Fine-tuned LLaMA for lexical simplification. In: Proceedings of the Second Workshop on Text Simplification, Accessibility and Readability 2023; 102–108.
• [20] Touvron H, Lavril T, Izacard G, Martinet X, Lachaux MA, Lacroix T, Roziere B, Goyal N, Hambro E, Azhar F, et al. Llama: Open and efficient foundation language models. arXiv preprint 2023; arXiv:2302.13971.
• [21] Qiang J, Li Y, Zhu Y, Yuan Y, Wu X. Lexical simplification with pretrained encoders. In: Proceedings of the AAAI Conference on Artificial Intelligence 2020; 34: 8649–8656.
• [22] Aumiller D, Gertz M. UniHD at TSAR-2022 shared task: Is compute all we need for lexical simplification?. In: Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022) 2022; 251–258.
• [23] Jones R, Rey B, Madani O, Greiner W. Generating query substitutions. In: Proceedings of the 15th International Conference on World Wide Web WWW’06 2006; 387–396
• [24] Hambarde KA, Proença H. Information retrieval: Recent advances and beyond. IEEE Access 2023; 11: 76581–76604.
• [25] Macdonald C, Tonellotto N, Ounis I. Efficient & effective selective query rewriting with efficiency predictions. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval SIGIR’17 2017; 495–504.
• [26] Thakur N, Reimers N, Rückle A, Srivastava A, Gurevych I. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In: Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) 2021.
• [27] Wachsmuth H, Syed S, Stein B. Retrieval of the best counterargument without prior topic knowledge. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics 2018; 1: 241–251.
• [28] Maia M, Handschuh S, Freitas A, Davis B, McDermott R, Zarrouk M, Balahur A. WWW’18 open challenge: Financial opinion mining and question answering. In: Companion Proceedings of the The Web Conference 2018 WWW’18 2018; 1941–1942.
• [29] Boteva V, Gholipour D, Sokolov A, Riezler S. A full-text learning to rank dataset for medical information retrieval. In: Advances in Information Retrieval 2016; 716–722.
• [30] Cohan A, Feldman S, Beltagy I, Downey D, Weld D. SPECTER: Document-level representation learning using citation informed transformers. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics 2020; 2270–2282.
• [31] Wadden D, Lin S, Lo K, Wang LL, Zuylen M, Cohan A, Hajishirzi H. Fact or fiction: Verifying scientific claims. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) 2020; 7534–7550.
• [32] Voorhees E, Alam T, Bedrick S, Demner-Fushman D, Hersh WR, Lo K, Roberts K, Soboroff I, Wang LL. Trec-covid: constructing a pandemic information retrieval test collection. SIGIR Forum 2021; 54(1)
• [33] Robertson S, Zaragoza H. The probabilistic relevance framework: Bm25 and beyond. Found. Trends Inf. Retr. 2009; 3: 333–389.
• [34] Berger A, Caruana R, Cohn D, Freitag D, Mittal V. Bridging the lexical chasm: statistical approaches to answer-finding. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval SIGIR’00 2000; 192–199.
• [35] Karpukhin V, Oguz B, Min S, Lewis P, Wu L, Edunov S, Chen D, Yih Wt. Dense passage retrieval for open-domain question answering. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) 2020; 6769–6781.
• [36] Johnson J, Douze M, Jegou H. Billion-scale similarity search with gpus. IEEE Transactions on Big Data 2019; 7: 535–547.
• [37] Yang Y, Cer D, Ahmad A, Guo M, Law J, Constant N, Hernandez Abrego G, Yuan S, Tar C, Sung Yh, Strope B, Kurzweil R. Multilingual universal sentence encoder for semantic retrieval. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations 2020; 87–94.
• [38] Hofstatter S, Lin SC, Yang JH, Lin J, Hanbury A. Efficiently teaching an effective dense retriever with balanced topic aware sampling. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval SIGIR ’21 2021; 113–122.
• [39] Xiong L, Xiong C, Li Y, Tang KF, Liu J, Bennett PN, Ahmed J, Overwijk A. Approximate nearest neighbor negative contrastive learning for dense text retrieval. In: International Conference on Learning Representations 2021.
• [40] Zamani H, Dehghani M, Croft WB, Learned-Miller E, Kamps J. From neural re-ranking to neural ranking: Learning a sparse representation for inverted indexing. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management CIKM’18 2018; 497–506.
• [41] Zhao T, Lu X, Lee K. SPARTA: Efficient open-domain question answering via sparse transformer matching retrieval. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies 2021; 565–575.
• [42] Mallia A, Khattab O, Suel T, Tonellotto N. Learning passage impacts for inverted indexes. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval SIGIR’21 2021; 1723–1727.
• [43] Lin J, Ma X. A few brief notes on deepimpact, coil, and a conceptual framework for information retrieval techniques. arXiv preprint 2021; arXiv:2106.14807.
• [44] Formal T, Piwowarski B, Clinchant S. Splade: Sparse lexical and expansion model for first stage ranking. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval SIGIR’21 2021; 2288–2292.
• [45] Lassance C, Clinchant S. An efficiency study for splade models. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval SIGIR’22 2022; 2220–2226.
• [46] Formal T, Lassance C, Piwowarski B, Clinchant S. Splade v2: Sparse lexical and expansion model for information retrieval. arXiv preprint 2021; arXiv:2109.10086.
• [47] Jarvelin K, Kekalainen J. Ir evaluation methods for retrieving highly relevant documents. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval SIGIR’00 2000; 41–48.
• [48] Jarvelin K, Kekalainen J. Cumulated gain-based evaluation of ir techniques. ACM Trans. Inf. Syst. 2002; 20: 422–446.
• [49] Dinçer BT, Macdonald C, Ounis I, Hypothesis testing for the risk-sensitive evaluation of retrieval systems. In: Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval SIGIR’14 2014; 23–32.