MOCS-BERT: Multi-Objective Cuckoo Search with Sentence-BERT Embeddings for Semantically Enhanced Extractive Summarization

Abstract

This study presents MOCS-BERT, a novel extractive text summarization framework that effectively integrates multi-objective Cuckoo Search Optimization with Sentence-BERT embeddings to generate semantically coherent and readable summaries. Evaluated on the full CNN/DailyMail test set comprising 11,490 documents, the proposed model demonstrates statistically significant superiority over three established metaheuristic algorithms namely Bat Algorithm (BA), Flower Pollination Algorithm (FPA), and Firefly Algorithm (FA) as confirmed by the Wilcoxon signed-rank test (p = 0.0432 and p = 0.0010, respectively). Although it does not show statistical significance against Flower Pollination Algorithm (p = 0.0990) among the three baseline metaheuristic algorithms evaluated (BA, FPA, FA), MOCS-BERT consistently achieves the highest ROUGE-L F1 score (0.1895), underscoring its exceptional ability to preserve narrative coherence and logical structure in generated summaries. Furthermore, the model produces highly readable output, with a Flesch Reading Ease of 66.9 and low grade-level indices, making it accessible to a broad audience. These results validate that the integration of deep semantic representations with a carefully designed multi-objective fitness function balancing semantic relevance, non-redundancy, and readability yields a robust, scalable summarization system with balanced performance across semantic coherence, redundancy control, and readability metrics. The research not only progresses the methodological boundaries of metaheuristic-based text summarization but also provides practical utility in real-world applications, including news aggregation, legal document analysis, and emergency response assistance.. Future work will focus on human evaluation, domain-specific adaptation, and automated hyperparameter tuning to further enhance performance and generalizability. 

Keywords

• Text summarization
• Cuckoo search
• Sentence-BERT
• Multi-objective optimization
• Natural language processing

References

1. [1] Luo, M., Xue, B. and Niu, B., “A comprehensive survey for automatic text summarization: techniques, approaches and perspectives”, Neurocomputing, 603: 128280, (2024). DOI: https://doi.org/10.1016/j.neucom.2024.128280
2. [2] Peyrard, M., “A simple theoretical model of importance for summarization”, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 1059–1073, (2019). DOI: 10.18653/v1/P19-1101
3. [3] Devlin, J., Chang, M.-W., Lee, K. and Toutanova, K., “BERT: Pre-training of deep bidirectional transformers for language understanding”, Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 4171–4186, (2019). DOI: 10.18653/v1/N19-1423
4. [4] Sharma, G. and Sharma, D., “Automatic text summarization methods: a comprehensive review”, SN Computer Science, 4(1): 33, (2022). DOI: https://doi.org/10.1007/s42979-022-01446-w
5. [5] Liu, Y. and Lapata, M., “Text summarization with pretrained encoders”, Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 3721–3731, (2019). DOI: https://doi.org/10.48550/arXiv.1908.08345
6. [6] Liu, Y., “Fine-tune BERT for extractive summarization”, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 3808–3818, (2019). DOI: https://doi.org/10.48550/arXiv.1903.10318
7. [7] See, A., Liu, P. J. and Manning, C. D., “Get to the point: summarization with pointer-generator networks”, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 1073–1083, (2017). DOI: https://doi.org/10.48550/arXiv.1704.04368
8. [8] Zhou, Q., Yang, N., Wei, F., Huang, S., Zhou, M. and Zhao, T., “Neural document summarization by jointly learning to score and select sentences”, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 654–663, (2018). DOI: 10.18653/v1/P18-1061

9. [9] Sidi, H. O., Ellaia, R., Pagnacco, E. and Zeine, A. T., “Multiobjective design optimization using flower pollination algorithm”, Proceedings of the 7th International Conference on Optimization and Applications (ICOA), 1–4, (2021). DOI: 10.1109/ICOA51614.2021.9442651.
10. [10] Krishnan, N., “KnowSum: knowledge inclusive approach for text summarization using semantic alignment”, Proceedings of the 7th International Conference on Web Research (ICWR), 227–231, (2021). DOI: 10.1109/ICWR51868.2021.9443149
11. [11] Reimers, N. and Gurevych, I., “Sentence-BERT: sentence embeddings using siamese bert-networks”, Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 3982–3992, (2019). DOI: 10.18653/v1/D19-1410
12. [12] Pati, S. P. and Rautray, R., “SMATS: single and multi automatic text summarization”, Karbala International Journal of Modern Science, 9(1), (2023). DOI: 10.33640/2405-609X.3281
13. [13] Ali, Z. H., Hussein, A. K., Abass, H. K. and Fadel, E., “Extractive multi document summarization using harmony search algorithm”, TELKOMNIKA (Telecommunication Computing Electronics and Control), 19(1): 89–95, (2021). DOI: http://doi.org/10.12928/telkomnika.v19i1.15766
14. [14] Rautray, R. and Balabantaray, R. C., “An evolutionary framework for multi document summarization using cuckoo search approach: MDSCSA”, Applied Computing and Informatics, 14(2): 134–144, (2018). DOI: https://doi.org/10.1016/j.aci.2017.05.003
15. [15] Hassan, A. Q., Al-onazi, B. B., Maashi, M., Darem, A. A., Abunadi, I. and Mahmud, A., “Enhancing extractive text summarization using natural language processing with an optimal deep learning model”, AIMS Mathematics, 9(5): 12588–12609, (2024). DOI: 10.3934/math.2024616
16. [16] Premalatha, M., Jayasudha, M., Čep, R., Priyadarshini, J., Kalita, K. and Chatterjee, P., “A comparative evaluation of nature-inspired algorithms for feature selection problems”, Heliyon, 10(1), (2024). DOI: https://doi.org/10.1016/j.heliyon.2023.e23571
17. [17] Dhas, M. M. A. and Chandrasekaran, S., “Particle swarm intelligence based univariate parameter tuning of recursive least square algorithm for optimal heart sound signal filtering”, Gazi University Journal of Science, 32(3): 928–943, (2019). DOI: https://doi.org/10.35378/gujs.501114
18. [18] Gürcan, Ö. F., Atıcı, U. and Beyca, Ö. F., “A hybrid deep learning–metaheuristic model for diagnosis of diabetic retinopathy”, Gazi University Journal of Science, 36(2): 693–703, (2023). DOI: 10.35378/gujs.919572
19. [19] Alharbi, A. H., Rizk, F. H., Gaber, K. S., Eid, M. M., El-Kenawy, E. S. M., Khodadadi, E. and Khodadadi, N., “Hybrid deep learning optimization for smart agriculture: Dipper throated optimization and polar rose search applied to water quality prediction”, PLOS ONE, 20(7): e0327230, (2025). DOI: https://doi.org/10.1371/journal.pone.0327230
20. [20] Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q. and Artzi, Y., “BERTScore: evaluating text generation with BERT”, International Conference on Learning Representations (ICLR), (2020). DOI: https://doi.org/10.48550/arXiv.1904.09675
21. [21] Ryu, S., Do, Y., Kim, Y., Lee, G. and Ok, J., “Multi-dimensional optimization for text summarization via reinforcement learning”, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 5858–5871, (2024). DOI: https://doi.org/10.48550/arXiv.2406.00303
22. [22] Hermann, K. M., Kocisky, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M. and Blunsom, P., “Teaching machines to read and comprehend”, Advances in Neural Information Processing Systems, 28, (2015). DOI: https://doi.org/10.48550/arXiv.1506.03340
23. [23] Yang, X.-S. and Deb, S., “Cuckoo search via lévy flights”, Proceedings of the World Congress on Nature & Biologically Inspired Computing (NaBIC), (2009). DOI: 10.1109/NABIC.2009.5393690
24. [24] Rani Krishna, K. M., Somasundaram, K., Arulmozhivarman, P., Immanuel, S. A. and Rajkumar, E. R., “Deep learning for text summarization using NLP for automated news digest”, Scientific Reports, 15(1): 36343, (2025). DOI: https://doi.org/10.1038/s41598-025-20224-1
25. [25] Lin, C.-Y., “ROUGE: A package for automatic evaluation of summaries”, Text Summarization Branches Out, (2004). https://aclanthology.org/W04-1013/
26. [26] Kincaid, J. P., Fishburne, R. P., Rogers, R. L. and Chissom, B. S., “Derivation of new readability formulas for navy enlisted personnel”, Research Report, (1975). DOI: 10.21236/ada006655
27. [27] Yaman, Y., Kızılörenli, E. and Tokuç, A., “Multi-objective optimization of a folding kinetic facade system proposal for thermal, daylight, and energy performances”, Gazi University Journal of Science, 38(1): 1–16, (2025). DOI: https://doi.org/10.35378/gujs.1433809
28. [28] Vrbančič, G., Brezočnik, L., Mlakar, U., Fister, D. and Fister Jr., I., “NiaPy: python microframework for building nature-inspired algorithms”, Journal of Open Source Software, 3(28): 613, (2018). DOI: 10.21105/joss.00613