Extractive Text Summarization System for News Texts

Abstract

In today's conditions, it is difficult to obtain information quickly and efficiently due to the size of the data. There are various text documents on the internet and a good extraction algorithm is essential to have the most relevant information from them. Long texts can be boring sometimes. So, readers are eager to get the main idea of the text or any useful information. For this reason, the importance of automatic summarization systems is understood. Text summarization systems can be considered as abstractive summarization or extractive summarization. While abstractive systems produce a summary with new sentences, extractive systems make a selection of sentences from the text used and combine them and present them as a summary. Creating a successful summarization algorithm increases in direct proportion to the success of applying text mining techniques. Text summary systems provide a summary of the text to the user by scoring words and sentences in the main text using various methods and combining high ranked sentences as a result of the process. In this context, many scoring methods have been used. In our study, news data sets are used. The algorithm used is based on extraction and has been evaluated using a task-independent method. After evaluation, the two highest scores taken are ROUGE-1 with 0.68 score and ROUGE-S with 0.54 score. Through all evaluation steps, Precision, Recall and F-Measure values are also specified to see the steps clearly.

Keywords

• Automatic text summarization
• Data processing
• Evaluation
• Feature extraction
• Sentence scoring

References

1. T. Sri Rama Raju and Bhargav Allarpu, Text Summarization using Sentence Scoring Method. April 2017. Volume: 04 Issue: 04 | pages 1777-1779
2. S.A. Babar and Pallavi D. Patil, Improving Performance of Text Summarization. Procedia Computer Science 46, 2015. 354 – 363, (ICICT 2014)
3. Lin, C.Y., ROUGE: A Package for Automatic Evaluation of Summaries. Spain, In Proceedings of the Workshop on Text Summarization Branches Out, 25 – 26 July 2004.
4. Josef Steinberger and Karel Ježek, Evaluatıon Measures For Text Summarızation. Computing and Informatics, March 2009, Vol. 28, 2009, 1001–1026.
5. Aysun Güran, Otomatik Metin Özetleme Sistemi. Yıldız Teknik Üniversitesi, Fen Bilimleri Enstitüsü, 2013
6. R. Satapathy and C. Guerreiro and I. Chaturvedi and E. Cambria, Phonetic-Based Microtext Normalization for Twitter Sentiment Analysis. IEEE International Conference on Data Mining Workshops (ICDMW), New Orleans, 2017, pp. 407-413, doi: 10.1109/ICDMW.2017.59
7. Kaynar, O. and Işık, Y.E and Görmez, Y. and Demirkoparan F., Genetic Algorithm Based Sentence Extraction For Automatic Text Summarization. Yönetim Bilişim Sistemleri Dergisi, 2017, Cilt:3, Sayı:2, Sayfa:62-75, ISSN: 2148-3752
8. Lin, Ch. and Hovy, E., Automatic Evaluation of Summaries Using n-Gram Co-Occurrence Statistics. Canada, In Proceedings of HLT-NAACL, 2003.

9. https://www.ccs.neu.edu/home/vip/teach/DMcourse/5_topicmodel_summ/notes_slides/What-is-ROUGE.pdf
10. https://github.com/nltk/nltk/blob/develop/nltk/util.py#L53
11. Gündoğdu, Ö.E. and Duru, N., Türkçe Metin Özetlemede Kullanılan Yöntemler. Aydın, 18. Akademik Bilişim Konferansı, , 30 Ocak-5 Şubat 2016, Adnan Menderes Üniversitesi.
12. P. Yıldırım and M. Uludağ and A. Görür, Hastane Bilgi Sistemlerinde Veri Madenciliği. Çanakkale, Akademik Bilişim, Ocak 2008, Çanakkale Onsekiz Mart Üniversitesi.
13. A.A. Akın and M.D. Akın, Zemberek, An Open Source Nlp Framework For Turkic Languages. 2007, Structure 10, 1-5, 185.
14. K. Deniz and B. Fatma and O. Akin and Y. Fatih and B. Emin, Metin Madenciliği Kullanılarak Yazılım Kullanımına Dair Bulguların Elde Edilmesi. 2015.
15. S. Çelik, Metin Madenciliği ile Shakespeare Külliyatının İncelenmesi. MANAS Sosyal Araştırmalar Dergisi, 9(3), 1343-1357.
16. Moratanch, N. and S. Chitrakala, A Survey On Extractive Text Summarization. Chennai, ICCCSP, 2017, 1-6. 10.1109/ICCCSP.2017.7944061.
17. https://www.kaggle.com/pariza/bbc-news-summary