A Journey Through Reviews: Exploring Machine Learning and Web Scraping to Decode Airline Sentiments

Abstract

This article explores the determinants of customer satisfaction in the airline industry by employing a data-driven approach that integrates web scraping with machine learning techniques. The study uses logistic regression and decision tree models to analyse a dataset of customer reviews from the top 10 airlines ranked by Skytrax. The primary focus is to assess how various service quality factors influence the likelihood of customers recommending an airline. The findings reveal that inflight entertainment, ground service, and value for money significantly impact customer recommendations, with inflight entertainment having a particularly strong negative effect. The originality of this article lies in its comprehensive application of both logistic regression and decision tree models to derive actionable insights for the airline industry, demonstrating the value of machine learning in predicting customer behaviour and enhancing service quality. This study analyzes customer sentiments and provides airlines with critical information to improve service offerings and ultimately increase customer loyalty. The results indicate that while both models perform well, logistic regression offers a slight edge in overall accuracy and recall, making it a robust tool for understanding and predicting customer satisfaction.

Keywords

• Web scraping
• customer satisfaction
• sentiment analysis
• logistic regression
• machine learning

References

1. Sanghyun Park, Jaehoon Lee, and José Nicolau. 2020. Understanding the dynamics of the quality of airline service attributes: Satisfiers and dissatisfiers. Tourism Management 81 (2020), 104163. https://doi.org/10.1016/j.tourman.2020.104163 google scholar
2. Chairul Pahrudin, Suci Anggiani, Renny Kristaung, Faisal Jasfar, and Wasis Arafah. 2023. The effect of service quality, price fairness, and corporate image on customer retention mediated by customer satisfaction on low-cost carrier airlines. Journal of Economics Finance and Management Studies 6, 1 (Jan. 2023), Article 09. https://doi.org/10.47191/jefms/v6-i1-09 google scholar
3. Muhammad A.-A. Yasim and Ahmad F. Ahmad Zaini. 2022. The influence of service quality towards customer satisfaction with low-cost airlines in Malaysia. Jurnal Evolusi 3, 2 (Dec. 2022), Article e42. https://doi.org/10.61688/jev.v3i2.42 google scholar
4. Theresia Angeline and Putu Belgiawan. 2023. The impact of airline responses to service failure towards customers’ satisfaction and loyalty in the airline industry. International Journal of Current Science Research and Review 6, 7 (Jul. 2023), 4968–4986. https:// doi.org/10.47191/ijcsrr/V6-i7-113 google scholar
5. Rajashree Dwesar and Debajani Sahoo. 2020. Does service failure criticality affect global travellers' service evaluations? Management Decision 60, 2 (2020), 426–448. https://doi.org/10.1108/md-01-2020-0107 google scholar
6. Sushil Thapa, Nischal Devkota, and Upama Paudel. 2020. Effect of service quality on customer satisfaction of domestic airlines in Nepal. Quest Journal of Management and Social Sciences 2, 2 (2020), 240–250. https://doi.org/10.3126/qjmss.v2i2.33285 google scholar
7. Maria Salamoura, Ioannis Chaniotakis, and Christos Lymperopoulos. 2017. Enhancing airline passengers’ satisfaction through service quality: The importance of the “human factor”. Journal of Air Transport Studies 8, 2 (2017), 54–69. https://doi.org/10. 38008/jats.v8i2.32 google scholar
8. Shadrack Dike, Zachary Davis, Alexander Abrahams, Amir Anjomshoae, and Patcharaphon Ractham. 2023. Evaluation of passengers' expectations and satisfaction in the airline industry: An empirical performance analysis of online reviews. Benchmarking: An International Journal 31, 2 (2023), 611–639. https://doi.org/10.1108/bij-09-2021-0563 google scholar

9. Xiaowen Xu, Wei Liu, and Dogan Gursoy. 2018. The impacts of service failure and recovery efforts on airline customers’ emotions and satisfaction. Journal of Travel Research 58, 6 (2018), 1034–1051. https://doi.org/10.1177/0047287518789285 google scholar
10. Ahmed Elbaz, Mohamed Soliman, Ahmed Al-Alawi, Basem Al-Romeedy, and Mohamed Mekawy. 2022. Customer responses to airline companies' service failure and recovery strategies. Tourism Review 78, 1 (2022), 1–17. https://doi.org/10.1108/tr-03-2022-0108 google scholar
11. Richard Matikiti, Mercy Mpinganjira, and Margaret Roberts-Lombard. 2019. Service recovery satisfaction and customer commitment in the airline business. African Journal of Economic and Management Studies 11, 1 (2019), 91–108. https://doi.org/10.1108/ ajems-01-2019-0005 google scholar
12. Shuai Wu and Ying Gao. 2023. Machine learning approach to analyze the sentiment of airline passengers’ tweets. Transportation Research Record 2678, 2 (2023), 48–56. https://doi.org/10.1177/03611981231172948 google scholar
13. Kittipong Chanpariyavatevong, Wichuda Wipulanusat, Thanapol Champahom, Somnuk Jomnonkwao, Duangporn Chonsalasin, and Vatanavongs Ratanavaraha. 2021. Predicting airline customer loyalty by integrating structural equation modeling and Bayesian networks. Sustainability 13, 13 (2021), Article 7046. https://doi.org/10.3390/su13137046 google scholar
14. Fotios Misopoulos, Maja Mitic, Andreas Kapoulas, and Constantinos Karapiperis. 2014. Uncovering customer service experiences with Twitter: The case of airline industry. Management Decision 52, 4 (2014), 705–723. https://doi.org/10.1108/md-03-2012-0235 google scholar
15. Maryam Tayaba. 2023. Transforming customer experience in the airline industry: A comprehensive analysis of Twitter sentiments using machine learning and association rule mining. Journal of Computer Science and Technology Studies 5, 4 (2023), 194–202. https://doi.org/10.32996/jcsts.2023.5.4.20 google scholar
16. Hossam Samir. 2023. Sentiment analysis model for airline customers’ feedback using deep learning techniques. International Journal of Engineering Business Management 15 (2023), Article 1026019. https://doi.org/10.1177/18479790231206019 google scholar
17. Gaurav Rasool and Ankur Pathania. 2021. Reading between the lines: Untwining online user-generated content using sentiment analysis. Journal of Research in Interactive Marketing 15, 3 (2021), 401–418. https://doi.org/10.1108/jrim-03-2020-0045 google scholar
18. Sharifah L. Idris and Mohamad Mohamad. 2023. A study on sentiment analysis on airline quality services: A conceptual paper. Information Management and Business Review 15, 4 (2023), 564–576. google scholar
19. Moe Win, Suhaili Ravana, and Liyana Shuib. 2022. Sentiment attribution analysis with hierarchical classification and automatic aspect categorization on online user reviews. Malaysian Journal of Computer Science 35, 2 (2022), 89–110. https://doi.org/10. 22452/mjcs.vol35no2.1 google scholar
20. Khairul Samah, Nurul Misdan, Mohd Jono, and Laila Riza. 2022. The best Malaysian airline companies visualization through bilingual Twitter sentiment analysis: A machine learning classification. JOIV: International Journal on Informatics Visualization 6, 1 (2022), Article 130. https://doi.org/10.30630/joiv.6.1.879 google scholar
21. Ghada Khairat and Amira Maher. 2014. Studying the influence of airlines' corporate social responsibility on consumers' loyalty. Journal of Association of Arab Universities for Tourism and Hospitality 11, 2 (2014), 167–186. https://doi.org/10.21608/jaauth.2014. 57124 google scholar
22. Ghada Khairat and Amira Maher. 2016. The influence of airlines' corporate social responsibility on customer loyalty. Journal of Association of Arab Universities for Tourism and Hospitality 13, 2 (2016), 71–92. https://doi.org/10.21608/jaauth.2016.48020 google scholar
23. Tarek Hassan and Ahmed Salem. 2021. Impact of service quality of low-cost carriers on airline image and consumers’ satisfaction and loyalty during the COVID-19 outbreak. International Journal of Environmental Research and Public Health 19, 1 (2021), Article 83. https://doi.org/10.3390/ijerph19010083 google scholar
24. Hani Alamoudi and Maha Alharthi. 2021. Antecedents and consequences of customer engagement: A case study of Saudi airline industry. Innovative Marketing 17, 3 (2021), 30–44. https://doi.org/10.21511/im.17(3).2021.03 google scholar
25. Ratna Hapsari, Michael Clemes, and David Dean. 2017. The impact of service quality, customer engagement, and selected marketing constructs on airline passenger loyalty. International Journal of Quality and Service Sciences 9, 1 (2017), 21–40. https:// doi.org/10.1108/ijqss-07-2016-0048 google scholar
26. Maxwell Sandada and Blessing Matibiri. 2016. An investigation into the impact of service quality, frequent flyer programs, and safety perception on satisfaction and customer loyalty in the airline industry in southern Africa. South East European Journal of Economics and Business 11, 1 (2016), 41–53. https://doi.org/10.1515/jeb-2016-0006 google scholar
27. Sunghoon Park, Minjeong Kim, and Yoonjung Kim. 2022. A deep learning approach to analyze airline customer propensities: The case of South Korea. Applied Sciences 12, 4 (2022), Article 1916. https://doi.org/10.3390/app12041916 google scholar
28. Brandon Christopher. 2023. Service quality impact on repurchase intentions: A pragmatic airline study. Tuijin Jishu/Journal of Propulsion Technology 44, 3 (2023), 2388–2397. https://doi.org/10.52783/tjjpt.v44.i3.718 google scholar
29. Hyungsoo Kwon, Hyeonsook Ban, Jaewook Jun, and Haejung Kim. 2021. Topic modeling and sentiment analysis of online reviews for airlines. Information 12, 2 (2021), Article 78. https://doi.org/10.3390/info12020078 google scholar
30. Wondimagegn Degife. 2024. A multi-aspect informed GRU: A hybrid model of flight fare forecasting with sentiment analysis. Applied Sciences 14, 10 (2024), Article 4221. https://doi.org/10.3390/app14104221 google scholar
31. Zhiwen Li. 2023. A comparative sentiment analysis of airline customer reviews using bidirectional encoder representations from transformers (BERT) and its variants. Mathematics 12, 1 (2023), Article 53. https://doi.org/10.3390/math12010053 google scholar
32. Naveen Singh and Manoj Upreti. 2023. HMRFLR: A hybrid model for sentiment analysis of social media surveillance on airlines. Wireless Personal Communications 132, 1 (2023), 97–112. https://doi.org/10.1007/s11277-023-10592-0 google scholar
33. Benedikt Badanik, Rebeka Remenysegova, and Antonin Kazda. 2023. Sentimental approach to airline service quality evaluation. Aerospace 10, 10 (2023), Article 883. https://doi.org/10.3390/aerospace10100883 google scholar
34. Nadine Hoffmann. 2022. Brand position in the eyes of customers: Assessment of selected airlines by the passengers' online reviews. Catallaxy 7, 1 (2022), 7–21. https://doi.org/10.24136/cxy.2022.001 google scholar
35. Airline Quality. 2024. Airline and airport reviews. (2024). Retrieved June 18, 2024 from https://www.airlinequality.com google scholar
36. Kenneth Reitz. 2023. Requests: HTTP for Humans. (2023). Retrieved from https://docs.python-requests.org/en/latest/ google scholar
37. Leonard Richardson. 2007. Beautiful Soup documentation. (2007). Retrieved from https://www.crummy.com/software/ BeautifulSoup/bs4/doc/ google scholar
38. Emmanuel Yaw Boateng and Daniel Asare Abaye. 2019. A review of the logistic regression model with emphasis on medical research. Journal of Data Analysis and Information Processing 7, 4 (2019), 190–207. https://doi.org/10.4236/jdaip.2019.74012 google scholar
39. Rui Duan, Yichi Ning, Jiayi Shi, Raymond J. Carroll, Tianxi Cai, and Yifei Chen. 2021. On the global identifiability of logistic regression models with misclassified outcomes. arXiv. Retrieved from https://doi.org/10.48550/arxiv.2103.12846 google scholar
40. Zaher Y. Algamal and Moon H. Lee. 2015. Applying penalized binary logistic regression with correlation-based elastic net for variable selection. Journal of Modern Applied Statistical Methods 14, 1 (2015), 168–179. https://doi.org/10.22237/jmasm/1430453640 google scholar
41. Mihyun Gil, Seonghoon Kim, and Eun Joo Min. 2022. Machine learning models for predicting risk of depression in Korean college students: Identifying family and individual factors. Frontiers in Public Health 10 (2022), Article 1023010. https://doi.org/10.3389/ fpubh.2022.1023010 google scholar
42. Arindam Basu, Arnab Ghosh, Maria Jaenada, and Leandro Pardo. 2021. Robust adaptive lasso in high-dimensional logistic regression with an application to genomic classification of cancer patients. arXiv. Retrieved from https://doi.org/10.48550/arxiv. 2109.03028 google scholar
43. Leo Breiman, Jerome Friedman, Richard A. Olshen, and Charles J. Stone. 1984. Classification and regression trees (1st ed.). Chapman and Hall/CRC, New York, NY. https://doi.org/10.1201/9781315139470 google scholar
44. Inmarsat. 2018. Inflight connectivity survey – Global whitepaper. Inmarsat Global Limited, London, UK. Retrieved from https://www.inmarsat.com/content/dam/inmarsat/corporate/documents/aviation/insights/2018/Inmarsat%20 Aviation%202018%20Inflight%20Connectivity%20Survey%20ENG.pdf google scholar
45. Patricia Lippitt, Nadine Itani, John F. O’Connell, David Warnock-Smith, and Marina Efthymiou. 2023. Investigating airline service quality from a business traveller perspective through the integration of the Kano model and importance–satisfaction analysis. Sustainability 15, 8 (2023), Article 6578. https://doi.org/10.3390/su15086578 google scholar
46. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 12 (2011), 2825–2830. google scholar