Why Sentiment Analysis-Based Metrics are Essential for Measuring Channel Performance on YouTube: An Experimental Study

Year 2024, Volume: 12 Issue: 2, 1086 - 1100, 29.04.2024

Hakan Elbaş Alparslan Mesri

https://doi.org/10.29130/dubited.1190860

Abstract

YouTube is a universal social medium that lets its users share videos and write comments on shared videos. Comments of the users on Youtube videos can be useful for YouTube channel owners, and it is worth analyzing them in terms of sentiment. The rate of like/dislike on a YouTube video is not sufficient for estimating the attitude of users towards it. This study proposes a three-step method for this attitude: In the first step, a sentiment analysis task is applied to the user comments on the videos of the YouTube channel "iJustin". In the second step, a new metric named Sentiment Index (SI) has been proposed and the Sentiment Indexes of the videos have been calculated. In the third step, an analysis is performed to show if the SI metric is time-independent. As a result, it was seen that most of the comments left on the videos (89%) have been written within the first 30 days after the video was published. Our experiments revealed that comments left more than 30 days after publishing the videos change the average SI values of the videos by only 0.4%, and in this respect, the SI metric is negligibly affected by the time parameter.

Keywords

Natural Language Processing, Sentiment Analysis, Data Science, Youtube, Opinion Mining, Social Media

Thanks

We want to thank Prof. Rahim Dehkharghani for his support.

YouTube'da Kanal Performansını Ölçmek İçin Neden Duygu Analizi Temelli Metrikler Gereklidir: Deneysel Bir Çalışma

Abstract

YouTube, kullanıcılarının video paylaşmasına ve paylaşılan videolara yorum yazmasına olanak tanıyan evrensel bir sosyal ortamdır. Kullanıcıların Youtube videolarına yaptığı yorumlar YouTube kanal sahipleri için faydalı olabilir. Bir YouTube videosundaki beğen/beğenme oranı, kullanıcıların videoya yönelik tutumunu tahmin etmek için yeterli değildir. Bu çalışma, bu tutum için üç aşamalı bir yöntem önermektedir: İlk adımda, "iJustin" YouTube kanalının videolarındaki kullanıcı yorumlarına bir duygu analizi görevi uygulanmaktadır. İkinci adımda, Duygu İndeksi (SI) adlı yeni bir metrik önerilmiş ve videoların Duygu İndeksleri hesaplanmıştır. Üçüncü adımda, SI metriğinin zamandan bağımsız olup olmadığını göstermek için bir analiz yapılmıştır. Sonuç olarak videolara yapılan yorumların çoğunun (%89) video yayınlandıktan sonraki ilk 30 gün içinde yazıldığı görüldü. Deneylerimiz, videoları yayınladıktan sonra 30 günden fazla kalan yorumların, videoların ortalama SI değerlerini yalnızca %0,4 oranında değiştirdiğini ve bu açıdan SI metriğinin zaman parametresinden ihmal edilebilir düzeyde etkilendiğini ortaya koymuştur.

Keywords

Doğal Dil İşleme, Duygu Analizi, Veri Bilimi, YouTube, Fikir Madenciliği, Sosyal Medya

References

• [1] Sandwine, “Global Internet Phenomena Report 2022”, sandvine.com, https://www.sandvine.com/hubfs/Sandvine_Redesign_2019/Downloads/2022/Phenomena%20Reports/GIPR%202022/Sandvine%20GIPR%20January%202022.pdf (accessed June 7, 2022)
• [2] “YouTube Ranking”, similarweb.com, https://www.similarweb.com/tr/website/youtube.com/#ranking (accessed June 10, 2022)
• [3] “Youtube’s Advertising Revenues”, statista.com, https://www.statista.com/statistics/289659/youtube-share-of-google-total-ad-revenues/#:~:text=In%202021%2C%20YouTube's%20advertising%20revenue,dollars%20in%20the%20previous%20year (accessed June 11, 2022)
• [4] M. Hall, “How Do People Make Money on YouTube”, investopedia.com, https://www.investopedia.com/ask/answers/012015/how-do-people-make-money-videos-they-upload-youtube.asp#:~:text=YouTube%20monetizes%20videos%20via%20pre,must%20adhere%20to%20advertising%20guidelines (accessed July 14, 2022)
• [5] GMI, “YouTube Users Statistics 2022”, globalmediainsight.com, https://www.globalmediainsight.com/blog/youtube-users-statistics/ (accessed June 28, 2022)
• [6] M. L. Khan, "Social media engagement: What motivates user participation and consumption on YouTube?", Computers in human behavior, vol. 66, pp. 236-247, 2017.
• [7] YouTube Team, “An update to dislikes on YouTube”, blog.youtube, https://blog.youtube/news-and-events/update-to-youtube/ (accessed September 7, 2022)
• [8] The Daily Iowan, “Ten Best Sites to Buy YouTube Likes”, dailyiowan.com, https://dailyiowan.com/2022/05/13/buy-youtube-likes-2/#:~:text=Buying%20a%20YouTube%20like%20is,give%20your%20channel%20a%20boost (accessed May 13, 2022)
• [9] D. M. Thomas and S. Mathur, "Data analysis by web scraping using python", in 2019 Proceedings of the 3rd International conference on Electronics, Communication and Aerospace Technology, Coimbatore, India, 2019, pp. 450-454.
• [10] S. Tokcaer, "Türkçe Metinlerde Duygu Analizi", Journal of Yaşar University, vol. 16, no. 63, pp. 1514-1534, 2021. [11] R. Dehkharghani and C. Yilmaz, "Automatically identifying a software product's quality attributes through sentiment analysis of tweets", IEEE 1st International Workshop on Natural Language Analysis in Software Engineering (NaturaLiSE), San Francisco, Ca, USA, 2013, pp. 25-30.
• [12] P. Mehta and S. Pandya, "A review on sentiment analysis methodologies, practices and applications", International Journal of Scientific and Technology Research, vol. 9.2, pp. 601-609, 2020.
• [13] Z. Drus and H. Khalid, "Sentiment analysis in social media and its application: Systematic literature review", Procedia Computer Science, vol. 161, pp. 707-714, 2019.
• [14] C. J. Hutto and E. Gilbert, "Vader: A parsimonious rule-based model for sentiment analysis of social media text", in Proceedings of the international AAAI conference on web and social media, vol. 8, no. 1, 2014, pp. 216-225.
• [15] R. F. Alhujaili and W. M. S. Yafooz, "Sentiment analysis for youtube videos with user comments", 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS), Coimbatore, India.
• [16] H. C. Ko and W. N. Wu, “Exploring the determinants of viewers' loyalty toward beauty YouTubers: a parasocial interaction perspective”, Proceedings of ICEMT-2017, Singapore, pp. 81-86.
• [17] A. Mesri, “Web Mining and Sentiment Analysis of a Bank Software by User Comments”, M.S. Thesis, Hacettepe University, Ankara, 2017, Available: https://www.academia.edu/72647117/Web_Mining_and_Sentiment_Analysis_of_a_Bank_Software_by_User_Comments
• [18] A. Severyn, A. Moschitti, O. Uryupina, B. Plank, K. Filippova, "Multi-lingual opinion mining on YouTube", Information Processing & Management, vol. 52.1, pp. 46-60, 2016.
• [19] H. Bhuiyan, J. Ara, R. Bardhan, R. Islam, "Retrieving YouTube video by sentiment analysis on user comment", IEEE ICSIPA-2017, Kuching, Malaysia, pp. 474-478.
• [20] G. M. H. C. Gajanayake and T. C. Sandanayake, "Trending Pattern Identification of YouTube Gaming Channels Using Sentiment Analysis", IEEE 20th ICTer-2020, Colombo, Sri Lanka, 2020.
• [21] R. Pradhan, "Extracting Sentiments from YouTube Comments", IEEE Sixth of ICIIP-2021, vol. 6, Shimla, India, pp. 1-4.
• [22] H. Timani, P. Shah, M. Joshi, "Predicting success of a movie from youtube trailer comments using sentiment analysis", IEEE 6th INDIACom-2019, New Delhi, India, pp. 584-586.
• [23] S. Singh and G. Sikka, "YouTube Sentiment Analysis on US Elections 2020", IEEE 2nd ICSCCC-2021, Jalandhar, India, pp. 250-254.
• [24] X. Chen, M. Vorvoreanu, K. Madhavan, "Mining social media data for understanding students’ learning experiences", IEEE Transactions on learning technologies, vol. 7.3, pp. 246-259, 2014.
• [25] C. S. Lee, H. Osop, D. H. Goh, G. Kelni, "Making sense of comments on YouTube educational videos: a self-directed learning perspective", Online Information Review, vol. 41.5, pp. 611-625, 2017.
• [26] C. Richier, E. Altman, R. Elazouzi, T. Altman, G. Linares, Y. Portilla, "Modelling view-count dynamics in youtube", arXiv preprint, arXiv:1404.2570, 2014.
• [27] J. C. M. Serrano, O. Papakyriakopoulos, S. Hegelich, "NLP-based feature extraction for the detection of COVID-19 misinformation videos on YouTube", in Proceedings of the 1st Workshop on NLP for COVID-19 2020, online.
• [28] “Distribution of total YouTube video content worldwide as of December 2018, by category”, statista.com, https://www.statista.com/statistics/1026914/global-distribution-youtube-video-content-by-category/ (accessed June 11, 2022)
• [29] A. Krouska, C. Troussas, M. Virvou, "The effect of preprocessing techniques on Twitter sentiment analysis", IEEE 7th IISA, Chalkidiki, Greece, 2016, pp. 1-5.
• [30] C. Dhaoui, C. M. Webster, L. P. Tan, "Social media sentiment analysis: lexicon versus machine learning", Journal of Consumer Marketing, vol. 34, no. 6, pp. 480-488, 2017.