<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.4 20241031//EN"
        "https://jats.nlm.nih.gov/publishing/1.4/JATS-journalpublishing1-4.dtd">
<article  article-type="research-article"        dtd-version="1.4">
            <front>

                <journal-meta>
                                                                <journal-id>saucis</journal-id>
            <journal-title-group>
                                                                                    <journal-title>Sakarya University Journal of Computer and Information Sciences</journal-title>
            </journal-title-group>
                                        <issn pub-type="epub">2636-8129</issn>
                                                                                            <publisher>
                    <publisher-name>Sakarya University</publisher-name>
                </publisher>
                    </journal-meta>
                <article-meta>
                                        <article-id pub-id-type="doi">10.35377/saucis...1675280</article-id>
                                                                <article-categories>
                                            <subj-group  xml:lang="en">
                                                            <subject>Computer Software</subject>
                                                            <subject>Software Engineering (Other)</subject>
                                                    </subj-group>
                                            <subj-group  xml:lang="tr">
                                                            <subject>Bilgisayar Yazılımı</subject>
                                                            <subject>Yazılım Mühendisliği (Diğer)</subject>
                                                    </subj-group>
                                    </article-categories>
                                                                                                                                                        <title-group>
                                                                                                                                                            <article-title>Bi-directional Encoder Representations from Transformers Based for Sentiment Analysis from Consumer Reviews</article-title>
                                                                                                    </title-group>
            
                                                    <contrib-group content-type="authors">
                                                                        <contrib contrib-type="author">
                                                                    <contrib-id contrib-id-type="orcid">
                                        https://orcid.org/0000-0003-0396-7885</contrib-id>
                                                                <name>
                                    <surname>Göker</surname>
                                    <given-names>Hanife</given-names>
                                </name>
                                                                    <aff>GAZİ ÜNİVERSİTESİ</aff>
                                                            </contrib>
                                                                                </contrib-group>
                        
                                        <pub-date pub-type="pub" iso-8601-date="20250930">
                    <day>09</day>
                    <month>30</month>
                    <year>2025</year>
                </pub-date>
                                        <volume>8</volume>
                                        <issue>3</issue>
                                        <fpage>484</fpage>
                                        <lpage>495</lpage>
                        
                        <history>
                                    <date date-type="received" iso-8601-date="20250413">
                        <day>04</day>
                        <month>13</month>
                        <year>2025</year>
                    </date>
                                                    <date date-type="accepted" iso-8601-date="20250817">
                        <day>08</day>
                        <month>17</month>
                        <year>2025</year>
                    </date>
                            </history>
                                        <permissions>
                    <copyright-statement>Copyright © 2018, Sakarya University Journal of Computer and Information Sciences</copyright-statement>
                    <copyright-year>2018</copyright-year>
                    <copyright-holder>Sakarya University Journal of Computer and Information Sciences</copyright-holder>
                </permissions>
            
                                                                                                                        <abstract><p>Structured data has a standardized format for easy access, organization, and categorization. However, approximately 95% of data, such as text files or online reviews, is unstructured, and these texts do not have standard rules. Unstructured data analysis, especially when the amount of data to be examined is substantial, requires considerable effort, cost, and time, and classical statistical methods are often insufficient. Transformer models, the latest technological models in natural language processing (NLP), are the strongest candidates to overcome these limits. In this paper, we propose the bi-directional encoder representations from transformers (BERT) model-based solution for sentiment analysis of consumer reviews. The dataset comprises 10975 consumer reviews of technological products from an e-commerce platform and was transformed into a structured dataset using data preprocessing. Then, we compared the performance of the BERT transformer model with deep learning models, specifically convolutional neural networks (CNN), long short-term memory (LSTM), and bidirectional long short-term memory (B-LSTM). Experimental results confirmed that the BERT transformer model achieved a higher kappa of 96.6% and an overall accuracy of 97.78% for multi-classification of consumer reviews. The proposed transformer-based model outperforms the state-of-the-art models, providing a reliable and efficient solution.</p></abstract>
                                                            
            
                                                                                        <kwd-group>
                                                    <kwd>Transformer model</kwd>
                                                    <kwd>  Deep learning</kwd>
                                                    <kwd>  Natural language processing</kwd>
                                                    <kwd>  Sentiment analysis</kwd>
                                                    <kwd>  E-commerce</kwd>
                                            </kwd-group>
                            
                                                                                                                                                    </article-meta>
    </front>
    <back>
                            <ref-list>
                                    <ref id="ref1">
                        <label>1</label>
                        <mixed-citation publication-type="journal">A. Andonov, G. P. Dimitrov, and V. Totev, &quot;Impact of e-commerce on business performance, &quot; TEM Journal, vol. 10, no. 4, pp. 1558-1564, 2021.</mixed-citation>
                    </ref>
                                    <ref id="ref2">
                        <label>2</label>
                        <mixed-citation publication-type="journal">Ö. Alkan and Y. C. Bayhan, &quot;COVID-19 pandemic and use of e-commerce: generational differences,&quot; Journal of Economics and Administrative Sciences, vol. 24, no. 2, pp. 170-185, 2023, doi: 10.37880/cumuiibf.1179003.</mixed-citation>
                    </ref>
                                    <ref id="ref3">
                        <label>3</label>
                        <mixed-citation publication-type="journal">TUBISAD, &quot;The transformative power of economy: e-commerce impact analysis,&quot; Jun. 2021. [Online]. Available: https://www.tubisad.org.tr/en/images/pdf/the-transformative-power.pdf. [Accessed: Feb. 5, 2025].</mixed-citation>
                    </ref>
                                    <ref id="ref4">
                        <label>4</label>
                        <mixed-citation publication-type="journal">J. Mosteller and C. Mathwick, &quot;Consumers helping consumers; the role of psychological need fulfillment in an online reviewer community,&quot; in Let’s get engaged! crossing the threshold of marketing’s engagement era, M. Obal, N. Krey, and C. Bushardt, Eds. Cham, Switzerland: Springer, 2016, pp. 106, doi: 10.1007/978-3-319-11815-4_106.</mixed-citation>
                    </ref>
                                    <ref id="ref5">
                        <label>5</label>
                        <mixed-citation publication-type="journal">S. C. Nune and I. Kozhıkode, &quot;Characterizing and predicting early reviewers for effective product marketing on ecommerce websites,&quot; Turkish Journal of Computer and Mathematics Education, vol. 12, no. 14, pp. 2870-2877, 2021.</mixed-citation>
                    </ref>
                                    <ref id="ref6">
                        <label>6</label>
                        <mixed-citation publication-type="journal">M. Yang, Y. Ren, and G. Adomavicius, &quot;Understanding user-generated content and customer engagement on Facebook business pages,&quot; Information Systems Research, vol. 30, no. 3, pp. 839-855, 2019, doi: 10.1287/isre.2019.0834.</mixed-citation>
                    </ref>
                                    <ref id="ref7">
                        <label>7</label>
                        <mixed-citation publication-type="journal">S. L. Shengli and L. Fan, &quot;The interaction effects of online reviews and free samples on consumers’ downloads: An empirical analysis,&quot; Information Processing and Management, vol. 56, pp. 1-12, 2019, doi: 10.1016/j.ipm.2019.102071.</mixed-citation>
                    </ref>
                                    <ref id="ref8">
                        <label>8</label>
                        <mixed-citation publication-type="journal">D. Gavilan, M. Avello, and G. Martinez-Navarro, &quot;The influence of online ratings and reviews on hotel booking consideration,&quot; Tourism Management, vol. 66, pp. 53-61, 2018, doi: 10.1016/j.tourman.2017.10.018.</mixed-citation>
                    </ref>
                                    <ref id="ref9">
                        <label>9</label>
                        <mixed-citation publication-type="journal">P.Y. Chen, S. Dhanasobhon, and M. D. Smith, &quot;All reviews are not created equal: The disaggregate impact of reviews and reviewers at Amazon.com,&quot; SSRN Electronic Journal, 2008, doi: 10.2139/ssrn.918083.</mixed-citation>
                    </ref>
                                    <ref id="ref10">
                        <label>10</label>
                        <mixed-citation publication-type="journal">R. Thakur, &quot;Customer engagement and online reviews,&quot; Journal of Retailing and Consumer Services, vol. 41, pp. 48-59, 2018, doi: 10.1016/j.jretconser.2017.11.002.</mixed-citation>
                    </ref>
                                    <ref id="ref11">
                        <label>11</label>
                        <mixed-citation publication-type="journal">J. H. Huang and Y. F. Chen, &quot;Herding in online product choice,&quot; Psychology and Marketing, vol. 23, no. 5, pp. 413-428, 2006, doi: 10.1002/mar.20119.</mixed-citation>
                    </ref>
                                    <ref id="ref12">
                        <label>12</label>
                        <mixed-citation publication-type="journal">A. Batta, A. K. Kar, and S. Satpathy, &quot;Cross-platform analysis of seller performance and churn for ecommerce using artificial intelligence,&quot; Journal of Global Information Management (JGIM), vol. 31, no. 1, pp. 1-21, 2023.</mixed-citation>
                    </ref>
                                    <ref id="ref13">
                        <label>13</label>
                        <mixed-citation publication-type="journal">K. Adnan, R. Akbar, and K. S. Wang, &quot;Information extraction from multifaceted unstructured big data,&quot; International Journal of Recent Technology and Engineering (IJRTE), vol. 8, pp. 1398-1404, 2019, doi: 10.35940/ijrte.B1074.0882S819.</mixed-citation>
                    </ref>
                                    <ref id="ref14">
                        <label>14</label>
                        <mixed-citation publication-type="journal">T. Jo, Text mining Concepts, Implementation, and Big Data Challenge, New York: Springer, 2019.</mixed-citation>
                    </ref>
                                    <ref id="ref15">
                        <label>15</label>
                        <mixed-citation publication-type="journal">H. Göker, H. I.Bülbül, and E. Irmak, &quot;The estimation of students&#039; academic success by data mining methods.&quot; In 12th International Conference on Machine Learning and Applications, vol. 2, pp. 535-539, 2013, doi: 10.1109/ICMLA.2013.173.</mixed-citation>
                    </ref>
                                    <ref id="ref16">
                        <label>16</label>
                        <mixed-citation publication-type="journal">W. Hong, C. Zheng, L. Wu, and X. Pu, &quot;Analyzing the relationship between consumer satisfaction and fresh e-commerce logistics service using text mining techniques,&quot; Sustainability, vol. 11, no. 13, pp. 1-16, 2019, doi: 10.3390/su11133570.</mixed-citation>
                    </ref>
                                    <ref id="ref17">
                        <label>17</label>
                        <mixed-citation publication-type="journal">Z. Shen, &quot;Mining sustainable fashion e-commerce: social media texts and consumer behaviors,&quot; Electronic Commerce Research, pp. 1-23, 2021, doi: 10.1007/s10660-021-09498-5.</mixed-citation>
                    </ref>
                                    <ref id="ref18">
                        <label>18</label>
                        <mixed-citation publication-type="journal">E. Sezgen, K. J. Mason, and R. Mayer, &quot;Voice of airline passenger: A text mining approach to understand customer satisfaction,&quot; Journal of Air Transport Management, vol. 77, pp. 65-74, 2019, doi: 10.1016/j.jairtraman.2019.04.001.</mixed-citation>
                    </ref>
                                    <ref id="ref19">
                        <label>19</label>
                        <mixed-citation publication-type="journal">P. Sobkowicz, M. Kaschesky, and G. Bouchard, &quot;Opinion mining in social media: Modeling, simulating, and forecasting political opinions in the web,&quot; Government Information Quarterly, vol. 29, no. 4, pp. 470-479, 2012, doi: 10.1016/j.giq.2012.06.005.</mixed-citation>
                    </ref>
                                    <ref id="ref20">
                        <label>20</label>
                        <mixed-citation publication-type="journal">M. O. Aftab, U. Ahmad, S. Khalid, A. Saud, A. Hassan, and M. S. Farooq, &quot;Sentiment analysis of customer for ecommerce by applying AI,&quot; In 2021 International Conference on Innovative Computing (ICIC), Nov. 2021, pp. 1-7, doi: 10.1109/ICIC53490.2021.9693026.</mixed-citation>
                    </ref>
                                    <ref id="ref21">
                        <label>21</label>
                        <mixed-citation publication-type="journal">J. Jabbar, I. Urooj, W. JunSheng, and N. Azeem, &quot;Real-time sentiment analysis on E-commerce application,&quot; In 16th International Conference on Networking, Sensing and Control (ICNSC), May 2019, pp. 391-396, doi: 10.1109/ICNSC.2019.8743331.</mixed-citation>
                    </ref>
                                    <ref id="ref22">
                        <label>22</label>
                        <mixed-citation publication-type="journal">S. Dey, S. Wasif, D. S. Tonmoy, S. Sultana, J. Sarkar, and M. Dey, &quot;A comparative study of support vector machine and Naive Bayes classifier for sentiment analysis on Amazon product reviews,&quot; In International Conference on Contemporary Computing and Applications (IC3A), Feb. 2020, pp. 217-220, doi: 10.1109/IC3A48958.2020.233300.</mixed-citation>
                    </ref>
                                    <ref id="ref23">
                        <label>23</label>
                        <mixed-citation publication-type="journal">M. R. Pratama, F. A. G. Soerawinata, R. R. Zhafari, and H. N. Irmanda, &quot;Sentiment analysis of beauty product e-commerce using support vector machine method,&quot; Jurnal RESTI (Rekayasa Sistem Dan Teknologi Informasi), vol. 6, no. 2, pp. 269-274, 2022, doi: 10.29207/resti.v6i2.3876.</mixed-citation>
                    </ref>
                                    <ref id="ref24">
                        <label>24</label>
                        <mixed-citation publication-type="journal">S. Paul and S. Saha, &quot;CyberBERT: BERT for cyberbullying identification: BERT for cyberbullying identification,&quot; Multimedia Systems, vol. 28, no. 6, pp. 1897-1904, 2022, doi: 10.1007/s00530-020-00710-4.</mixed-citation>
                    </ref>
                                    <ref id="ref25">
                        <label>25</label>
                        <mixed-citation publication-type="journal">M. Panda, &quot;Developing an efficient text pre-processing method with sparse generative Naive Bayes for text mining,&quot; International Journal of Modern Education and Computer Science, vol. 11, no. 9, pp. 11-19, 2018, doi: 10.5815/ijmecs.2018.09.02.</mixed-citation>
                    </ref>
                                    <ref id="ref26">
                        <label>26</label>
                        <mixed-citation publication-type="journal">S. Vijayarani and R. Janani, &quot;Text mining: Open source tokenization tools-an analysis,&quot; Advanced Computational Intelligence: An International Journal (ACII), vol. 3, no. 1, pp. 37-47, 2016, doi: 10.5121/acii.2016.3104.</mixed-citation>
                    </ref>
                                    <ref id="ref27">
                        <label>27</label>
                        <mixed-citation publication-type="journal">S. Alav and K.S. Benli, &quot;Turkish stance detection on social media using bert models: A case study of stray animals law,&quot; Sakarya University Journal of Computer and Information Sciences, vol. 8, no. 1, pp. 76-88, 2025, doi: 10.35377/saucis...1564138.</mixed-citation>
                    </ref>
                                    <ref id="ref28">
                        <label>28</label>
                        <mixed-citation publication-type="journal">J. Su, Q. Dai, F. Guerin, and M. Zhou, &quot;BERT-hLSTMs: BERT and hierarchical LSTMs for visual storytelling,&quot; Computer Speech &amp; Language, vol. 67, no. 101169, pp. 1-14, 2021, doi: 10.1016/j.csl.2020.101169.</mixed-citation>
                    </ref>
                                    <ref id="ref29">
                        <label>29</label>
                        <mixed-citation publication-type="journal">M. Bulut, &quot;Improving deep learning forecasting model based on LSTM for Türkiye’s hydro-electricity generation,&quot; Sakarya University Journal of Computer and Information Sciences, vol. 7, no. 3, pp. 325-337, 2024, doi: 10.35377/saucis...1503018.</mixed-citation>
                    </ref>
                                    <ref id="ref30">
                        <label>30</label>
                        <mixed-citation publication-type="journal">S. Varsamopoulos, K. Bertels, and C. G. Almudever, &quot;Designing neural network based decoders for surface codes,&quot; IEEE Transactions on Quantum Engineering, vol. 3, pp. 1-13, 2022, doi: 10.1109/TQE.2022.3195723.</mixed-citation>
                    </ref>
                                    <ref id="ref31">
                        <label>31</label>
                        <mixed-citation publication-type="journal">H. Göker and A. Said, &quot; Spectral analysis and Bi-LSTM deep network-based approach in detection of mild cognitive impairment from electroencephalography signals,&quot; Cognitive Neurodynamics, vol. 18, no.2, pp. 597-614, 2024, doi: 10.1007/s11571-023-10010-y.</mixed-citation>
                    </ref>
                                    <ref id="ref32">
                        <label>32</label>
                        <mixed-citation publication-type="journal">G. Liu and J. Guo, &quot;Bidirectional LSTM with attention mechanism and convolutional layer for text classification,&quot; Neurocomputing, vol. 337, pp. 325-338, 2019, doi: 10.1016/j.neucom.2019.01.078.</mixed-citation>
                    </ref>
                                    <ref id="ref33">
                        <label>33</label>
                        <mixed-citation publication-type="journal">S. Soni, S. S. Chouhan, and S. S. Rathore, &quot;TextConvoNet: A convolutional neural network based architecture for text classification,&quot; Applied Intelligence, vol. 53, no. 11, pp. 14249-14268, 2023, doi: 10.1007/s10489-022-04221-9.</mixed-citation>
                    </ref>
                                    <ref id="ref34">
                        <label>34</label>
                        <mixed-citation publication-type="journal">X. Guo,  Q. Liu, Y. Hu, and H. Liu, &quot;MDCNN: Multi-teacher distillation-based CNN for news text classification,&quot; IEEE Access, vol. 13, pp. 56631-56641, 2025, doi: 10.1109/ACCESS.2025.3555224.</mixed-citation>
                    </ref>
                                    <ref id="ref35">
                        <label>35</label>
                        <mixed-citation publication-type="journal">A. B. Cantor, &quot;Sample-size calculations for Cohen&#039;s kappa,&quot; Psychological Methods, vol. 1, no. 2, pp. 150-151, 1996, doi: 10.1037/1082-989X.1.2.150.</mixed-citation>
                    </ref>
                                    <ref id="ref36">
                        <label>36</label>
                        <mixed-citation publication-type="journal">M. Siering, A. V. Deokar, and C. Janze, &quot;Disentangling consumer recommendations: Explaining and predicting airline recommendations based on online reviews,&quot; Decision Support Systems, vol. 107, pp. 52-63, 2018, doi: 10.1016/j.dss.2018.01.002.</mixed-citation>
                    </ref>
                                    <ref id="ref37">
                        <label>37</label>
                        <mixed-citation publication-type="journal">H. Y. A. Shihabeldeen, &quot;Using text mining to predicate exchange rates with sentiment indicators,&quot; Journal of Business Theory and Practice, vol. 7, no. 2, pp. 60-75, 2019, doi: 10.22158/jbtp.v7n2p60.</mixed-citation>
                    </ref>
                                    <ref id="ref38">
                        <label>38</label>
                        <mixed-citation publication-type="journal">M. Afzaal, M. Usman, A. C. Fong, and S. Fong, &quot;Multiaspect‐based opinion classification model for tourist reviews,&quot; Expert Systems, vol. 36, no. 2, e12371, 2019, doi: 10.1111/exsy.12371.</mixed-citation>
                    </ref>
                                    <ref id="ref39">
                        <label>39</label>
                        <mixed-citation publication-type="journal">F. R. Lucini, L. M. Tonetto, F. S. Fogliatto, and M. J. Anzanello, &quot;Text mining approach to explore dimensions of airline customer satisfaction using online customer reviews,&quot; Journal of Air Transport Management, vol. 83, no. 101760, pp. 1-12, 2020, doi: 10.1016/j.jairtraman.2019.101760.</mixed-citation>
                    </ref>
                                    <ref id="ref40">
                        <label>40</label>
                        <mixed-citation publication-type="journal">C. F. Tsai, K. Chen, Y. H. Hu, and W. K. Chen, &quot;Improving text summarization of online hotel reviews with review helpfulness and sentiment,&quot; Tourism Management, vol. 80, no. 104122, pp. 1-13, 2020, doi: 10.1016/j.tourman.2020.104122.</mixed-citation>
                    </ref>
                                    <ref id="ref41">
                        <label>41</label>
                        <mixed-citation publication-type="journal">J. Guerreiro and P. Rita, &quot;How to predict explicit recommendations in online reviews using text mining and sentiment analysis,&quot; Journal of Hospitality and Tourism Management, vol. 43, pp. 269-272, 2020, doi: 10.1016/j.jhtm.2019.07.001.</mixed-citation>
                    </ref>
                                    <ref id="ref42">
                        <label>42</label>
                        <mixed-citation publication-type="journal">X. Zhou, &quot;Sentiment analysis of the consumer review text based on BERT-BiLSTM in a social media environment,&quot; International Journal of Information Technologies and Systems Approach (IJITSA), vol. 16, no. 2, pp. 1-16, 2023.</mixed-citation>
                    </ref>
                            </ref-list>
                    </back>
    </article>
