Research Article
PDF EndNote BibTex Cite

Bir Öğrenci Bilgi Sisteminin Kullanılabilirliğinin Makine Öğrenmesi Teknikleriyle Tahmin Edilmesi

Year 2019, Volume 2, Issue 1, 10 - 18, 13.07.2019

Abstract

Sistem kullanılabilirliği, bir sistemin özellikle tasarım ve test aşamalarında odaklanılması gereken unsurlardan biridir, çünkü sistemin daha iyi hale getirilmesi için sistem yöneticilerine geri bildirim sağlamaktadır. Literatürde, sistem kullanılabilirliğinin değerlendirilmesi için Sistem Kullanılabilirlik Ölçeği (System Usability Scale-SUS) altın standart yöntem olarak yaygın şekilde kullanılmaktadır. Bunun yanı sıra günümüzde yapay zekânın alt çalışma alanlarından biri olan makine öğrenmesi de sistem kullanılabilirliğinin değerlendirilmesi konusunda araştırmacılara yeni ufuklar sağlamaktadır. Bu çalışmada, bir Öğrenci Bilgi Sisteminin (ÖBS) kullanılabilirliğinin makine öğrenmesi teknikleriyle tahmin edilmesi hedeflenmiştir. Çalışma yönteminde; Veri Madenciliği için Çapraz Endüstri Standard Süreç Modeli (CRoss-Industry Standard Process for Data Mining–CRISP-DM) kullanılmıştır. Analizler; Türkiye’deki bir vakıf üniversitesine ait bir ÖBS’yi kullanan 324 öğrencinin SUS’un Türkçe versiyonuna (SUS-TR) verdiği yanıtların bulunduğu “sus1” adlı veri seti ile “sus1” veri setine öğrencilerin yaş, cinsiyet, öğrenim gördüğü bölüm eklenerek oluşturulan “sus0” adlı veri setleri üzerinde gerçekleştirilmiştir. C4.5 Karar Ağacı Algoritması, Naive Bayes Sınıflandırıcı ve k-En Yakın Komşu Algoritması ile farklı modeller kurularak performans değerlendirmesi yapılmıştır. %80’e %20’lik Hold-out ayrımıyla gerçekleştirilen analizlerde en iyi performans, k-En Yakın Komşu Algoritmasıyla “sus0” veri seti üzerinde elde edilmiştir (k=20 için doğruluk = 0.698, F-ölçütü = 0.796).

References

  • [1] ISO 9241-11, “Ergonomic requirements for office work with visual display terminals (VDTs) -- Part 11: Guidance on usability”. International Organization for Standartization, 1998.
  • [2] Jordan PW. An Introduction to Usability. Padstow, UK, Taylor & Francis Ltd., 1998.
  • [3] Nielsen J. “Finding usability problems through heuristic evaluation”. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, USA, 1992, pp. 373–380.
  • [4] Nielsen J. “Usability 101: Introduction to Usability”. 2012. https://www.nngroup.com/articles/usability-101-introduction-to-usability/ adresinden 3 Ekim 2018 tarihinde erişildi.
  • [5] Hornbæk K. “Current practice in measuring usability: Challenges to usability studies and research”. International Journal of Human-Computer Studies, 64(2), 79–102, 2006.
  • [6] Sweeney M, Maguire M, and Shackel B. “Evaluating user-computer interaction: a framework”. Int J Man Mach Stud, 38(4), 689–711, 1993.
  • [7] Rubin J, Chisnell D. "Handbook of usability testing: how to plan, design, and conduct effective tests". 2nd ed. Indianapolis, USA, Wiley Publishing Inc., 2008.
  • [8] Kieras D. “Model Based Evaluation”. In The Human–Computer Interaction Handbook Fundamentals, Evolving Technologies, and Emerging Applications, 2nd ed., Sears A, Jacko JA. Eds. Boca Raton, USA, CRC Press, 2007.
  • [9] Mukerjee S. “Student information systems – implementation challenges and the road ahead”. Journal of Higher Education Policy and Management, 34(1), 51–60, 2012.
  • [10] Carcary M, Long G, Remenyi D. “The Implementation of a New Student Management Information System (MIS) at an Irish Institute of Technology -- An Ex Post Evaluation of its Success”. Electronic Journal of Information Systems Evaluation, 10(1), 14, 2007.
  • [11] Karat CM. “A Business Case Approach to Usability Cost Justification for the Web”. In Cost-justifying usability: an update for an Internet age, 2nd ed., Bias RG, Mayhew DJ. Eds. San Francisco, USA, Morgan Kaufman Publishers, 2005.
  • [12] Brooke J. “SUS - A quick and dirty usability scale”. Digital Equipment Corporation, 7, 1986.
  • [13] Demirkol D. “Evaluation of student information system (SIS) in terms of user emotions, performance, and perceived usability: A Turkish university case”. Yüksek Lisans Tezi, Yeditepe Üniversitesi, İstanbul, 2018.
  • [14] Korvald C, Kim E, Reza H. “Evaluation and implementation of machine learning techniques in usability testing for web sites”. In Proceedings of the 47th Annual Midwest Instruction and Computing Symposium, Verona, WI, 1–11, 2014.
  • [15] Oztekin A. “A decision support system for usability evaluation of web-based information systems”. Expert Syst Appl, 38(3), 2110–2118, 2011.
  • [16] Oztekin A, Delen D, Turkyilmaz A, Zaim S. “A machine learning-based usability evaluation method for eLearning systems”. Decis Support Syst, 56, 63–73, 2013.
  • [17] Davis PA. “Learning usability assessment models for web sites”. Doctoral Thesis, Texas A&M University, USA, 2010.
  • [18] Horng D, Kittur A, Hong JI, Faloutsos C. “Making Sense of Large Network Data: Combining Rich User Interaction and Machine Learning”. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, BC, Canada, 167–176, 2011.
  • [19] Maragoudakis M, Tselios NK, Fakotakis N, Avouris NM. “Improving SMS Usability Using Bayesian Networks”. In Methods and Applications of Artificial Intelligence, 2308, Vlahavas IP, Spyropoulos CD. Eds. Berlin, Springer, 179–190, 2002.
  • [20] Shearer C. “The CRISP-DM model: the new blueprint for data mining”. Journal of data warehousing, 5(4), 13–22, 2000.
  • [21] Balaban ME, Kartal E. Veri Madenciliği ve Makine Öğrenmesi Temel Algoritmaları ve R Dili ile Uygulamaları. 2nd ed. Beyoğlu, İstanbul, Çağlayan Kitabevi, 2018.
  • [22] Brooke J. “SUS - A quick and dirty usability scale”. In Usability Evaluation In Industry, Jordan PW, Thomas B, McClelland IL, Weerdmeester B. Eds. London, UK, Taylor & Francis Ltd., 189–194, 1996.
  • [23] Bangor A, Kortum P, Miller J. “Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale”. Journal of Usability Studies, 4(3), 114–123, 2009.
  • [24] Selçukcan Erol Ç. “Sağlık Bilimlerinde R ile Veri Madenciliği”. In R ile Veri Madenciliği Uygulamaları. 1st ed., Balaban ME, Kartal E. Eds. İstanbul, Çağlayan Kitabevi, 2016.
  • [25] Quinlan JR, “Constructing Decision Trees”. In C4.5 Programs for Machine Learning, California, USA, Morgan Kaufman Publishers, 17–26, 1993.
  • [26] Lim TS, Loh WY, Shih YS. “A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms”. Machine Learning, 40(3), 203–228, 2000.
  • [27] Ruggieri S. “Efficient C4.5”. IEEE Transactions on Knowledge and Data Engineering, 14(2), 438–444, 2002.
  • [28] McCallum A, Nigam K. “A Comparison of Event Models for Naive Bayes Text Classification”. AAAI-98 Workshop on Learning for Text Categorization, 752, 41–48, 1998.
  • [29] Zhang H. “The optimality of naive Bayes”. In Proceedings of 17th International Florida Artificial Intelligence Research Society Conference, Florida, USA, 562–567, 2004.
  • [30] Gou J, Xiong T, Kuang Y. “A Novel Weighted Voting for K-Nearest Neighbor Rule”. Journal of Computers, 6(5), 833–840, 2011.
  • [31] The R Foundation. “R: The R Project for Statistical Computing”. 2018. https://www.r-project.org/ adresinden 14 Kasım 2018 tarihinde erişildi.
  • [32] RStudio. “RStudio - Open source and enterprise-ready professional software for R”. RStudio, 2014. https://www.rstudio.com/ adresinden 14 Kasım 2018 tarihinde erişildi.
  • [33] Kuhn M. "caret: Classification and Regression Training". 2018.
  • [34] Acuna E, The CASTLE research group. "dprep: Data Pre-Processing and Visualization Functions for Classification". 2015.
  • [35] Meyer D, Dimitriadou E, Hornik K, Weingessel A, Leisch F. "e1071: Misc Functions of the Department of Statistics", Probability Theory Group (Formerly: E1071), TU Wien. 2017.
  • [36] Hornik K, Buchta C, Zeileis A. "Open-Source Machine Learning: R Meets Weka". Computational Statistics, 24(2), 225–232, 2009.
  • [37] Witten IH, Frank E. "Data Mining: Practical Machine Learning Tools and Techniques", 2nd ed. San Francisco, USA, Morgan Kaufmann, 2005.
  • [38] Dragulescu AA, Arendt C. "xlsx: Read, Write, Format Excel 2007 and Excel 97/2000/XP/2003 Files". 2018.
  • [39] Harper BD, Norman KL. “Improving user satisfaction: The questionnaire for user interaction satisfaction version 5.5”. In Proceedings of the 1st Annual Mid-Atlantic Human Factors Conference, 224–228, 1993.
  • [40] Lewis JR. “IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use”. Int J Hum-Comput Int, 7(1), 57–78, 1995.

Predicting Usability of a Student Information System by Using Machine Learning Techniques

Year 2019, Volume 2, Issue 1, 10 - 18, 13.07.2019

Abstract

System usability is one of the key elements that should be focused on, especially during the design and test phases of a system, because it provides feedback to system administrators in order to improve the system. In the literature, System Usability Scale (SUS) is widely used as the gold standard method to evaluate system usability. Today, machine learning, which is one of the subfields of artificial intelligence, also provide new perspectives on the evaluation of system usability. In this study, it is aimed to predict usability of a Student Information System (SIS) by using machine learning techniques. In the study method, the Cross-Industry Standard Process for Data Mining (CRISP-DM) steps have been followed. Analysis are performed on two different dataset namely “sus1” and “sus0”. “sus1” dataset is consisted of demographic characteristics (age, gender, department) of 324 students using a SIS of a foundation university in Turkey, also their responses to the Turkish version of the SUS. “sus0” includes only responses to the Turkish version of the SUS. C4.5 Decision Tree Algorithm, Naive Bayes Classifier and k-Nearest Neighbor Algorithm are used to create models and their performance are evaluated. The best performance was obtained on the “sus0” data set with 80% to 20% hold-out method (accuracy = 0.698, F-measure = 0.796 for k = 20).

References

  • [1] ISO 9241-11, “Ergonomic requirements for office work with visual display terminals (VDTs) -- Part 11: Guidance on usability”. International Organization for Standartization, 1998.
  • [2] Jordan PW. An Introduction to Usability. Padstow, UK, Taylor & Francis Ltd., 1998.
  • [3] Nielsen J. “Finding usability problems through heuristic evaluation”. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, USA, 1992, pp. 373–380.
  • [4] Nielsen J. “Usability 101: Introduction to Usability”. 2012. https://www.nngroup.com/articles/usability-101-introduction-to-usability/ adresinden 3 Ekim 2018 tarihinde erişildi.
  • [5] Hornbæk K. “Current practice in measuring usability: Challenges to usability studies and research”. International Journal of Human-Computer Studies, 64(2), 79–102, 2006.
  • [6] Sweeney M, Maguire M, and Shackel B. “Evaluating user-computer interaction: a framework”. Int J Man Mach Stud, 38(4), 689–711, 1993.
  • [7] Rubin J, Chisnell D. "Handbook of usability testing: how to plan, design, and conduct effective tests". 2nd ed. Indianapolis, USA, Wiley Publishing Inc., 2008.
  • [8] Kieras D. “Model Based Evaluation”. In The Human–Computer Interaction Handbook Fundamentals, Evolving Technologies, and Emerging Applications, 2nd ed., Sears A, Jacko JA. Eds. Boca Raton, USA, CRC Press, 2007.
  • [9] Mukerjee S. “Student information systems – implementation challenges and the road ahead”. Journal of Higher Education Policy and Management, 34(1), 51–60, 2012.
  • [10] Carcary M, Long G, Remenyi D. “The Implementation of a New Student Management Information System (MIS) at an Irish Institute of Technology -- An Ex Post Evaluation of its Success”. Electronic Journal of Information Systems Evaluation, 10(1), 14, 2007.
  • [11] Karat CM. “A Business Case Approach to Usability Cost Justification for the Web”. In Cost-justifying usability: an update for an Internet age, 2nd ed., Bias RG, Mayhew DJ. Eds. San Francisco, USA, Morgan Kaufman Publishers, 2005.
  • [12] Brooke J. “SUS - A quick and dirty usability scale”. Digital Equipment Corporation, 7, 1986.
  • [13] Demirkol D. “Evaluation of student information system (SIS) in terms of user emotions, performance, and perceived usability: A Turkish university case”. Yüksek Lisans Tezi, Yeditepe Üniversitesi, İstanbul, 2018.
  • [14] Korvald C, Kim E, Reza H. “Evaluation and implementation of machine learning techniques in usability testing for web sites”. In Proceedings of the 47th Annual Midwest Instruction and Computing Symposium, Verona, WI, 1–11, 2014.
  • [15] Oztekin A. “A decision support system for usability evaluation of web-based information systems”. Expert Syst Appl, 38(3), 2110–2118, 2011.
  • [16] Oztekin A, Delen D, Turkyilmaz A, Zaim S. “A machine learning-based usability evaluation method for eLearning systems”. Decis Support Syst, 56, 63–73, 2013.
  • [17] Davis PA. “Learning usability assessment models for web sites”. Doctoral Thesis, Texas A&M University, USA, 2010.
  • [18] Horng D, Kittur A, Hong JI, Faloutsos C. “Making Sense of Large Network Data: Combining Rich User Interaction and Machine Learning”. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, BC, Canada, 167–176, 2011.
  • [19] Maragoudakis M, Tselios NK, Fakotakis N, Avouris NM. “Improving SMS Usability Using Bayesian Networks”. In Methods and Applications of Artificial Intelligence, 2308, Vlahavas IP, Spyropoulos CD. Eds. Berlin, Springer, 179–190, 2002.
  • [20] Shearer C. “The CRISP-DM model: the new blueprint for data mining”. Journal of data warehousing, 5(4), 13–22, 2000.
  • [21] Balaban ME, Kartal E. Veri Madenciliği ve Makine Öğrenmesi Temel Algoritmaları ve R Dili ile Uygulamaları. 2nd ed. Beyoğlu, İstanbul, Çağlayan Kitabevi, 2018.
  • [22] Brooke J. “SUS - A quick and dirty usability scale”. In Usability Evaluation In Industry, Jordan PW, Thomas B, McClelland IL, Weerdmeester B. Eds. London, UK, Taylor & Francis Ltd., 189–194, 1996.
  • [23] Bangor A, Kortum P, Miller J. “Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale”. Journal of Usability Studies, 4(3), 114–123, 2009.
  • [24] Selçukcan Erol Ç. “Sağlık Bilimlerinde R ile Veri Madenciliği”. In R ile Veri Madenciliği Uygulamaları. 1st ed., Balaban ME, Kartal E. Eds. İstanbul, Çağlayan Kitabevi, 2016.
  • [25] Quinlan JR, “Constructing Decision Trees”. In C4.5 Programs for Machine Learning, California, USA, Morgan Kaufman Publishers, 17–26, 1993.
  • [26] Lim TS, Loh WY, Shih YS. “A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms”. Machine Learning, 40(3), 203–228, 2000.
  • [27] Ruggieri S. “Efficient C4.5”. IEEE Transactions on Knowledge and Data Engineering, 14(2), 438–444, 2002.
  • [28] McCallum A, Nigam K. “A Comparison of Event Models for Naive Bayes Text Classification”. AAAI-98 Workshop on Learning for Text Categorization, 752, 41–48, 1998.
  • [29] Zhang H. “The optimality of naive Bayes”. In Proceedings of 17th International Florida Artificial Intelligence Research Society Conference, Florida, USA, 562–567, 2004.
  • [30] Gou J, Xiong T, Kuang Y. “A Novel Weighted Voting for K-Nearest Neighbor Rule”. Journal of Computers, 6(5), 833–840, 2011.
  • [31] The R Foundation. “R: The R Project for Statistical Computing”. 2018. https://www.r-project.org/ adresinden 14 Kasım 2018 tarihinde erişildi.
  • [32] RStudio. “RStudio - Open source and enterprise-ready professional software for R”. RStudio, 2014. https://www.rstudio.com/ adresinden 14 Kasım 2018 tarihinde erişildi.
  • [33] Kuhn M. "caret: Classification and Regression Training". 2018.
  • [34] Acuna E, The CASTLE research group. "dprep: Data Pre-Processing and Visualization Functions for Classification". 2015.
  • [35] Meyer D, Dimitriadou E, Hornik K, Weingessel A, Leisch F. "e1071: Misc Functions of the Department of Statistics", Probability Theory Group (Formerly: E1071), TU Wien. 2017.
  • [36] Hornik K, Buchta C, Zeileis A. "Open-Source Machine Learning: R Meets Weka". Computational Statistics, 24(2), 225–232, 2009.
  • [37] Witten IH, Frank E. "Data Mining: Practical Machine Learning Tools and Techniques", 2nd ed. San Francisco, USA, Morgan Kaufmann, 2005.
  • [38] Dragulescu AA, Arendt C. "xlsx: Read, Write, Format Excel 2007 and Excel 97/2000/XP/2003 Files". 2018.
  • [39] Harper BD, Norman KL. “Improving user satisfaction: The questionnaire for user interaction satisfaction version 5.5”. In Proceedings of the 1st Annual Mid-Atlantic Human Factors Conference, 224–228, 1993.
  • [40] Lewis JR. “IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use”. Int J Hum-Comput Int, 7(1), 57–78, 1995.

Details

Primary Language Turkish
Subjects Engineering
Journal Section Articles
Authors

Denizhan DEMİRKOL> (Primary Author)
SÖKE İŞLETME FAKÜLTESİ
0000-0001-7343-9610
Türkiye


Elif KARTAL>
İSTANBUL ÜNİVERSİTESİ, REKTÖRLÜK, ENFORMATİK BÖLÜMÜ
0000-0003-4667-1806
Türkiye


Çağla ŞENELER This is me
YEDİTEPE ÜNİVERSİTESİ, TİCARİ BİLİMLER FAKÜLTESİ, YÖNETİM BİLİŞİM SİSTEMLERİ BÖLÜMÜ
Türkiye


Sevinç GÜLSEÇEN>
İSTANBUL ÜNİVERSİTESİ, REKTÖRLÜK, ENFORMATİK BÖLÜMÜ
0000-0001-8537-7111
Türkiye

Publication Date July 13, 2019
Published in Issue Year 2019, Volume 2, Issue 1

Cite

Bibtex @research article { veri489216, journal = {Veri Bilimi}, eissn = {2667-582X}, address = {}, publisher = {Murat GÖK}, year = {2019}, volume = {2}, number = {1}, pages = {10 - 18}, title = {Bir Öğrenci Bilgi Sisteminin Kullanılabilirliğinin Makine Öğrenmesi Teknikleriyle Tahmin Edilmesi}, key = {cite}, author = {Demirkol, Denizhan and Kartal, Elif and Şeneler, Çağla and Gülseçen, Sevinç} }
APA Demirkol, D. , Kartal, E. , Şeneler, Ç. & Gülseçen, S. (2019). Bir Öğrenci Bilgi Sisteminin Kullanılabilirliğinin Makine Öğrenmesi Teknikleriyle Tahmin Edilmesi . Veri Bilimi , 2 (1) , 10-18 . Retrieved from https://dergipark.org.tr/en/pub/veri/issue/47049/489216
MLA Demirkol, D. , Kartal, E. , Şeneler, Ç. , Gülseçen, S. "Bir Öğrenci Bilgi Sisteminin Kullanılabilirliğinin Makine Öğrenmesi Teknikleriyle Tahmin Edilmesi" . Veri Bilimi 2 (2019 ): 10-18 <https://dergipark.org.tr/en/pub/veri/issue/47049/489216>
Chicago Demirkol, D. , Kartal, E. , Şeneler, Ç. , Gülseçen, S. "Bir Öğrenci Bilgi Sisteminin Kullanılabilirliğinin Makine Öğrenmesi Teknikleriyle Tahmin Edilmesi". Veri Bilimi 2 (2019 ): 10-18
RIS TY - JOUR T1 - Bir Öğrenci Bilgi Sisteminin Kullanılabilirliğinin Makine Öğrenmesi Teknikleriyle Tahmin Edilmesi AU - Denizhan Demirkol , Elif Kartal , Çağla Şeneler , Sevinç Gülseçen Y1 - 2019 PY - 2019 N1 - DO - T2 - Veri Bilimi JF - Journal JO - JOR SP - 10 EP - 18 VL - 2 IS - 1 SN - -2667-582X M3 - UR - Y2 - 2019 ER -
EndNote %0 Veri Bilimi Bir Öğrenci Bilgi Sisteminin Kullanılabilirliğinin Makine Öğrenmesi Teknikleriyle Tahmin Edilmesi %A Denizhan Demirkol , Elif Kartal , Çağla Şeneler , Sevinç Gülseçen %T Bir Öğrenci Bilgi Sisteminin Kullanılabilirliğinin Makine Öğrenmesi Teknikleriyle Tahmin Edilmesi %D 2019 %J Veri Bilimi %P -2667-582X %V 2 %N 1 %R %U
ISNAD Demirkol, Denizhan , Kartal, Elif , Şeneler, Çağla , Gülseçen, Sevinç . "Bir Öğrenci Bilgi Sisteminin Kullanılabilirliğinin Makine Öğrenmesi Teknikleriyle Tahmin Edilmesi". Veri Bilimi 2 / 1 (July 2019): 10-18 .
AMA Demirkol D. , Kartal E. , Şeneler Ç. , Gülseçen S. Bir Öğrenci Bilgi Sisteminin Kullanılabilirliğinin Makine Öğrenmesi Teknikleriyle Tahmin Edilmesi. Veri Bilim Derg. 2019; 2(1): 10-18.
Vancouver Demirkol D. , Kartal E. , Şeneler Ç. , Gülseçen S. Bir Öğrenci Bilgi Sisteminin Kullanılabilirliğinin Makine Öğrenmesi Teknikleriyle Tahmin Edilmesi. Veri Bilimi. 2019; 2(1): 10-18.
IEEE D. Demirkol , E. Kartal , Ç. Şeneler and S. Gülseçen , "Bir Öğrenci Bilgi Sisteminin Kullanılabilirliğinin Makine Öğrenmesi Teknikleriyle Tahmin Edilmesi", Veri Bilimi, vol. 2, no. 1, pp. 10-18, Jul. 2019



Dergimizin Tarandığı Dizinler (İndeksler)


Academic Resource Index

logo.png

journalseeker.researchbib.com

Google Scholar

scholar_logo_64dp.png

ASOS Index

asos-index.png

Rooting Index

logo.png

www.rootindexing.com

The JournalTOCs Index

journal-tocs-logo.jpg?w=584

www.journaltocs.ac.uk

General Impact Factor (GIF) Index

images?q=tbn%3AANd9GcQ0CrEQm4bHBnwh4XJv9I3ZCdHgQarj_qLyPTkGpeoRRmNh10eC

generalif.com

Directory of Research Journals Indexing

DRJI_Logo.jpg

olddrji.lbp.world/indexedJournals.aspx

I2OR Index

8c492a0a466f9b2cd59ec89595639a5c?AccessKeyId=245B99561176BAE11FEB&disposition=0&alloworigin=1

http://www.i2or.com/8.html



logo.png