Investigating the Performance of Artificial Neural Networks in Predicting Affective Responses

Abstract

In this study it is aimed to examine the performance of an artificial neural network trained using items reflecting a latent trait in predicting responses to an item reflecting the same trait. This latent trait is the awareness of being able to communicate with people from different cultures, which is included in the PISA 2018 application. Relevant scale items were used as research variables. In addition to determining the extent to which the predicted responses overlap with the actual responses by analyzing the artificial neural network models, it was examined how the predicted responses affect the assumed latent construct and the reliability of the responses. Thus, the performance of artificial neural networks in predicting responses to affective items was evaluated.

Keywords

• Artifical neural networks
• machine learning
• affective responses
• prediction

References

1. Agresti, A. (2013). Categorical data analysis. Wiley.
2. Al-Saleem, M., Al-Kathiry, N., Al-Osimi, S., & Badr, G. (2015). Mining educational data to predict students’ academic performance. In Machine Learning and Data Mining in Pattern Recognition: 11th International Conference, MLDM 2015, Hamburg, Germany, July 20-21, 2015, Proceedings 11 (pp. 403-414). Springer International Publishing.
3. Ashwin, T. S., & Guddeti, R. M. R. (2020). Automatic detection of students’ affective states in classroom environment using hybrid convolutional neural networks. Education and Information Technologies, 25(2), 1387-1415. https://doi.org/10.1007/s10639-019-10004-6
4. Aybek, H. S. Y., & Okur, M. R. (2018). Predicting achievement with artificial neural networks: The case of Anadolu University open education system. International Journal of Assessment Tools in Education, 5(3), 474-490. https://doi.org/10.21449/ijate.435507
5. Aydoğan, İ., & Zırhlıoğlu, G. (2018). Öğrenci başarılarının yapay sinir ağları ile kestirilmesi. Van Yüzüncü Yıl Üniversitesi Eğitim Fakültesi Dergisi, 15(1), 577-610. http://dx.doi.org/10.23891/efdyyu.2018.80
6. Aydoğdu, Ş. (2020). Predicting student final performance using artificial neural networks in online learning environments. Education and Information Technologies, 25(3), 1913-1927. https://doi.org/10.1007/s10639-019-10053-x
7. Bakkialakshmi, V. S., Sudalaimuthu, T., & Winkler, S. (2022). Effective Prediction System for Affective Computing on Emotional Psychology with Artificial Neural Network. Easy Chair Preprint.
8. Beck, M.W. (2018). NeuralNetTools: Visualization and Analysis Tools for Neural Networks. Journal of Statistical Software, 85(11), 1 .https://doi.org/10.18637/jss.v085.i11

9. Beck, M.W. (2022). Visualization and analysis tools for neural networks, R package version 1.5.3. Retrieved from https://cran.r-project.org/web/packages/NeuralNetTools/index.html
10. Brownlee, J. (2020). Data preparation for machine learning: data cleaning, feature selection, and data transforms in Python. Machine Learning Mastery.
11. Carstensen, S. L., Madsen, J., & Larsen, J. (2016). Predicting Changes in Affective States using Neural Networks. arXiv preprint arXiv:1612.00582. https://doi.org/10.48550/arXiv.1612.00582
12. Chan, K. Y., Kwong, C. K., Wongthongtham, P., Jiang, H., Fung, C. K., Abu-Salih, B., ... & Jain, P. (2020). Affective design using machine learning: a survey and its prospect of conjoining big data. International Journal of Computer Integrated Manufacturing, 33(7), 645-669. https://doi.org/10.1080/0951192X.2018.1526412
13. Chavez, H., Chavez-Arias, B., Contreras-Rosas, S., Alvarez-Rodríguez, J. M., & Raymundo, C. (2023, February). Artificial neural network model to predict student performance using nonpersonal information. In Frontiers in Education (Vol. 8, p. 1106679). Frontiers Media SA.
14. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37-46. https://doi.org/10.1177/001316446002000104
15. Ennett, C. M., Frize, M., & Walker, C. R. (2001). Influence of missing values on artificial neural network performance. In MEDINFO 2001 (pp. 449-453). Ios Press.
16. Feng, H. (2022). A Novel Adaptive Affective Cognition Analysis Model for College Students Using a Deep Convolution Neural Network and Deep Features. Computational Intelligence and Neuroscience, 2022(1), 2114114. https://doi.org/10.1155/2022/2114114
17. Flitman, A. M. (1997). Towards analysing student failures: neural networks compared with regression analysis and multiple discriminant analysis. Computers & Operations Research, 24(4), 367-377. https://doi.org/10.1016/S0305-0548(96)00060-3
18. Fritsch, S., Guenther, F., Wright, M.N., Suling, M., Mueller, S.M. (2019). Training of neural Networks, R package version 1.44.2. Retrieved from https://cran.r-project.org/web/packages/neuralnet/index.html
19. George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference. 11.0 update (4th ed.). Allyn & Bacon.
20. Goel, A., Goel, A. K., & Kumar, A. (2023). The role of artificial neural network and machine learning in utilizing spatial information. Spatial Information Research, 31(3), 275-285. https://doi.org/10.1007/s41324-022-00494-x
21. Grosan, C., & Abraham, A. (2011). Artificial neural networks. Intelligent Systems: A Modern Approach, 281-323.
22. Guarín, C. E. L., Guzmán, E. L., & González, F. A. (2015). A model to predict low academic performance at a specific enrollment using data mining. IEEE Revista Iberoamericana de tecnologias del Aprendizaje, 10(3), 119-125. https://doi.org/10.1109/RITA.2015.2452632
23. Hasson, U., Nastase, S. A., & Goldstein, A. (2020). Direct fit to nature: an evolutionary perspective on biological and artificial neural networks. Neuron, 105(3), 416-434. https://doi.org/10.1016/j.neuron.2019.12.002
24. Hox, J., Moerbeek, M., & van de Schoot, R. (2018). Multilevel analysis: Techniques and applications (3rd ed.). Routledge.
25. Huang, S., & Fang, N. (2012, October). Work in progress: Early prediction of students' academic performance in an introductory engineering course through different mathematical modeling techniques. In 2012 Frontiers in Education Conference Proceedings (pp. 1-2). IEEE.
26. Jadhav, N., & Sugandhi, R. (2018, November). Survey on human behavior recognition using affective computing. In 2018 IEEE Global Conference on Wireless Computing and Networking (GCWCN) (pp. 98-103). IEEE.
27. Jamisola, R. S. (2016). Conceptualizing a Questionnaire-Based Machine Learning Tool that Determines State of Mind and Emotion. Lovotics, 4(115), 2. http://dx.doi.org/10.4172/2090-9888.1000115
28. Kardan, A. A., Sadeghi, H., Ghidary, S. S., & Sani, M. R. F. (2013). Prediction of student course selection in online higher education institutes using neural network. Computers & Education, 65, 1-11. https://doi.org/10.1016/j.compedu.2013.01.015
29. Kose, U., & Arslan, A. (2017). Optimization of self‐learning in Computer Engineering courses: An intelligent software system supported by Artificial Neural Network and Vortex Optimization Algorithm. Computer Applications in Engineering Education, 25(1), 142-156. https://doi.org/10.1002/cae.21787
30. Kuhn, M., Wing, J., Weston, S., Williams, A., Keefer, C., Engelhardt, A., … & Hunt, T. (2023). Classification and regression training, R package version 6.0-94. Retrieved from https://cran.r-project.org/web/packages/caret/index.html
31. Landis, J, R., & Koch, G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159-174.https://doi.org/10.2307/2529310
32. Lau, E. T., Sun, L., & Yang, Q. (2019). Modelling, prediction and classification of student academic performance using artificial neural networks. SN Applied Sciences, 1(9), 982. https://doi.org/10.1007/s42452-019-0884-7
33. Lek, S., Delacoste, M., Baran, P., Dimopoulos, I., Lauga, J., & Aulagnier, S. (1996). Application of neural networks to modeling nonlinear relationships in ecology. Ecological Modelling, 90, 39–52. https://doi.org/10.1016/0304-3800(95)00142-5
34. Lillicrap, T. P., Santoro, A., Marris, L., Akerman, C. J., & Hinton, G. (2020). Backpropagation and the brain. Nature Reviews Neuroscience, 21(6), 335-346. https://doi.org/10.1038/s41583-020-0277-3
35. Liu, J., Ang, M. C., Chaw, J. K., Kor, A. L., & Ng, K. W. (2023). Emotion assessment and application in human–computer interaction interface based on backpropagation neural network and artificial bee colony algorithm. Expert Systems with Applications, 232, 120857. https://doi.org/10.1016/j.eswa.2023.120857
36. McCarthy, N. (June19, 2019). Lebanon has by far the most refugees per 1,000 population. Retrieved from https://www.statista.com/chart/8800/lebanon-has-by-far-the-most-refugees-per-capita/
37. OECD. (2018). PISA 2018 Technical report. OECD Publishing, Retrieved from https://www.oecd.org/pisa/data/pisa2018technicalreport/
38. Orozco-del-Castillo, M. G., Orozco-del-Castillo, E. C., Brito-Borges, E., Bermejo-Sabbagh, C., & Cuevas-Cuevas, N. (2021, November). An artificial neural network for depression screening and questionnaire refinement in undergraduate students. In International Congress of Telematics and Computing (pp. 1-13). Springer International Publishing.
39. Öztemel, E. (2003).Yapay sinir ağları. PapatyaYayıncılık.
40. Park, C. W., Seo, S. W., Kang, N., Ko, B., Choi, B. W., Park, C. M., ... & Yoon, H. J. (2020). Artificial intelligence in health care: Current applications and issues. Journal of Korean Medical Science, 35(42). https://doi.org/10.3346/jkms.2020.35.e379
41. Rashid, T. A., & Ahmad, H. A. (2016). Lecturer performance system using neural network with Particle Swarm Optimization. Computer Applications in Engineering Education, 24(4), 629-638. https://doi.org/10.1002/cae.21737
42. Ripley, B., & Venables, W. (2023). Feed-forward neural networks and multinomial log-linear models, R package version 7.3-18. Retrieved from https://cran.r-project.org/web/packages/nnet/index.html
43. Rodríguez-Hernández, C. F., Musso, M., Kyndt, E., & Cascallar, E. (2021). Artificial neural networks in academic performance prediction: Systematic implementation and predictor evaluation. Computers and Education: Artificial Intelligence, 2, 100018. https://doi.org/10.1016/j.caeai.2021.100018
44. Rosseel, Y., Oberski, D., Byrnes, J., Vanbrabant, L., Savalei, V., Merkle, E., … & Jorgensen, T. (2024). Latent variable analysis, R package version 0.6-18. Retrieved from https://cran.r-project.org/web/packages/lavaan/lavaan.pdf
45. Shahiri, A. M., & Husain, W. (2015). A review on predicting student's performance using data mining techniques. Procedia Computer Science, 72, 414-422. https://doi.org/10.1016/j.procs.2015.12.157
46. Şen, S. (2020). Mplus ile yapısal eşitlik modellemesi uygulamaları. Nobel Akademik Yayıncılık.
47. Tabachnick, B. G., & Fidell, L. (2019). Using multivariate statistics (7th ed.). Pearson.
48. Tu, J. V. (1996). Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. Journal of Clinical Epidemiology, 49(11), 1225-1231. https://doi.org/10.1016/S0895-4356(96)00002-9
49. Umar, M. A. (2019). Student academic performance prediction using artificial neural networks: A case study. International Journal of Computer Applications, 178(48), 24-29. http://dx.doi.org/10.5120/ijca2019919387
50. Vandamme, J. P., Meskens, N., & Superby, J. F. (2007). Predicting academic performance by data mining methods. Education Economics, 15(4), 405. https://doi.org/10.1080/09645290701409939
51. Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., ... & Zhang, W. (2022). A systematic review on affective computing: Emotion models, databases, and recent advances. Information Fusion, 83, 19-52. https://doi.org/10.1016/j.inffus.2022.03.009
52. Zacharis, N. Z. (2016). Predicting student academic performance in blended learning using artificial neural networks. International Journal of Artificial Intelligence and Applications, 7(5), 17-29. http://dx.doi.org/10.5121/ijaia.2016.7502
53. Zou, J., Han, Y., & So, S. S. (2009). Overview of artificial neural networks. Artificial neural networks: methods and applications, 14-22. https://doi.org/10.1007/978-1-60327-101-1_2