Performance Evaluation Using the Discrete Choquet Integral: Higher Education Sector

Abstract

Performance evaluation functions as an essential tool for decision makers in the field of measuring and assessing the performance under the multiple evaluation criteria aspect of the systems such as management, economy, and education system. Besides, academic performance evaluation is one of the critical issues in higher institution of learning. Even though the academic evaluation criteria are inherently dependent, most of the traditional evaluation methods take no account of the dependency. Currently, the discrete Choquet integral can be proposed as a useful and effective aggregation operator due to being capable of considering the interactions among the evaluation criteria. In this paper, it is aimed to solve an academic performance evaluation problem of students in a university in Turkey using the discrete Choquet integral with the complexity-based method and the entropy-based method. Moreover, the k-means method, which has been widely used for evaluating students’ performance over 50 years, is used to compare the effectiveness and the success of two different frameworks based on discrete Choquet integral in the robustness check. Our results indicate that the entropy-based Choquet integral outperforms the complexity-based Choquet and k-means method in most of the cases.

Keywords

• Performance evaluation
• fuzzy measure
• discrete Choquet integral
• k-means

Performance Evaluation Using the Discrete Choquet Integral: Higher Education Sector

Öz

Performance evaluation functions as an essential tool for decision makers in the field of measuring and assessing the performance under the multiple evaluation criteria aspect of the systems such as management, economy, and education system. Besides, academic performance evaluation is one of the critical issues in higher institution of learning. Even though the academic evaluation criteria are inherently dependent, most of the traditional evaluation methods take no account of the dependency. Currently, the discrete Choquet integral can be proposed as a useful and effective aggregation operator due to being capable of considering the interactions among the evaluation criteria. In this paper, it is aimed to solve an academic performance evaluation problem of students in a university in Turkey using the discrete Choquet integral with the complexity-based method and the entropy-based method. Moreover, the k-means method, which has been widely used for evaluating students’ performance over 50 years, is used to compare the effectiveness and the success of two different frameworks based on discrete Choquet integral in the robustness check. Our results indicate that the entropy-based Choquet integral outperforms the complexity-based Choquet and k-means method in most of the cases.

Anahtar Kelimeler

• Performance evaluation
• Fuzzy measure
• Discrete Choquet integral
• k-means

References

1. Angilella, S., Arcidiacono, S.G., Corrente, S,, Greco, S., & Matarazzo, B. (2017). An application of the SMAA–Choquet method to evaluate the performance of sailboats in offshore regattas. Operational Research, doi:10.1007/s12351-017-0340-7.
2. Angilella, S., Corrente, S., & Greco, S. (2015). Stochastic multiobjective acceptability analysis for the Choquet integral preference model and the scale construction problem. European Journal of Operational Research, 240(1), 172-182, doi: 10.1016/j.ejor.2014.06.031.
3. Baker, R.S.J.D., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3-17.
4. Branke, J., Corrente, S., Greco, S., Słowiński, R., & Zielniewicz, P. (2016). Using Choquet integral as preference model in interactive evolutionary multiobjective optimization. European Journal of Operational Research, 250(3), 884-901, doi: 10.1016/j.ejor.2015.10.027.
5. Calvo, T., Mayor, G., & Mesiar, R. (2002). Aggregation operators: New trends and applications. Physica-Verlag, Heidelberg, New York.
6. Catlett, J. (1991). On changing continuous attributes into ordered discrete attributes. Kodratoff, Y. (Eds) Machine Learning Lecture Notes in Computer Science, Springer, Berlin, Heidelberg.
7. Chang, H.J., Liu, H.C., Tseng, S.W., & Chang, F.M. (2009). A comparison on Choquet integral with different information-based fuzzy measures. In Proceedings of the 8th International Conference on Machine Learning and Cybernetics (pp. 3161-3166).
8. Chmielewski, M.R., & Grzymala-Busse, J.W. (1996). Global discretization of continuous attributes as preprocessing for machine learning. International Journal of Approximate Reasoning, 15(4), 319-331. doi: 10.1016/S0888-613X(96)00074-6.

9. Choquet, G. (1954). Theory of capacities. Annals of Institute of Fourier, 5, 131–295.
10. Cui, L., & Li, Y. (2008). Linguistic quantifiers based on Choquet integrals. International Journal of Approximate Reasoning, 48(2), 559-582. doi: 10.1016/j.ijar.2007.11.001.
11. Dougherty, J., Kohavi, R., & Sahami, M. (1995). Supervised and unsupervised discretization of continuous features. In 12th International Conference on Machine Learning. Los Altos, CA: Morgan Kaufmann, (pp. 194-202).
12. Flynt, A., & Dean, N. (2016) A survey of popular R packages for cluster analysis. Journal of Educational and Behavioral Statistics, 41(2), 205-225. doi: 10.3102/1076998616631743.
13. Garcia, S., Luengo, J., Saez, A., Lopez, V., & Herrera, F. (2013). A survey of discretization techniques: Taxonomy and empirical analysis in supervised learning. IEEE Transactions on Knowledge and Data Engineering, 25(4), 734-750. doi: 10.1109/TKDE.2012.35.
14. Grabisch, M. (1996). The application of fuzzy integrals in multicriteria decision making. European Journal of Operational Research, 89(3), 445-456, doi: 10.1016/0377-2217(95)00176-X.
15. Greene, W.H. (2016). Econometric Analysis. 7th Edition, Pearson Education, Inc., Publishing as Prentice Hall. NJ 074458, USA.
16. Han, L., & Wei, C. (2017). Group decision making method based on single valued neutrosophic Choquet integral operator. Operations Research Transactions, 21(2), doi: 10.15960/j.cnki.issn.1007-6093.2017.02.012.
17. Herde, C.N., Wüstenberg, S., & Greiff, S. (2016). Assessment of complex problem solving: What we know and what we don’t know. Applied Measurement in Education, 29(4), 265-277, doi: 10.1080/08957347.2016.1209208.
18. Huber, S.G., & Skedsmo, G. (2016). Teacher evaluation accountability and improving teaching practices. Educational Assessment, Evaluation and Accountability, 28(3), 105-109, doi: 10.1007/s11092-016-9241-1.
19. Jain, A.K. (2010). Data clustering: 50 years beyond k-means. Pattern Recognition Letters, 31(8), 651-666, doi: 10.1016/j.patrec.2009.09.011.
20. Kasparian, J., & Rolland, A. (2012). OECD's ‘Better Life Index’: Can any country be well ranked?. Journal of Applied Statistics, 39(10), 2223-2230, doi: 10.1080/02664763.2012.706265.
21. Kononenko, I., & Kukar, M. (2007). Machine learning and data mining: Introduction to principles and algorithms. Harwood Publishing Limited.
22. Liu, W.F., Du Y. X., & Chang J. (2018). Intuitionistic fuzzy interaction choquet integrals operators and applications in decision making. Fuzzy systems and Mathematics, 32(2), doi: 1001-7402(2018)02-0110-11.
23. Liu, H., Hussain, F., Tan, C.L., & Dash, M. (2002). Discretization: An enabling technique. Data Mining and Knowledge Discovery, 6(4), 393-423, doi: 10.1023/A:1016304305535.
24. Marichal, J.L., & Roubens, M. (2000). Determination of weights of interacting criteria from a reference set. European Journal of Operational Research, 124(3), 641-650, doi: 10.1016/S0377-2217(99)00182-4.
25. Peña-Ayala, A. (2014). Educational data mining: A survey and a data mining-based analysis of recent works. Expert Systems with Applications, 41(4), 1432-1462, doi: 10.1016/j.eswa.2013.08.042.
26. Pötzelberger, K., & Felsenstein, K. (1993). On the fisher information of discretized data. The Journal of Statistical Computation and Simulation, 46(3-4), 125-144.
27. Pyle, D. (1999). Data Preparation for Data Mining. Morgan Kaufmann Publishers, Inc.
28. Schmeidler D. (1989). Subjective probability and expected utility without additivity. Econometrica, 57(5), 571-587, doi: 10.2307/1911053.
29. Shieh, J.I., Wu, H.H., & Liu, H.C. (2009). Applying a complexity-based Choquet integral to evaluate students’ performance. Expert Systems with Applications, 36(3), 5100-5106, doi: 10.1016/j.eswa.2008.06.003.
30. Slater, S., Joksimovic, S., Kovanovic, V., & Baker, R.S. (2017). Tools for educational data mining: A review. Journal of Educational and Behavioral Statistics, 42(1), 85-106, doi: 10.1016/j.eswa.2008.06.003.
31. Sun H. X., Yao O. Y., Wu J. Z. (2015). Interval neutrosophic numbers Choquet integral operator for multi-criteria decision making. Journal of Intelligent and Fuzzy Systems, doi: 10.3233/IFS-141524.
32. Tan, P.N., Steinbach, M., & Kumar, V. (2005). Introduction to data mining. Pearson, Addison Wesley.
33. Veeramuthu, P., Periyasamy, R., & Sugasini, V. (2014). Analysis of student result using clustering techniques International Journal of Computer Science and Information Technologies, 5(4), 5092-5094.
34. Wang, R.-S. & Ha, M.-H. (2008). On the properties of sequences of fuzzy-valued Choquet integrable functions. Fuzzy Optimization and Decision Making, 7, 417-431, doi: 10.1007/s10700-008-9040-3.
35. Wang, Z., Nian, Y., Chu, J., & Shi, Y. (2012). Emerging Computation and Information Technologies for Education. Mao, E., Xu, L., & Tian, W. (Eds.), Springer Verlag, Berlin Heidelberg.
36. Wen, X., Yan, M., Xian, J., Yue, R. & Peng, A. (2016). Supplier selection in supplier chain management using Choquet integral-based linguistic operators under fuzzy heterogeneous environment. Fuzzy Optimization and Decision Making, 15, 307-330.
37. Xu, Z. (2010). Choquet integrals of weighted intuitionistic fuzzy information. Information Sciences, 180(1), 726-736, doi: 10.1016/j.ins.2009.11.011.
38. Zopounidis, C., Doumpos, M. (2002). Multicriteria classification and sorting methods: A literature review. European Journal of Operational Research, 138(2), 229-246, doi: 10.1016/S0377-2217(01)00243-0.