Integrating Usability into Software Engineering Course Projects

Year 2024, Volume: 12 Issue: 23, 209 - 240, 21.03.2024

Nihal Menzi Çetin

https://doi.org/10.18009/jcer.1417245

Abstract

This study presents a case study of the integration of usability evaluation into the development of task based information system prototypes in undergraduate software engineering course projects within a problem-based learning (PBL) approach. As part of the course, usability evaluation was integrated into the software development process, and the usability evaluation performance (UEP) of the projects was assessed in terms of the following criteria: effectiveness, efficiency and problem validity. It also analyzed the problem solution rate of the projects and their correlation between UEP criteria. Additionally, the study in-depth analyzed the students' final conclusions regarding the design process. The relationship between UEP criteria and the problem solving rate was examined using Spearman correlation analysis. In addition, final conclusions and problem solving behaviors were analyzed through document and thematic analysis techniques. The results supported that there is a significant correlation between usability evaluation criteria and problem solving rate. The main themes that emerged regarding usability problem solving behaviors were navigation design, error handling, database connection, algorithm design, search matching, system-user communication design and others. Final conclusions showed that project teams gained valuable insights into user-centered design, solved critical usability problems and improved their prototype design. It can be concluded that the integration of usability into software engineering education using the above approach contributes to students' understanding of user-centered interaction design.

Keywords

Usability evaluation, use case modeling, software engineering education, prototype development

Ethical Statement

Name of the board that carries out ethical assessment: Adana Alparslan Türkeş Bilim and Technology University Scientific Research and Publication Ethics Board The date and number of the ethical assessment decision: 28.02.2022 -29675

References

• Allen, D.E., Donham, R.S. and Bernhardt, S.A. (2011), Problem-based learning. New Directions for Teaching and Learning, 2011, 21-29. https://doi.org/10.1002/tl.465
• Alshamari, M. & Mayhew, P. (2008). Task design: Its impact on usability testing. Proceedings of the third international conference on internet and web applications and services, (pp. 583-589). https://doi.org/10.1109/ICIW.2008.20
• Bernhaupt, R., Palanque, P., Manciet, F., Martinie, C. (2016). User-test results injection into task-based design process for the assessment and improvement of both usability and user experience. In C. Bogdan, et al. (Eds.), Human-Centered and Error-Resilient Systems Development. HESSD HCSE 2016 2016. Lecture Notes in Computer Science, (pp. 56-72). Springer, Cham. https://doi.org/10.1007/978-3-319-44902-9_5 Boehm, B. (2005). The future of software processes. In E. Bertino, W. Gao, B. Steffen & M. Yung (Eds.). Lecture Notes in Computer Science, Software Process Workshop (pp. 10-24). Springer Berlin Heidelberg.
• Bolchini, D., Garzotto, F. (2007). Quality of web usability evaluation methods: An empirical study on MiLE+”. In M. Weske, M. S. Hacid, C. Godart, (Eds.), Web Information Systems Engineering – WISE 2007 Workshops, LNCS (vol. 4832), (pp. 481-492).
• Borys, M. (2016) Teaching software usability engineering: classroom experience, INTED2016 Proceedings, (pp. 2712-2717). http://doi.org/10.21125/inted.2016.1590
• Brodie, L., Zhou, H., & Gibbons, A. (2008). Steps in developing an advanced software engineering course using problem based learning. Engineering education, 3(1), 2-12. https://doi.org/10.11120/ened.2008.03010002
• Chattratichart, J., & Brodie, J. (2004, April). Applying user testing data to UEM performance metrics. In CHI'04 extended abstracts on Human factors in computing systems (pp. 1119-1122). https://doi.org/10.1145/985921.986003
• Ciancarini, P., Missiroli, M., & Russo, D. (2019). Cooperative Thinking: Analyzing a new framework for software engineering education. Journal of Systems and Software, 157, 110401. https://doi.org/10.1016/j.jss.2019.110401
• Cico, O., Jaccheri, L., Nguyen-Duc, A., & Zhang, H. (2021). Exploring the intersection between software industry and Software Engineering education-A systematic mapping of Software Engineering Trends. Journal of Systems and Software, 172, 110736.
• Cohen, L., & Manion, L. (1989). Research methods in education (Third edition). Routledge. Constantine, L. L. (1995). Essential modeling: Use cases for user interfaces. Interactions, 2(2), 34-46.
• Curcio, K., Santana, R., Reinehr, S., & Malucelli, A. (2019). Usability in agile software development: A tertiary study. Computer Standards & Interfaces, 64, 61-77.
• Dix, A., Finlay, J., Abowd, G. D., & Beale, R. (2004). Human computer interaction (third edition). Pearson Prentice Hall.
• Elkoutbi, M., Khriss, I., & Keller, R. K. (2006). Automated prototyping of user interfaces based on UML scenarios. Automated Software Engineering, 13, 5-40.
• Ferreira, J. M., Acuña, S. T., Dieste, O., Vegas, S., Santos, A., Rodriguez, F., & Juristo, N. (2020). Impact of usability mechanisms: An experiment on efficiency, effectiveness and user satisfaction. Information and Software Technology, 117, 106195.
• Firmenich, S., Garrido, A., Grigera, J., Rivero, J. M., & Rossi, G. (2019). Usability improvement through A/B testing and refactoring. Software Quality Journal, 27, 203-240.
• Fischer, H. (2012). Integrating usability engineering in the software development lifecycle based on international standards. In S. D. J. Barbosa, J. C. Campos, R. Kazman, P. Palanque, M. Harrison (Eds.), Proceedings of the 4th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, (pp. 321-324). Association for Computing Machinery, New York. https://doi.org/10.1145/2305484.2305541
• Georgsson, M., & Staggers, N. (2016). Quantifying usability: An evaluation of a diabetes mHealth system on effectiveness, efficiency, and satisfaction metrics with associated user characteristics. Journal of the American Medical Informatics Association, 23(1), 5-11.
• Ghezzi, C., & Mandrioli, D. (2005). The challenges of software engineering education. In Proceedings of the 27th international conference on Software engineering (ICSE '05). Association for Computing Machinery, New York, NY, USA, (pp.637–638).
• Gulati, A., Dubey, S. K. (2012). Critical analysis on usability evaluation techniques. International Journal of Engineering Science and Technology, 4(3), 990-997.
• Gupta, D., Ahlawat, A., & Sagar, K. (2017). Usability prediction & ranking of SDLC models using fuzzy hierarchical usability model. Open Engineering, 7, 161-168.
• Hartson, H. R., Andre, T. S., & Williges, R. C. (2003). Criteria for evaluating usability evaluation methods. International Journal of Human-Computer Interaction, 15(1), 145-181.
• Hertzum, M. (2006). Problem prioritization in usability evaluation: From severity assessments toward impact on design. International Journal of Human-Computer Interaction, 21(2), 125-146. https://doi.org/10.1207/s15327590ijhc2102_2
• Hollnagel, E. (2012). Task analysis: Why, what and how. In G. Salvendy (Ed.), Handbook of human factors and ergonomics (fourth edition). (pp.385-396). John Wiley & Sons, Inc.
• Hornbæk, K. (2006). Current practice in measuring usability: Challenges to usability studies and research. International Journal of Human-Computer Studies, 64(2), 79-102.
• Koutsabasis, P., Spyrou, T. & Darzentas, J. (2007). Evaluating usability evaluation methods: criteria, method and a case study. In Proceedings of International Conference on Human-Computer Interaction, (pp. 569-578).
• Kuhnel, M., Seiler, L., Honal, A., & Ifenthaler, D. (2018). Mobile learning analytics in higher education: Usability testing and evaluation of an app prototype, Interactive Technology and Smart Education, 15(4), 332-347.
• Lethbridge, T.C. & Laganiere, R. (2004). Object-oriented software engineering. Practical software development using UML and Java (second edition). London: McGraw-Hill.
• Liaghati, C., Mazuchi, T., & Sarkani, S. (2020). A method for the inclusion of human factors in system design via use case definition. Human-Intelligent Systems Integration, 2, 45–56.
• Lim, Y. K., Pangam, A., Periyasami, S., & Aneja, S. (2006, October). Comparative analysis of high-and low-fidelity prototypes for more valid usability evaluations of mobile devices. In Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles (pp. 291-300). https://doi.org/10.1145/1182475.1182506
• Lingaard, G., & Chattratichart, J. (2007). Usability testing: What have we overlooked? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, (pp. 1415-1425). San Jose, USA. https://doi.org/10.1145/1240624.1240839
• Madan, A., & Dubey, S. K. (2012). Usability evaluation methods: A literature review. International Journal of Engineering Science and Technology, 4(2, February), 590-599.
• Maguire, M., & Isherwood, P. (2018). A comparison of user testing and heuristic evaluation methods for identifying website usability problems. In Design, User Experience, and Usability: Theory and Practice: 7th International Conference, DUXU 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, Proceedings, Part I 7 (pp. 429-438). Springer International Publishing.
• Martins, H. F., Oliveira, A. C., Canedo, E. D., Kosloski, R. A. D., Paldês, R. A., & Oliveira, E. C. (2019). Design thinking: Challenges for software requirements elicitation. Information, 10(12), 371. https://doi.org/10.3390/info10120371 McCloskey, M. (2021). Turn user goals into task scenarios for usability testing. https://www.nngroup.com/articles/task-scenarios-usability-testing/ 09.11.2021.
• Mitchell, G. G., & Delaney, J. D. (2004). An assessment strategy to determine learning outcomes in a software engineering problem-based learning course. International Journal of Engineering Education, 20(3), 494-502.
• Myers, L. & Sirois, M. J. (2006). Spearman correlation coefficients, differences between. Encyclopedia of statistical sciences, 12. https://doi.org/10.1002/0471667196.ess5050.pub2
• Nielsen, J. (1994). Severity ratings for usability problems. https://www.nngroup.com/articles/how-to-rate-the-severity-of-usability-problems/ Nielsen, J. & Landauer, T. K. (1993). A mathematical model of the finding of usability problems. In Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems, (pp. 206-213). https://doi.org/10.1145/169059.169166
• Ponce, P., Peffer, T., & Molina, A., (2018). Framework for evaluating usability problems: a case study low-cost interfaces for thermostats. International Journal of Interactive Design and Manufacturing, 12(2), 439-448. https://doi.org/10.1007/s12008-017-0392-1
• Pressman, R. S. (2010). Software engineering. A practitioner’s approach (seventh edition). McGraw Hill International Edition.
• Puerta, A. R. (1996). The Mecano Project: Enabling user-task automation during interface development. In AAAI Technical Report (AAAI, 1996), (vol. 96, pp. 117-121).
• Segura, J. (2021). The teaching of usability in software development: Case study in the computer engineering career at the university of Matanzas. International Journal of Engineering Pedagogy, 11(1), 4-15. Silva, T. S., Silveira, M. S., & Maurer, F. (2015, January). Usability evaluation practices within agile development. In 2015 48th Hawaii International Conference on System Sciences (pp. 5133-5142). IEEE. https://doi.org/10.1109/HICSS.2015.607
• Sinnig, D., Chalin, P., & Khendek, F. (2008). Consistency between task models and use cases. In Engineering Interactive Systems: EIS 2007 Joint Working Conferences, EHCI 2007, DSV-IS 2007, HCSE 2007, Salamanca, Spain, (pp. 71-88). Springer Berlin Heidelberg.
• URL: J. Nielsen, Why you only need to test with 5 users?. https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ 09.09.2021.
• Velmourougan, S., Dhavachelvan, P., Baskaran, R., & Ravikumar, B. (2014, September). Software development Life cycle model to build software applications with usability. In 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI) (pp. 271-276). IEEE.
• Yoon, H., Park, S. H., Lee, K. T., Park, J. W., Dey, A. K., & Kim, S. (2017). A case study on iteratively assessing and enhancing wearable user interface prototypes. Symmetry, 9(7), 114. https://doi.org/10.3390/sym9070114
• Yue, T., Briand, L. C., & Labiche, Y. (2013). Facilitating the transition from use case models to analysis models: Approach and experiments. ACM Transactions on Software Engineering and Methodology (TOSEM), 22(1), 1-38. https://doi.org/10.1145/2430536.2430539
• Zainol, A. & Almukadi, W. S. (2020). Implementing problem-based learning in the software engineering course. International Journal of Advanced and Applied Sciences, 7(12), 19-26. https://doi.org/10.21833/ijaas.2020.12.002