Machine Learning Implementation in Automated Software Testing: A Review

Abstract

The integration of Machine Learning (ML) in automated software testing represents a transformative approach aimed at enhancing the efficiency, accuracy, and scope of testing processes. This paper explores the theoretical and practical aspects of employing ML techniques within the realm of software testing, focusing on key areas such as test case generation, defect prediction, and test suite optimisation. Through a comprehensive literature review and case studies, this study illustrates the potential benefits associated with ML-driven testing methodologies. The findings indicate that ML can significantly reduce manual intervention and improve defect detection rates, thereby facilitating more reliable software delivery. This paper also addresses the benefits of ML implementation in automated testing and future research directions to bridge existing gaps and further leverage ML in software testing.

Keywords

• Artificial intelligence
• automated testing
• software testing activities
• machine learning algorithm

References

1. Gerhard Lakemeyer and Bernhard Nebel. 1994. Foundations of Knowledge Representation and Reasoning. Springer, Berlin, 1-12. https://doi.org/10.1007/3-540-58107-3_1 google scholar
2. Santiago Matalonga, Domenico Amalfitano, Andrea Doreste, Anna Rita Fasolino, and Guilherme Horta Travassos. 2022. Alterna-tives for testing of context-aware software systems in non-academic settings: Results from a rapid review. Info. Softw. Technol. 149 (2022), 106937. https://doi.org/10.1016/j.infsof.2022.106937 google scholar
3. Tariq M. King, Jason Arbon, Dionny Santiago, David Adamo,Wendy Chin, and Ram Shanmugam. 2019. AI for testing today and tomorrow: Industry perspectives. In Proceedings of the IEEE International Conference On Artificial Intelligence Testing (AITest’19). IEEE, 81-88. https://doi.org/10.1109/AITest.2019.000-3 google scholar
4. P. Paygude and S. D. Joshi. 2020. Use of evolutionary algorithm in regression test case prioritization: A review. In Proceeding of the International Conference on Computer Networks, Big Data and IoT (ICCBI’18). Lecture Notes on Data Engineering and Communications Technologies, A. Pandian, T. Senjyu, S. Islam, and H.Wang (Eds.). Vol. 31, Springer, Cham, 56-66. https://doi.org/ 10.1007/978-3-030-24643-3_6 google scholar
5. Vahid Garousi, Sara Bauer, and Michael Felderer. 2020. NLP-assisted software testing: A systematic mapping of the literature. Info. Softw. Technol. 126 (2020), 106321. https://doi.org/10.1016/j.infsof.2020.106321 google scholar
6. M. Craglia, A. Annoni, P. Benczur, P. Bertoldi, B. Delipetrev, G. De Prato, C. Feijoo, E. Fernandez Macias, E. Gomez Gutierrez, M. Iglesias Portela, H. Junklewitz, M. Lopez Cobo, B. Martens, S. Figueiredo Do Nascimento, S. Nativi, A. Polvora, J. I. Sanchez Martin, S. Tolan, I. Tuomi, and L. Vesnic Alujevic. 2018. Artificial Intelligence: A European Perspective. Technical Report KJ-NA-29425-EN-N. Luxembourg. https://doi.org/10.2760/11251 google scholar
7. Domenico Amalfitano, Stefano Faralli, Jean Carlo Rossa Hauck, Santiago Matalonga, and Damiano Distante. 2023. Artificial Intel-ligence Applied to Software Testing: A Tertiary Study. ACM Comput. Surv. 56, 3, Article 5 (October 2023), 38 pages.https://doi.org/ 10.1145/3616372 google scholar
8. J. J. Hopfield. 1982. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. U.S.A. 79, 8 (Apr. 1982), 2554-2558. https://doi.org/10.1073/pnas.79.8.2554 google scholar

9. David E. Rumelhart, BernardWidrow, and Michael A. Lehr. 1994. The basic ideas in neural networks. Commun. ACM 37, 3 (Mar. 1994), 87-92. https://doi.org/10.1145/175247.175256 google scholar
10. Leo Breiman. 2000. Bias, Variance, and Arcing Classifiers. Technical Report 460, Statistics Department, University of California. google scholar
11. S. B. Kotsiantis, I. D. Zaharakis, and P. E. Pintelas. 2006. Machine learning: A review of classification and combining techniques. Artific. Intell. Rev. 26, 3 (Nov. 2006), 159-190. https://doi.org/10.1007/s10462-007-9052-3 google scholar
12. Dongkuan Xu and Yingjie Tian. 2015. A comprehensive survey of clustering algorithms. Ann. Data Sci. 2, 2 (2015), 165-193. https:// doi.org/10.1007/s40745-015-0040-1 google scholar
13. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. Retrieved from http://www.deeplearningb ook.org google scholar
14. Bernard M. E. Moret. 1982. Decision trees and diagrams. ACM Comput. Surv. 14, 4 (Dec. 1982), 593-623. https://doi.org/10.1145/356893.356898 google scholar
15. D. Opitz and R. MacLin. 1999. Popular ensembLe methods: An empiricaL study. J. Artific. InteLL. Res. 11 (Aug. 1999), 169-198. https://doi.org/10.1613/jair.614 google scholar
16. Trevor Hastie, Robert Tibshirani, and Jerome Friedman. 2009. The ELements of StatisticaL Learning. Springer, New York. https:// doi.org/10.1007/978-0-387-84858-7 google scholar
17. Y. Bengio, P. Simard, and P. Frasconi. 1994. Learning Long-term dependencies with gradient descent is difficuLt. IEEE Trans. NeuraL Netw. 5, 2 (1994), 157-166. https://doi.org/10.1109/72.279181 google scholar
18. L. P. KaeLbLing,M. L. Littman, and A.W. Moore. 1996. Reinforcement Learning: A survey. J. Artific. InteLL. Res. 4 (1996), 237-285. https:// doi.org/10.1613/jair.301 google scholar
19. G. Udny YuLe. 1897. On the theory of correLation. J. Roy. Stat. Soc. 60, 4 (1897), 812-854. https://doi.org/10.1111/j.2397-2335.1897.tb 02784.x google scholar
20. Stuart RusseLL and Peter Norvig. 2016. ArtificiaL InteLLigence: A Modern Approach. Pearson. Retrieved from https:// books.googLe.it/books?id=XS9CjwEACAAJ google scholar
21. Corinna Cortes and VLadimir Vapnik. 1995. Support-vector networks. Mach. Learn. 20, 3 (Sep. 1995), 273-297. https://doi.org/10.1007/BF00994018 google scholar
22. InternationaL Organization for Standardization. 2013. ISO/IEC/IEEE internationaL standard-software and systems engineering-software testing-Part 1: Concepts and definitions. ISO/IEC/IEEE 29119-1:2013(E) (2013), 64. https://doi.org/10.1109/IEEESTD.2013. 658853 google scholar
23. George Candea, Stefan Bucur, and Cristian Zamfir. 2010. Automated software testing as a service. In Proceedings of the 1st ACM Sysmposium on CLoud Computing (SOCC ‘10). Assocoation for Computing Machinery, New York, NY, USA, 155-160. https://doi.org/ 10.1145/1807128.1807153 google scholar
24. Anna Trudova, MichaL DoLezeL, and ALena BuchaLcevova. 2020. ArtificiaL inteLLigence in software test automation: A systematic Literature review. In Proceedings of the 15th InternationaL Conference on EvaLuation of NoveL Approaches to Software Engineering (ENASE’20). INSTICC, SciTePress, 181-192. https://doi.org/10.5220/0009417801810192 google scholar
25. Vinicius H. S. DureLLi, RafaeL S. DureLLi, Simone S. Borges, Andre T. Endo, MarceLo M. ELer, Diego R. C. Dias, and MarceLo P. Guimar.es. 2019. Machine Learning appLied to software testing: A systematic mapping study. IEEE Trans. ReLiabiL. 68, 3 (2019), 1189-1212. https://doi.org/10.1109/TR.2019.2892517 google scholar
26. Afonso Fontes and Gregory Gay. 2021. Using machine Learning to generate test oracLes: A systematic Literature review. In Proceedings of the 1st InternationaL Workshop on Test OracLes (TORACLE’21). ACM, New York, NY, 1-10. https://doi.org/10.1145/ 3472675.3473974 google scholar
27. Muhammad Khatibsyarbini, Mohd Adham Isa, Dayang N. A. Jawawi, Muhammad Luqman Mohd Shafie, Wan Mohd Nasir Wan-Kadir, Haza NuzLy AbduL Hamed, and Muhammad Dhiauddin Mohamed Suffian. 2021. Trend appLication of machine Learning in test case prioritization: A review on techniques. IEEE Access 9 (2021), 166262-166282. https://doi.org/10.1109/ACCESS.2021.3135508 google scholar
28. Rongqi Pan, Mojtaba Bagherzadeh, Taher A. GhaLeb, and LioneL Briand. 2021. Test case seLection and prioritization using machine Learning: A systematic Literature review. Empir. Softw. Eng. 27, 2 (Dec. 2021), 29. https://doi.org/10.1007/s10664-021-10066-6 google scholar
29. Gerson Barbosa, Erica Ferreira de Souza, Luciana BrasiL RebeLo dos Santos, MarLon da SiLva, JuLiana Marino BaLera,and Nandamudi LankaLapaLLi Vijaykumar. 2022. A systematic Literature review on prioritizing software test cases using Markov chains. Info. Softw. TechnoL. 147 (2022), 106902. https://doi.org/10.1016/j.infsof.2022.106902 google scholar