A Comparative Study for Evaluating Automated Software Testing Tools

Year 2022, , 301 - 316, 31.07.2022

Ekrem Erol , Sibel Senan

https://doi.org/10.17671/gazibtd.1057380

Abstract

The quality of software depends on the use of software testing tools effectively. While the time and importance devoted to the testing process increasing in the software development lifecycle, it is seen that there are not enough resources to show which automated software testing tool is more efficient in which software. The aim of this paper is to examine and evaluate by comparing different test automation tools according to categories such as performance, cost, usability, etc. to support this resource deficit. For this purpose, the most preferred software test automation tools have been analyzed by experimenting on computers with different hardware specifications and different websites. Operating conditions of software testing tools under the same conditions are also examined and compared. In this way, we aim to make guidance to evaluate the capabilities and properties of examined testing tools. We believe that this study will be a useful guide for reducing testing costs and time in the software developing process of websites.

Keywords

software testing, automated testing tools, user acceptance testing

Otomatik Yazılım Test Araçlarını Değerlendirmek için Karşılaştırmalı Bir Çalışma

Abstract

Yazılımın kalitesi, yazılım test araçlarının etkin kullanımına bağlıdır. Yazılım geliştirme yaşam döngüsünde test sürecine ayrılan zaman ve önem artarken, hangi otomatik yazılım test aracının hangi yazılımda daha verimli olduğunu gösterecek yeterli kaynağın olmadığı görülmektedir. Bu makalenin amacı, bu kaynak açığını desteklemek için farklı test otomasyon araçlarını performans, maliyet, kullanılabilirlik vb. kategorilere göre karşılaştırarak incelemek ve değerlendirmektir. Bu amaçla en çok tercih edilen yazılım test otomasyon araçları farklı donanım özelliklerine sahip bilgisayarlarda ve farklı web sitelerinde denenerek analiz edilmiştir. Yazılım test araçlarının aynı koşullardaki çalışma koşulları da incelenerek karşılaştırılmıştır. Bu sayede incelenen test araçlarının yeteneklerini ve özelliklerini değerlendirmek için rehberlik etmeyi amaçlıyoruz. Bu çalışmanın web sitelerinin yazılım geliştirme sürecinde test maliyetlerini ve süresini azaltmak için faydalı bir rehber olacağına inanıyoruz.

Keywords

yazılım testi, otomatik test araçları, kullanıcı kabul testi

References

• Erdem, O. A. , Younis, A. "Yazılım Projelerinin Geliştirme Sürecinde Yönetim", Bilişim Teknolojileri Dergisi, 7(1), 1-9, 2014.
• Erdem, O. A. , Younis, A. E. "Yazılım Projelerinde Risk Yönetimi", Bilişim Teknolojileri Dergisi, 5(1), 1-6, 2012.
• I. Burnstein, Practical Software Testing: A Process-Oriented Approach, Springer Professional Computing, 2013.
• V.N. Nair, D. A. James, W. K. Ehrlich, et al, “A Statistical Assessment of Some Software Testing Strategies And Application Of Experimental Design Techniques”, Statistica Sinica, 8(1), 165–84, 1998.
• Grindal, Mats, Jeff Offutt, and Sten F. Andler. "Combination Testing Strategies:A Survey", Software Testing, Verification and Reliability, 15(3), 167-199, 2005.
• O. Taipale, J. Kasurinen, K. Karhu, et al, “Trade-Off Between Automated And Manual Software Testing”, Int. Journal of System Assurance Engineering and Management, 2(2), 114-125, 2011.
• M. Fewster and D. Graham, Software Test Automation: Effective Use of Test Execution Tools, ACM Press/Addison-Wesley Publishing Co., 1999.
• T. Maneela and D. Gaurav, “A Research Study on importance of Testing and Quality Assurance in Software Development Life Cycle (SDLC) Models”, International Journal of Soft Computing and Engineering, 2(3), 2231-2307, 2012.
• R.Torkar, “Towards automated software Testing Techniques, classifications and frame work”, Blekinge Institute of Technology, 2006.
• E. Dijkstra, “Notes on Structured Programming”, Technical Report 70-WSK-03, 1970.
• K.M. Mustafa et.al, “Classification of Software Testing Tools Based on the Software Testing Methods”, Second International Conference on Computer and Electrical Engineering, Al-Zaytoonah University of Jordan, 2009.
• R.K. Keller, R. Weber, & S. Berner, “Observations and lessons learned from Automated testing”, Proceedings of the 27th International Conference on Software Engineering, 571-579, 2005.
• S. Uspenskiy, A survey and classification of software testing tools, Master Thesis, Lappeenranta University of Technology, 2010.
• H. Kaur, G. Gupta, “Comparative Study of Automated Testing Tools: Selenium, Quick Test Professional and Testcomplete”, Int. Journal of Engineering Research and Applications, 3(5), 1739-1743, 2013.
• K. Saravanan, E. Sreedevi, “Open Source Automated Testing: An Insight Into Current Trends And Scope For Further Research”, Int. Journal of Pure and Applied Mathematics, 118(17), 669-679, 2018.
• R.R. Polamreddy, S.A. Irtaza, Software Testing: A Comparative Study Model Based Testing vs Test Case Based Testing, Master Thesis, School of Computing Blekinge Institute of Technology, 2012.
• S.C.R. Chitirala, Comparing the effectiveness of automated test generation tools “EVOSUITE” and “Tpalus”. Master Thesis, University Of Minnesota, 2015.
• M. Shtakova, Evaluation of methods for automated testing in large-scale financial systems, Master Thesis, Uppsala University, 2012.
• D. Atesogullari, A. Mishra, “Automation Testing Tools: A Comparative View”, Int. Journal on Information Technologies & Security, 12 (4), 63-76, 2020.
• R. Khalid, “Towards an Automated Tool for Software Testing and Analysis”, Proc. of Int. Bhurban Conf. on Applied Sciences and Technology, Islamabad, Pakistan, 461-465, 2017.
• M. Imran, M. Hebaishy, A.S. Alotaibi, “A Comparative Study of QTP and Load Runner Automated Testing Tools and their Contributions to Software Project Scenario”, International Journal of Innovative Research in Computer and Communication Engineering, 4(1), 457-465, 2016.
• L. Rajamanickam, “The Various Software Automated Web Testing Tools”, Journal of Engineering and Technological Advances, 2(2), 69-76, 2017.
• K. Shaukat, U. Shaukat, F. Feroz, et al, “Taxonomy of Automated Software Testing Tools”, Int. Journal of Computer Science and Innovation, 1, 7-18, 2015.
• H. Dandan, “A Research on Automated Software Test Case Generation Based on Control Flow”, Proc. of IEEE Int. Conf. on E-Commerce and Internet Technology, 204-207, 2020.
• D.R. Mohammad, S. Al-Momani, Y.M. Tashtoush, et al., “A Comparative Analysis of Quality Assurance Automated Testing Tools for Windows Mobile Applications”, Proc. of IEEE 9th Annual Computing and Communication Workshop and Conference, Las Vegas, NV, USA. 0414-0419, 2019.
• D.R. Graham, “Software Testing Tools: A New Classification Scheme”, Software Testing, Verification and Reliability, 1(3), 17-34, 1991.
• Internet: Medium, Best Automation Testing Tools For 2018, https://medium.com/@briananderson2209/best-automation-testing-tools-for-2018-top-10-reviews-8a4a19f664d2, 10.02.2021.
• Internet: Software Testing Help, Top 20 Automation Testing Tools, https://www.softwaretestinghelp.com/top-20-automation-testing-tools/, 10.02.2021.
• Internet: DZone, Top 10 Automated Software Testing Tools, https://dzone.com/articles/top-10-automated-software-testing-tools, 06.01.2022.
• Internet: Guru99, 20 BEST Automation Testing Tools, https://www.guru99.com/automated-testing-tools, 06.01.2022.