Planning Activities in Software Testing Process: A Literature Review and Suggestions for Future Research

Abstract

Software testing process consists of activities that implemented after it is planned and including to document related testing activities. Test processes must be applied necessarily for able to clearly see the quality of software, the degree of reliability, whether it is ready for delivery, the degree of effectiveness and remains of how much testing. One of the most important phase of these processes is test planning activities. Test planning activities directly affects the project's success in software projects. In this study, software testing process, and test planning activities carried out in this process was first clearly demonstrated by the literature review. Later, a basic software testing process and test planning process were determined and explained step by step. In this way, it was aimed to give a different and deep perspective to the test planning activities and to raise awareness on the topic. As a result of the research, it was seen that the test planning activities are not applied adequately at present, software testers, experts or researchers do not have enough knowledge about the details of the method, and this situation causes very serious negative results in the delivery and cost in software projects. Finally, the topic was discussed in detail, and some conclusions and recommendations were given for the personnel in the field of software testing. In this point, the study is expected to contribute the literature.

Keywords

• software testing,testing process,test planning,test reporting

References

1. [1] Gürbüz, A., Kurt, A., Özbek, M., “Yazılım Test Aracı Seçiminde Tuzaklar”, III. Ulusal Yazılım Mühendisliği Sempozyumu, Ankara, 1, 2007.
2. [2] T. Britton, L. Jeng, G. Carver, P. Cheak, T. Katzenellenbogen, Reversible Debugging Software, University of Cambridge, Judge Business School, 2013.
3. [3] Garousi, V., Felderer, M., & Hacaloğlu, T. Software test maturity assessment and test process improvement: A multivocal literature review. Information and Software Technology, 85, 16-42, (2017).
4. [4] V. Garousi, A. Coşkunçay, A. B. Can , O. Demirörs , A survey of software engineering practices in Turkey, J. Syst. Softw. 108, 148–177, (2015).
5. [5] V. Garousi, J. Zhi , A survey of software testing practices in Canada„J. Syst. Softw. 86, 1354–1376, (2013).
6. [6] M. Grindal, J. Offutt, J. Mellin, On the testing maturity of software pro- ducing organizations, Testing: Academia & Industry Conference-Prac- tice And Research Techniques, 2006.
7. [7] Walia, M., & Gupta, A., Improvement in Key Project Performance Indicators through deployment of a Comprehensive Test Metrics Advisory Tool. International Journal of Advanced Research in Computer Science, 8(5), (2017).
8. [8] Afzal, Wasif, "Metrics in software test planning and test design processes." (2007).

9. [9] Pressman, Roger S, "Software engineering: a practitioner's approach", Palgrave Macmillan, 2005.
10. [10] Bormann, J., Fedeli, A., Frank, R. ve Winkelmann,K., “Combined Static and Dynamic Verification”, Research Report, FP6-IST-507219, Version 2-Public Version, 31 March 2005.
11. [11] Korkmaz, Ö., Akman, İ., “Yazılım Kalitesini Değerlendirmek için Bir MDP Tabanlı Simülasyon Modeli ve Bir Örnek Uygulama”, VI. Ulusal Yazılım Mühendisliği Sempozyumu (UYMS), Ankara, 1, 2009.
12. [12] Galin, D., “Software Quality Assurance – From theory to implementation”, Pearson – Addison Wesley, England, 2004.
13. [13] Jamil, M. A., Arif, M., Abubakar, N. S. A., & Ahmad, A. (2016, November). Software Testing Techniques: A Literature Review. In Information and Communication Technology for The Muslim World (ICT4M), 2016 6th International Conference on (pp. 177-182). IEEE.
14. [14] D. Nicola et al. "A grey-box approach to the functional testing of complex automatic train protection systems." Dependable Computing-EDCC 5. Springer Berlin Heidelberg, 305-317, 2005.
15. [15] Garousi, V., & Mäntylä, M. V., When and what to automate in software testing? A multi-vocal literature review. Information and Software Technology, 76, 92-117, (2016).
16. [16] Khan, R., Srivastava, A. K., & Pandey, D. (2016, November). Agile approach for Software Testing process. In System Modeling & Advancement in Research Trends (SMART), International Conference (pp. 3-6). IEEE.
17. [17] Garousi, V., & Pfahl, D., When to automate software testing? A decision‐support approach based on process simulation. Journal of Software: Evolution and Process, 28(4), 272-285, (2016).
18. [18] Dustin E, Garrett T, Gauf B. Implementing automated software testing: How to save time and lower costs while raising quality. Addison-Wesley Professional, (2009).
19. [19] Nidagundi, P., & Novickis, L. (2016). Introduction to lean canvas transformation models and metrics in software testing. Applied Computer Systems, 19(1), 30-36.;
20. [20] K. M. Mustafa, R. E. Al-Qutaish, M. I. Muhairat, “Classification of Software Testing Tools Based on software Testing Methods,” in Second Int. Conf. on Computer and Electrical Engineering, ICCEE '09, 2009, pp. 229–233. http://dx.doi.org/10.1109/ICCEE.2009.9.
21. [21] O’regan, G., “A pratical approach to software quality”, Maple-Vail Book Manufacturing Group, Newyork, 73-77 (2002).
22. [22] Kaveh, P., “A framework of the use of information in software testing”, The Faculty Of The School Of Engineering and Applied Science Of The George Washington University in Partial Satisfaction of The Requirements for The Degree of Doctor of Science, May 16, 2010.
23. [23] Black, R., “Managing the Testing Process”, Microsoft Press, December 1, 2004.
24. [24] Calp, M.H., “Nesne Yönelimli Yazılım Testi ve Metrik Kümesi Değerlendiren Uzman Modülün Gerçekleştirilmesi”, Yüksek Lisans Tezi, Gazi Üniversitesi Bilişim Enstitüsü, Ankara, 2011.
25. [25] Akyüz, D., “Yazılım Doğrulama Sürecinde Gereksinim Yönetim Aracı Kullanımı”, II. Yazılım Kalitesi ve Yazılım Geliştirme Araçları Sempozyumu, Ankara, 3-4 Aralık, 1-2, 2010.
26. [26] S. Amland, "Risk-based testing:" Journal of Systems and Software, vol. 53, no. 3, pp. 287–295, Sep. 2000.
27. [27] Redmill and Felix, “Theory and Practice of Risk-based Testing”, Software Testing, Verification and Reliability, Vol. 15, No. 1, March 2005.
28. [28] B. Agarwal et al., “Software engineering and testing”. Jones & Bartlett Learning, 2010.
29. [29] Crespo AN, Jino M, Argollo M, Bueno PMS, Barros CP (2010) Generic Process Model for Software Testing. Online. http://www.softwarepublico.gov.br/5cqualibr/xowiki/Teste-item13.
30. [30] Hass, AMJ, Testing processes. In: IEEE International Conference on Software Testing Verification and Validation Workshop (ICSTW). IEEE, Lillehammer, Norway. pp 321–327, (2008).
31. [31] Kasoju, A., Petersen, K., & Mäntylä, M. V., Analyzing an automotive testing process with evidence-based software engineering. Information and Software Technology, 55(7), 1237-1259, (2013).
32. [32] T. Dybå, T. Dingsøyr, Empirical studies of agile software development: a systematic review, Information & Software Technology 50 (9-10) (2008) 833–859.
33. [33] Çatal, Ç. & Diri, B., Yazılım Kalite Sınıflandırma Probleminde Yeni Yaklaşımlar: Yapay Bağışıklık Sistemleri, III. Ulusal Yazılım Mühendisliği Sempozyumu, Ankara, 1, (2007).
34. [34] Tuna, O., “Yazılım geliştirme süreci ve mimari gösterime dayalı yazılım testi”, Yüksek Lisans Tezi, Dokuz Eylül Üniversitesi Fen Bilimleri Enstitüsü, İzmir, 1, (2005).
35. [35] Lewis, E. W., “Software Testing and Continuous Quality Improvement, 2nd ed.”, A CRC Pres Company, USA, 10, 60, 117-127, 129-155, 183-185, 230-256, (2005).
36. [36] Mosley, D.J. ve Posey B.A., “Just Enough Software Test Automation”, Prentice Hall PTR, USA, 10, (2002).
37. [37] Muller, T., Graham, D., Friedenberg, D. ve Veendendal, E., “International Software Testing Qualifications Board (ISTQB)”, Foundation Level Syllabus, USA, 10, 2007.
38. [38] Camargo, K. G., Ferrari, F. C., & Fabbri, S. C. (2015). Characterising the state of the practice in software testing through a TMMi-based process. Journal of Software Engineering Research and Development, 3(1), 7.
39. [39] IEEE (2009) IEEE Standard for Software and System Test Documentation, Institute of Electric and Electronic Engineers, Std 829-1988. IEEE Computer Society, New York, NY, USA.
40. [40] Rao, D. N., Srinath, M. V., & Bala, P. H. (2013, February). Reliable code coverage technique in software testing. In Pattern Recognition, Informatics and Mobile Engineering (PRIME), 2013 International Conference on (pp. 157-163). IEEE.
41. [41] Gürbüz, A., “Yazılım test mühendisliği”, Papatya Yayıncılık Eğitim, İstanbul, 34, 2010.
42. [42] İnternet: SoftwareQATest.com, “What's a 'test plan'?”, http://www.softwareqatest.com/qatfaq2.html#FAQ2_6b, 2011. Access Date: 01.02.2018.
43. [43] Mustafa, K. ve Khan, R.A., “Software Testing: Concept and Practices”, India, Lucknow, Oxford: Alpha Science International, 194-211, 2007.
44. [44] Tiftik, N., Öztarak, H., Ercek, G. ve Özgün, S., “Sistem/yazılım geliştirme sürecinde doğrulama faaliyetleri”, III.Ulusal Yazılım Mühendisliği Sempozyumu, Ankara, 1-2, 2007.
45. [45] Afzal, W., Alone, S., Glocksien, K., & Torkar, R. (2016). Software testing process improvement approaches: A systematic literature review and an industrial case study. Journal of Systems and Software, 111, 1-33.].
46. [46] Bentley, J. E., Bank, W., Charlotte, N.C., “Software Testing Fundamentals—Concepts, Roles, and Terminology”, SAS Institute Inc., SUGI 30 Proceeding, USA, 10-13 April, 2005.
47. [47] Demir, D., “Endüstride yazılım testi ve kalite güvencesi etkinlikleri”, I. Ulusal Yazılım Mühendisliği Sempozyumu, İzmir, 23-25 Ekim 2003.
48. [48] IEEE 610.12.1990: IEEE 610.12, Glossary of Software Engineering Terminology, New York, 1990.
49. [49] Smith, M.D., Robson, D.J, “Object oriented programming the problems of validation”, Software Maintenance, 1990. (ICSM '90) Proceedings, IEEE International Conference, CH2921-5/90/0000/0272, 1990.
50. [50] ANSI Std-1991, “Standart glossary of software engineering terminology (ANSI)”, The institute of electrical and electronics engineers inc., 1991.
51. [51] İnternet: CaBIG-NCI, “Test Plan”, https://cabig.nci.nih.gov/.../Test%20Plan%20Template_Baseline.doc, 2007.
52. [52] Bayar, V., “Bileşen Yönelimli Yazılım Geliştirme için Süreç Modeli”, II. Ulusal Yazılım Mühendisliği Sempozyumu (UYMS), Ankara, 2, 2005.
53. [53] İnternet: Türkiye Linux Kullanıcıları Derneği (TLKD), “Pardus Yazılım Sistemi ve Hata Takip Sistemi”, http://seminer.linux.org.tr/wpcontent/uploads/2010/06/semen_cirit_pardus_yazilim_testleri_ve_hata_takip_sistemi_03042010.pdf , 3 Nisan 2010.
54. [54] İnternet: Coley Consulting, “Software Testing Exit Criteria”, http://www.coleyconsulting.co.uk/testing-exit-criteria.htm, 2010.
55. [55] İnternet: Wikipedia, “Regression testing”, http://en.wikipedia.org/wiki/Regression_testing, 2011.
56. [56] Eickehnann, N. S. and Richardson, D. J., “An Evaluation of Software Test Environment Architectures”, In Proceedings of ICSE'1996. pp.353~364, 1996.
57. [57] Clarke, L. A., Richardson, D.J. and Zeil, S.J., “Team A support environment for testing, evaluation, and analysis.” In Proceedings of ACM SIGSOFT’88: Third Symposiumon Software Development Environments,pages153-162.November 1988.
58. [58] Karaduman, Ö. ve ark., “Gerçek Zamanlı Sistemlerde Otomatik Test Yaklaşımı”, II. Yazılım Kalitesi ve Yazılım Geliştirme Araçları Sempozyumu, Ankara, 3-4 Aralık, 1-2, 2010.
59. [59] Software Test Automation: Myths and Facts, M.N Alami IMI Systems Inc.
60. [60] Why Automation Projects Fail (and what you can do to prevent failure), Art Beall, 2008.
61. [61] İnternet: Aptest.com, “Bug and Defect Tracking Tools”, http://www.aptest.com/bugtrack.html, 2011.
62. [62] Hoffman, M., Kuhn, N., Weber, M., Bittner, M., “Requirements for Requirements Management Tools”, Proceedings of the Requirements Engineering Conference 12th IEEE International Washington, DC, USA. IEEE Computer Society, 2004.
63. [63] Akar, B., Dinçer, K., Tümay, A., “Yazılım Ürün ve Süreç Ölçümlerinin Otomasyonu”, II.Ulusal Yazılım Mühendisliği Sempozyumu (UYMS), Ankara, 2, 2005.
64. [64] İnternet: Process Impact, Wiegers, K. E., “A Software Metrics Primer”, http://www.processdox.com/pix/metrics_primer.pdf, 1999.