Developing a Model for Measuring Project Performance with Software Life Cycle Process Metrics and Calculating Project Success Score

Abstract

Despite the developments in the process and tool infrastructure in the software world, project success has not significantly improved. In software projects, the definition of project success means to produce products that the customer desires in the planned effort, time and budget. To achieve this goal, planning, analysis, design, coding, integration, testing and delivery processes are operated interactively from the beginning to the end of a software project. Metrics of these processes are used to measure the performance of software projects. Since the literature review shows that project management process metrics such as budget, effort, schedule, customer satisfaction, product quality are used in measuring project performance, more comprehensive and effective criteria are needed to be defined and applied in measuring project performances. Due to the importance of the project performance evaluation, a general evaluation model was created in this study. The proposed model is designed for use in the software industry. In terms of project performance, a model has been developed that focuses on management of project, requirement, risk, quality and configuration, development, verification and validation processes. The purpose of this article is to present a model that evaluates the performance of software projects and expresses project success with a numerical value. Analytical hierarchy process (AHP) was used to calculate the relative importance of each process metric criterion and sub-criteria that provide input to the performance evaluation. Statistical process control method was used in the evaluation of project performance and calculation of the project success score. It was operated in an R&D organization to verify the proposed model and the performance of a project in delivery phase to the customer was measured. It is thought that the model presented in this study will help the managers, who monitor the project status, to evaluate project performance, as well as provide the numerical comparison of performance between projects.

Keywords

• Project Performance Evaluation,Project Success Score,Analytical Hierarchy Process,Statistical Process Control

References

1. [1] Presidency of the Republic of Turkey Strategy And Budget Department (in 681 article).
2. [2] Presidency of the Republic of Turkey Strategy and Budget Department Medium Term Program 2016-2018 (in 180 article).
3. [3] Republic of Turkey Ministry of Industry and Technology, Turkey Software Sector Strategy and Action Plan 2017-2019.
4. [4] M. Radujković and M. Sjekavica, “Project Management Success Factors,” in Creative Construction Conference 2017, Procedia Engineering 196, Croatia, pp. 607-615, 2017.
5. [5] Merrow, E.: Independent Project Analysis Report, IPA, 2011.
6. [6] 12207-2017 ISO/IEC/IEEE International Standard Systems and software engineering - Software life cycle processes. DOI:10.1109/IEEESTD.2017.8100771
7. [7] R. Santhanam and M. J. Schniederjans, “A model formulation system for information system project selection,” Computers & operations research, vol. 20, no. 7, pp. 755- 767, 1993.
8. [8] M. M. Akyol, “Yazılım Süreçlerini Etkileyen Faktörlerin Belirlenmesine İlişkin Bir Ölçüm Ve İyileştirme Modeli Ve Uygulaması,” Ph.D. dissertation, Istanbul University, Istanbul, Turkey, 2013.

9. [9] M. Ayyıldız, O. Kalıpsız, and S. Yavuz, “YEEM: Yazılım Projeleri Maliyet Tahminleme Ölçev Seti ve Modeli,” ELECO, Bursa, Turkey, pp. 1-4, 2006.
10. [10] O. P. Sanchez, M. A. Terlizzi, and H. Moraes, “Cost and time project management success factors for information systems development projects,” International Journal of Project Management, vol. 35, pp. 1608–1626, 2017.
11. [11] F. A. Mir and A. H. Pinnington, “Exploring the value of project management: linking project management performance and project success,” International Journal of Project Management, vol. 32, pp. 202–217, 2014.
12. [12] M. J. Todorović, D. Č. Petrović, M. M. Mihić, V. L. Obradović, and S. D. Bushuyev, “Project success analysis framework: a knowledge-based approach in project management,” International Journal of Project Management, vol. 33, pp. 772–783, 2015.
13. [13] J. Verner, B. Kitchenham, and N. Cerpa, “Estimating Project Outcomes,” ICSSEA Conference, 2007-11, 2007.
14. [14] V. Garousi, A. Tarhan, D. Pfahl, A. Coşkunçay, and O. Demirörs, “Correlation of critical success factors with success of software projects: an empirical investigation,” Software Quality Journal, vol. 27, pp. 429–493, 2019.
15. [15] R. R. M. M. K. Wadugodapitiya, Y. G. Sandanayake, and N. Thurairajah, “Building project performance evaluation model,” Proc. CIB World Congress, Salford, UK, 2010.
16. [16] M. Ilbeigi and G. Heravi, “Developing a model to evaluate project performance: contractor company’s viewpoint,” in Joint Int. Symp. CIB, Rotterdam, Netherlands, pp. 130–139, 2009.
17. [17] R.G. Koelmans, “Project success and performance evaluation,” Int. Platinum Conf. The South African Institute of Mining and Metallurgy, Sun City, South Africa, Oct. 3-7, pp. 229–236, 2004.
18. [18] S. S. Leu and Y. C. Lin, “Project performance evaluation based on statistical process control techniques,” Journal of Construction Engineering and Management, vol. 134, no. 10, pp. 813- 819, 2008.
19. [19] G. T. Bauch and C. A. Chung, “A statistical project control tool for engineering managers,” Project Management Journal, vol. 32, no. 2, pp. 37-44, 2001.
20. [20] D. C. Bower and A. D. Finegan, “New approaches in project performance evaluation techniques,” International Journal of Managing Projects in Business, vol. 2, no. 3, pp. 435-444, 2009.
21. [21] B. Shahzad and A. M. Said, “Identification and quantitative analysis of project success factors for large scale projects,” International Journal of Knowledge Society Research (IJKSR), vol. 5, no. 1, pp. 83-95, 2014.
22. [22] T. G. Viglioni, J. A. O. Cunha, and H. P. Moura, “A performance evaluation model for project management office based on a multicriteria approach,” Procedia Computer Science, vol.100, pp. 955-962, 2016.
23. [23] R. Doskočil, S. Škapa, and P. Olšová, “Success evaluation model for project management,” Economics and Management, 2016.
24. [24] K. N. Shashi, T. G. Nair, and V. Suma, “SLI, a new metric to determine success of a software project,” 2014 International Conference on Electronics and Communication Systems (ICECS), pp. 1-5, 2014.
25. [25] A. Badewi, “The impact of project management (PM) and benefits management (BM) practices on project success: Towards developing a project benefits governance framework,” International Journal of Management, vol. 34, no. 4, pp. 761-778, 2016.
26. [26] A. Mossalam and M. Arafa, “The role of project manager in benefits realization management as a project constraint/driver,” HBRC Journal, vol. 12, no. 3, pp. 305-315, 2016.
27. [27] J. Gomes and M. Romão, “Improving project success: A case study using benefits and project management,” Procedia Computer Science, vol. 100, pp. 489-497, 2016.
28. [28] D. J. Bryde and L. Robinson, “Client versus contractor perspectives on project success criteria,” International Journal Of Project Management, vol. 23, no. 8, pp. 622-629, 2005.
29. [29] K. Jugdev and R. Müller, “A retrospective look at our evolving understanding of project success,” Project Management Journal, vol. 36, no. 4, pp. 19-31, 2005.
30. [30] D. Baccarini, “The logical framework method for defining project success,” Project Management Journal, vol. 30, no. 4, pp. 25-32, 1999.
31. [31] C. Besner and B. Hobbs, “The project management tools and techniques: The portrait of current professional practice,” Project Management Journal, vol. 37, no. 3, pp. 37-48, 2006.
32. [32] C. S. Lim and M. Z. Mohamed, “Criteria of project success: an exploratory reexamination, International Journal of Project Management,” vol. 17, no. 4, pp. 243-248, 1999.
33. [33] M. Radujković, “The Role of Project Management in Construction Industry Modernization,” 13th China International Construction Project Management Summit, Hangzhou, Zhejiang, pp. 10-28, 2014.
34. [34] T. Cooke-Davies, “The "real" success factors on projects,” International Journal of Project Management, vol. 20, no. 3, pp. 337-342, 2002.
35. [35] R. Müller and K. Jugdev, “Critical success factors in projects: Pinto, Slevin, and Prescott - the elucidation of project success,” International Journal of Managing Projects in Business, vol. 5, no. 4, pp.757-775, 2012.
36. [36] D. J. Bryde, “Project management concepts, methods and application,” International Journal of Operations & Production Management, vol. 23, no. 7, pp. 775 – 793, 2003.
37. [37] J. S. Chou, N. Irawan, and A. D. Pham, “Project management knowledge of construction professionals: Cross-country study of effects on project success,” Journal of Construction Engineering and Management, vol. 139, no. 11, 04013015, 2013.
38. [38] S. M. S. Ellatar, “Towards developing an improved methodology for evaluating performance and achieving success in construction projects,” Scientific Research and Essay, vol. 4, no. 6, pp. 549-554, 2009.
39. [39] PricewaterCuppers (PWC) Insights and Trends: Current Portfolio, Programme and Project Management Practices, 3rd Global survey on the current state of project management, PWC, 2012.
40. [40] A. L. R. Feger and G. A. Thomas, “A framework for exploring the relationship between project manager leadership style and project success,” International Journal of Management, vol. 1, no. 1, pp. 1-19, 2012.
41. [41] M. Keil, A. Rai, and S. Liu, “How user risk and requirements risk moderate the effects of formal and informal control on the process performance of IT projects,” European Journal of Information Systems, vol. 22, no. 6, pp. 650-672, 2013.
42. [42] H. Taylor, “Critical risks in outsourced IT projects: the intractable and the unforeseen,” Communications of the ACM, vol. 49, no. 11, pp. 74-79, 2006.
43. [43] S. M. H. M. Al-Tmeemy, H. Abdul- Rahman, and Z. Harun, “Future criteria for success of building projects in Malaysia,” International Journal of Project Management, vol. 29, no. 3, pp. 337-348, 2011.
44. [44] F. Blindenbach-Driessen, J. Van Den Ende, “Innovation in project-based firms: The context dependency of success factors,” Research Policy, vol. 35, no. 4, pp. 545-561, 2006.
45. [45] D. J. Bryde, “Perceptions of the impact of project sponsorship practices on project success,” International Journal of Project Management, vol. 26, no. 8, pp. 800–809, 2008.
46. [46] M. M. Nahod and M. V. M. Radujković, “The impact of ICB 3.0 competences on project management success,” Procedia- Social and Behavioral Sciences, vol. 74(0), pp. 244-254, 2013.
47. [47] J. Ramazani and G. Jergeas, “Project managers and the journey from good to great: The benefits of investment in project management training and education,” International Journal of Project Management, vol. 33, no. 1, pp. 41-52, 2015.
48. [48] J. R. Turner, R. Müller, and V. Dulewicz, “Comparing the leadership styles of functional and project managers,” International Journal of Managing Projects in Business, vol. 2, no. 2, pp. 198-216, 2009.
49. [49] L.A. Ika, “Project success as a topic in project management journals,” Project Management Journal, vol. 40, no. 4, pp. 6- 19, 2009.
50. [50] M. Radujković, “Voditelj projekta,” Građevinar, vol. 52, no. 03, pp. 143-151, 2000.
51. [51] E. Westerveld, “The Project Excellence Model®: linking success criteria and critical success factors,” International Journal of Project Management, vol. 21, no. 6, pp. 411-418, 2003.
52. [52] G. Skulmoski, “Project maturity and competence interface,” Cost Engineering - Ann Arbor Then Morgantown, vol. 43, no. 6, pp. 11-24, 2001.
53. L. R. Yang, C. F. Huang, and [53] K. S. Wu, “The association among project manager's leadership style, teamwork and project success,” International Journal of Project Management, vol. 29, no. 3, pp. 258–267, 2011.
54. [54] M. S. Ozdemir and T. L. Saaty, “The unknown in decision making: What to do about it,” European Journal of Operational Research, vol. 174, no. 1, pp. 349-359, 2006.
55. [55] R. W. Saaty, “The Analytic Hierarchy Process-what it is and how it is used,” Mathematical Modelling, vol. 9, no. 3-5, pp. 161-176, 1987.
56. [56] E. H. Forman and M. A. Selly, “Decision By Objectives (How To Convince Others That You Are Right),” World Scientific Pub. Co., Petersburg, USA, 2001.
57. [57] T. L. Saaty, “Decision Making with the Analytic Hierarchy Process,” International Journal of Services Sciences, vol. 1, no. 1, pp. 83–98, 2008.
58. [58] T. L. Saaty and S.M. Ozdemir, “Why the Magic Number Seven Plus or Minus Two,” Mathematical and Computer Modelling, vol. 38, no. 3-4, pp. 233-244, 2003.
59. [59] P. T. Garker and L. G. Vargas, “The Theory of Ratio Scale Estimation: Saaty’s Analytic Hierarchy Process,” Management Science, vol. 33, no. 11, pp. 1383-1403, 1987.
60. [60] T. L. Saaty, The Analytic Hierarchy Process. McGraw-Hill, New York, 1980.
61. [61] W. A. Florac and A. D. Carleton, “Measuring the Software Process,” SEI Series in Software Engineering, 1999.
62. [62] A. L. Jacob and S. K. Pillai, “Statistical process control to improve coding and code review,” IEEE software, vol. 20, no. 3, pp. 50-55, 2003.
63. [63] W. A. Florac and A. D. Carleton, J. R. Barnard, “Statistical process control: analyzing space shuttle onboard software process,” IEEE Software, vol. 17, no. 4, pp. 97-106, 2000.
64. [64] P. Jalote and A. Saxena, “Optimum control limits for employing statistical process control in software process,” IEEE Transactions on Software Engineering, vol. 28, no. 12, pp. 1126-1134, 2002.
65. [65] Q. Wang, N. Jiang, L. Gou, X. Liu, M. Li, and Y. Wang, “BSR : A Statistic-based Approach for Establishing and Refining Software Process Performance Baseline,” pp. 585–594, 2006.
66. [66] D. Card, “Statistical process control for software?,” IEEE Software, vol. 11, no. 3, pp. 95–97, 1994.
67. [67] W. A. Florac, R. E. Park, and A. D. Carleton, “Practical software measurement: measuring for Process Management and Improvement,” Guidebook, CMU/SEI-97-HB-003, 1997.
68. [68] B. D. Günel, O. Erdoğan and A. Dikici, "Learner Control Charts: A Structured Way for Finding Common Causes with Decision Trees." Euro SPI, Dundalk, Ireland, 2013.