A Literature Review on Maturity Assessment Tools Used in System Development Projects

Abstract

The system development projects in the defense industry differ from the ones in other sectors with their big budgets, complexity, and provision of important capability. The organizations taking part in the acquisition process of such projects are in search of developing various kinds of mechanisms, processes, and methodologies to implement projects with better performance but with less cost and less lost time while minimizing their cost, time, and performance risk in the rapidly changing competitive environment. One of the solutions developed in this direction is maturity assessment tools. In this study, how the technology/system maturity assessment tools are used, what their benefits and limitations are, and how the system risk is calculated have been analyzed. It has been determined that the current maturity assessment tools are inadequate for calculating the system risk and that there is a need to develop a new tool/methodology that can compute the system risk. In addition, an assessment on the status of the usage of maturity assessment tools in Turkey has been made.

Keywords

• Technology/System Readiness Level, Maturity Assessment Tools, Technology/System Risk.

Sistem Geliştirme Projelerinde Kullanılan Olgunluk Değerlendirme Araçları Üzerine Bir Literatür Araştırması

Abstract

Savunma alanındaki sistem geliştirme projeleri büyük bütçeleri, önemli yetenek kazandırmaları ve karmaşıklığı ile diğer projelerden farklılık gösterirler. Bu projelerin tedarik sürecinde yer alan organizasyonlar, hızla değişen rekabet ortamında, maliyet, zaman ve performans risklerini en aza indirgeyerek daha az kayıp süre ve daha düşük maliyetle daha yüksek performansa sahip projeleri hayata geçirebilecek mekanizmaların, süreçlerin veya metodolojilerin arayışı içindedirler. Bu doğrultuda geliştirilmiş çözümlerden bir tanesi de olgunluk değerlendirme araçlarıdır. Bu çalışmada, literatürde kullanılan teknoloji/sistem olgunluk değerlendirme araçlarının neler olduğu, metodolojileri, nasıl kullanıldıkları, faydaları ve sınırlılıkları ile sistem riskini hesaplama yönü incelenmiştir. Özellikle sistem geliştirme projelerinde kullanılan olgunluk değerlendirme araçlarının sistem riskini nasıl hesapladığına, hesaplamanın güçlü ve zayıf yönlerine cevap aranmıştır. Olgunluk değerlendirme araçlarının sistem riskini hesaplamada yetersiz oldukları tespit edilmiş ve sistem riskinin hesaplanmasına yönelik yeni bir araç ve/veya metodoloji geliştirilmesi ihtiyacı ortaya konmuştur. Ayrıca, Türkiye’de olgunluk değerlendirme araçlarının kullanım durumuyla ilgili bir değerlendirme yapılmıştır.

Keywords

• Teknoloji/Sistem Hazırlık Seviyesi, Olgunluk Değerlendirme Araçları, Teknoloji/Sistem Riski.

References

1. Altunok, T. ve Çakmak, T., (2010). A Technology Readiness Levels (TRLs) Calculator Software for Systems Engineering and Technology Management Tool. Advances in Enginerring Software.
2. Azizian, N., Sarkani, S. ve Mazzuchi, T. (2009). A Comprehensive Review and Analysis of Maturity Assessment Approaches for Improved Decision Support to Achieve Efficient Defense Acquisition. Proceedings of the World Congress on Engineering and Computer Science, II, San Francisco, USA, Babaçoglu S., Akgün İ. ve Altın Kayhan A. Sistem Geliştirme Projelerinde Teknik Risk Analizi. (Yayımlanmamış Doktora Tezi), KHO Savunma Bilimleri Enstitüsü, 2014.
3. Bilbro, W.J. (2007a). TRL Calculator. JB Consulting International. www.jbconsultinginternational.com adresinden alınmıştır.
4. Bilbro, W.J. (2007b). Systematic Assessment of the Program/Project Impacts of Technological Advancement and Insertion Revision A. JB Consulting International, 10.
5. Bilbro, W.J. (2014). Asessing System Readiness- Is it ready for prime time? www.jbconsultinginternational.com adresinden alınmıştır.
6. Cornford, S. ve Sarsfield, L. (2004). Quantitative Methods for Maturing and Infusing Advanced Spacecraft Technology. IEEE Aerospace Conference Proceedings.
7. Cueller, R. ve Sauser, B. (2009). Dynamic Multipoint Optimization Application to Corporate Portfolio Management. Acquisition Research Symposium, Monterey, CA.
8. Çetindamar, D. ve Günsel, A. (2009). Teknoloji Yetenek Kapasitesinin Değerlendirmesi: Nedir ve Nasıl Uygulanır?, TÜSİAD-Sabancı Üniversitesi Rekabet Forumu, İstanbul.

9. Defence Science and Technology Organization. (2010). Technical Risk Assessment Handbook. http://www.dsto.defence.gov.au/attachments/ Technical-Risk-Assessment-Handbook 2.pdf adresinden alınmıştır.
10. Demirci, E.A. ve Ersoy, N.F. (2008). Technology Readiness for Innovative HighTech Products: How Consumers Perceive and Adopt New Technologies. Department of Defense. (2011). Work Breakdown Structures for Defense Material Items. http:// www.navair.navy.mil/nawctsd/Resources/Library/Acqguide/ MIL-STD.881C adresinden alınmıştır.
11. Dion-Schwarz, C. (2008). How the Department of Defense Uses Technology Readiness Levels. Office of the Under Secretary of Defense for Acquisition, Technology and Logistics.
12. DoD Instrucitons 5000.02 (2008). Operation of the Defense Acquisition System.
13. DoD Technology Readiness Assessment (TRA) Deskbook. (2005).
14. DoD Technology Readiness Assessment (TRA) Guidance. (2011). Assistant Secretary of Defense for Research and Engineering. http://www.acq.osd.mi/ddre/publications/ docs/ TRA2011.pdf adresinden alınmıştır.
15. Fernandez, A.J. (2010). Contextual Role of TRLs and MRLs in Technology Management. Sandia National Laboratories, United States of America.
16. Government Accountability Office. (2008). Defense Acquisitions Assessments of Selected Weapon Programs. http://www.gao.gov.new.items/d08467sp.pdf adresinden alınmıştır.
17. Kujawski, E. (2010). The Trouble with The System Readiness Level (SRL) Index For Managing The Acquisition Of Defense Systems. National Defense Industrial Association 13th Annual Systems Engineering Conference, San Diego, 10.
18. Leveson, N. (2012). Drawbacks in Using the Term “System of Systems”. Technical Report, System Safety Research Lab., MIT.
19. Magnaye R.B., Sauser B.J. ve Ramirez-Marquez, J.E. (2010). System Development Planning Using Readiness Levels in a Cost of Development Minimization Model. System Engneering, vol. 13, 4.
20. Mankins, J. C. (1995). Technology Readiness Levels. A White Paper, Advanced Concepts Office, Office of Space Access and Technology, NASA.
21. McConkie, E., Mazzuchi, T.A., Sarkani, S. ve Marchette, D. (2012). Mathematical Properties of System Readiness Levels. System Engineering 2012, Wiley Periodicals.
22. Nolte, W., (2003). Technology Readiness Calculator. NDIA System Engineering Conference. www.dtic.mil/ndia/2003systems/nolte2.pdf adresinden alınmıştır.
23. OSD Manufacturing Technology Program. (2011). Manufacturing Readiness Level (MRL) Deskbook. http://www.dodmrl.com/MRL_Deskbook_V2.pdf adresinden alınmıştır.
24. Özer, P.S., Eriş, E.D., Özmen, Ö.N.T. (2011). Teknolojiye Hazırlık Düzeyinin Belirlenmesine Yönelik Bir Ölçek Uyarlama Çalışması. Dokuz Eylül Üniversitesi İşletme Fakültesi Dergisi, Cilt 12, Sayı 2, 321-333.
25. Özkil, A. (2009). Sistem Mühendisliği Konfigürasyon Yönetimi, Basılmamış Ders Notları, Ankara.
26. Parasuraman, A. (2000). Technology Readiness Index (TRI): A Multiple-Item Scale to Measure Readiness to Embrace New Technologies. Journal of Service Research, Vol.2 (4): 307-320.
27. Ramirez-Marquez, J.E. ve Sauser, B.J. (2009). System Development Planning via System Maturity Optimization. IEEE Transactions On Engineering Management, III, 8.
28. Sauser, B.J. ve Forbes, E. (2009). Defining an Integration Readiness Level for Defense Acquisition. International Symposium of the International Council on Systems Engineering, Singapore, 7.
29. Sauser, B.J., Ramirez-Marquez, J.E. ve Henry, D. (2008). System Maturity Index for The System Engineering Life Cycle. International Systems Engineering, III, USA.
30. Schwartz, M. (2013). Defense Acquisitions: How DOD Acquires Weapon Systems and Recent Efforts to Reform the Process. CRS Report for Congress.
31. Smith, J. (2004). An Alternative to Technology Readiness Levels for NonDevelopmental Item (NDI) Software, Integration of Software-Intensive Systems Initiative. Carnegie Mellon Software Engineering Institute .
32. Tan, W., Sauser, B.J., ve Ramirez-Marquez, J.E. (2009). Monte-Carlo Simulation Approach For System Readiness Level Estimation. Int.Symp.Int.Counc.Syst.Eng., Singapore.
33. Tan, W., Sauser, B.J. ve Ramirez-Marquez, J.E. (2011). Analyzing Component Importance in Multifunction Multicapability Systems Developmental Maturity Assessment. IEEE Transactions On Engineering Management, Vol. 58, 5.
34. Tetlay, A. ve John, P. (2009). Determining the Lines of System Maturity, System Readiness and Capability Readiness in the System Development Lifecycle. 7th Annual Conference on Systems Engineering Research.
35. U.S.Air Force Technology Development Team. (2008). Risk Identification: Integration & Ilities (RI3) Guidebook. https://acc.dau.mil/ CommunityBrowser.aspx?id=318289 adresinden alınmıştır.