Monitoring and Evaluation of Transformative Service Delivery: Evidence from a Provincial Department of Transport in South Africa

Abstract

The distribution of transportation amenities is a critical component of public goods and services essential to the daily lives of citizens in South Africa. However, this distribution frequently diverges from strategic objectives, with many projects and programmes managed by the Eastern Cape Department of Transport (ECDoT) failing to meet their targets. This study quantitatively assesses the efficacy of Monitoring and Evaluation (M&E) within the South African public sector, focusing on the ECDoT. The absence of a robust and systematic M&E framework was identified, resulting in practices marked by minimal compliance, lack of transparency, inadequate communication, and insufficient focus on data quality and security. Structured questionnaires were administered to 200 ECDoT officials, with responses statistically analyzed to evaluate current M&E practices and their limitations. The findings indicate that the existing system fails to provide actionable insights or support effective decision-making, with major challenges including insufficient human capital, weak planning, poor data governance, and the lack of a multi-stakeholder approach. This study proposes a quantitative, phased M&E framework that prioritizes capacity building, strategic planning, and enhanced data security. The proposed framework emphasizes collaboration and accountability across all levels of the department to address systemic weaknesses. Transformative service delivery requires the department to adopt a well-structured and transparent M&E system supported by evidence-based decision-making.

Keywords

• Data performance
• monitoring and evaluation
• performance management
• service delivery

Dönüştürücü Hizmet Sunumuna Yönelik İzleme ve Değerlendirme: Güney Afrika’daki Bir İl Ulaştırma Departmanından Kanıtlar

Abstract

Güney Afrika’da ulaştırma olanaklarının dağılımı, vatandaşların günlük yaşamını doğrudan etkileyen kamu mal ve hizmetlerinin temel unsurlarından biridir. Ancak uygulamada bu dağılımın çoğu zaman belirlenen stratejik hedeflerle örtüşmediği; Doğu Cape Ulaştırma Departmanı (Eastern Cape Department of Transport-ECDoT) tarafından yürütülen birçok proje ve programın hedeflenen çıktılara ulaşamadığı görülmektedir. Bu çalışma, Güney Afrika kamu sektöründe İzleme ve Değerlendirme (Monitoring and Evaluation-M&E) sistemlerinin etkinliğini nicel bir yaklaşımla incelemekte ve ECDoT’yi örnek olay olarak ele almaktadır. Araştırma bulguları, kurumsal düzeyde sağlam, bütüncül ve sistematik bir M&E çerçevesinin bulunmadığını ortaya koymaktadır. Bu durum, uygulamaların büyük ölçüde asgari mevzuat uyumuna indirgenmesine, şeffaflık ve kurumsal iletişimin sınırlı kalmasına, veri kalitesi ile veri güvenliğine yeterli önemin verilmemesine yol açmaktadır. Çalışma kapsamında ECDoT bünyesinde görev yapan 200 personele yapılandırılmış anketler uygulanmış; elde edilen veriler, mevcut M&E uygulamalarının işleyişini ve sınırlılıklarını ortaya koymak amacıyla istatistiksel yöntemlerle analiz edilmiştir. Bununla birlikte, analiz sonuçları, mevcut M&E sisteminin karar alma süreçlerini destekleyecek eyleme dönük bilgi üretmede yetersiz kaldığını göstermektedir. Bu yetersizliğin temel nedenleri arasında beşerî sermaye eksikliği, zayıf stratejik planlama, yetersiz veri yönetişimi uygulamaları ve çok paydaşlı bir yaklaşımın kurumsal olarak benimsenmemiş olması yer almaktadır. Bu bağlamda çalışma, kapasite geliştirme, stratejik planlama ve veri güvenliğinin güçlendirilmesini merkeze alan, nicel temelli ve aşamalı bir M&E çerçevesi önermektedir. Önerilen çerçeve, kurumsal düzeyde iş birliği ve hesap verebilirliğin artırılmasını hedefleyerek, departman genelinde tespit edilen yapısal zayıflıkların giderilmesine katkı sağlamayı amaçlamaktadır. Sonuç olarak, dönüştürücü hizmet sunumunun hayata geçirilebilmesi, kanıta dayalı karar alma süreçleriyle desteklenen, iyi yapılandırılmış, şeffaf ve sürdürülebilir bir İzleme ve Değerlendirme sisteminin benimsenmesini zorunlu kılmaktadır.

Keywords

• Veri performansı
• izleme ve değerlendirme
• performans yönetimi
• hizmet sunumu

References

1. Abrahams, M. A. (2015). A review of the growth of monitoring and evaluation in South Africa: Monitoring and evaluation as a profession, an industry and a governance tool. African Evaluation Journal, 3(1), 1–8. https://doi.org/10.4102/aej.v3i1.142
2. Babbie, E. (2010). The practice of social research. Oxford University Press.
3. Basheka, B. C., & Byamugisha, A. K. (2015). The state of monitoring and evaluation (M&E) as a discipline in Africa: From infancy to adulthood? African Journal of Public Affairs, 8(3), 75–95. Retrieved from: https://repository.up.ac.za/handle/2263/58169
4. Bennett, N. J., & Satterfield, T. (2018). Environmental governance: A practical framework to assess the success of social equity in conservation. Conservation Biology, 32(1), 10–20. https://doi.org/10.1111/cobi.12988
5. Binnendijk, A. (2019). Results-based management in the development cooperation agencies: A review of experience. OECD DAC Working Party on Aid Evaluation. https://doi.org/10.1787/5kml5r3qbnr4-en
6. Brownson, K., & Fowler, L. (2020). Evaluating how we evaluate success: Monitoring, evaluation and adaptive management in Payments for Watershed Services programs. Land Use Policy, 94, 104505. https://doi.org/10.1016/j.landusepol.2020.104505
7. Chirau, T. J., & Blaser-Mapitsa, C. (2020). How performance management regulations shape evaluation practice in South African municipalities. Evaluation and Program Planning, 82, 101831. https://doi.org/10.1016/j.evalprogplan.2020.101831
8. Ghalem, A., Okar, C., Chroqui, R., & Semma, E. (2016). Performance: A concept to define. Retrieved from https://www.researchgate.net/publication/303880344_Performance_A_concept_to_define

9. De Silva, M., Lee, L., & Ryan, G. (2014). Using theory of change in the development, implementation and evaluation of complex health interventions. London: Centre for Global Mental Health at the London School of Hygiene & Tropical Medicine and the Mental Health Innovation Network. Retrieved from: https://www.mhinnovation.net/sites/default/files/content/document/MHIN%20ToC%20guidelines_May_2015.pdf
10. De Vos, A. S., Strydom, H., Fouche, C. B., & Delport, C. S. L. (2011). Research at the grass roots: For the social sciences and human professions. Van Schaik.
11. Deogratias, L. (2020). Monitoring and evaluation practices and challenges of HIV/AIDS projects in Tanzania: A case of WAMATA [Doctoral dissertation], The Open University of Tanzania, Dar es Salaam. Retrieved from: http://repository.out.ac.tz/2645/
12. Eastern Cape Department of Transport. (2020). Strategic plan (2020–2025). Bhisho: ECDoT. Retrieved from: https://www.ectransport.gov.za/files/uploads/Strategic_Plan_2020-2025.pdf
13. Eichhorn, P., & Towers, I. (2018). Principles of management: Efficiency and effectiveness in the private and public sector. Springer. https://doi.org/10.1007/978-3-319-70332-5
14. Epstein, M. J., & Yuthas, K. (2014). Measuring and improving social impacts: A guide for nonprofits, companies, and impact investors. Berrett-Koehler Publishers.
15. Fengu, M. (2019, November 26). MEC in hot water as PSC report reveals rot at transport department. City Press. Retrieved from: https://www.news24.com/citypress/news/mec-in-hot-water-as-psc-report-reveals-rot-at-transport-department-20191126
16. Fletcher, A. J. (2017). Applying critical realism in qualitative research: Methodology meets method. International Journal of Social Research Methodology, 20(2), 181–194. https://doi.org/10.1080/13645579.2016.1144401
17. Goldman, I., Byamugisha, A., Gounou, A., Smith, L. R., Ntakumba, S., Lubanga, T., Sossou, D., & Rot-Munstermann, K. (2018). The emergence of government evaluation systems in Africa: The case of Benin, Uganda and South Africa. African Evaluation Journal, 6(1), 1–11. https://doi.org/10.4102/aej.v6i1.253
18. Görgens, M., & Kusek, J. Z. (2010). Making monitoring and evaluation systems work: A capacity development toolkit. World Bank. https://doi.org/10.1596/978-0-8213-8186-1
19. Hennink, M. M. (2013). Focus group discussions. Oxford University Press.
20. Immordino, K. M. (2009). Organizational assessment and improvement in the public sector. CRC Press. https://doi.org/10.1201/9781420078337
21. Joshi, A., & Islam, A. M. (2018). Are public officials motivated to serve? Evidence from a large-scale survey in six African countries. World Development, 101, 1881–1891. https://doi.org/10.1016/j.worlddev.2017.09.016
22. Kaplan-Hallam, M., & Bennett, N. J. (2018). Adaptive social impact management for conservation and environmental management. Conservation Biology, 32(2), 308–317. https://doi.org/10.1111/cobi.12985
23. Kanyamuna, V. (2021a). Monitoring and evaluation systems in Africa: Public sector perspectives. International Journal of Public Administration, 44(2), 170–185. https://doi.org/10.1080/01900692.2020.1728312
24. Kanyamuna, V. (2021b). Developing institutional monitoring and evaluation systems: Insights from Zambia. African Evaluation Journal, 9(1), 90–98. https://doi.org/10.4102/aej.v9i1.499
25. Kanyamuna, V., Katowa, T., Mubita, A., Kanenga, H., Simui, F., & Kotze, D. A. (2020). Analysis of structural and organisational arrangements of monitoring and evaluation status for the public sector in Zambia. Advances in Social Sciences Research Journal, 7(6), 504–527. https://doi.org/10.14738/assrj.76.8490
26. Kools, S. (2019). Participatory monitoring and evaluation in public organisations: The missing bottom-up perspective. Evaluation, 25(1), 15–30. https://doi.org/10.1177/1356389018813708
27. Kothari, C. R. (2004). Research methodology: Methods and techniques. New Age International.
28. Kumar, R. (2019). Research methodology: A step-by-step guide for beginners. SAGE.
29. Kusek, J. Z., & Rist, R. C. (2004). Ten steps to a results-based monitoring and evaluation system: A handbook for development practitioners. World Bank. https://doi.org/10.1596/0-8213-5823-5
30. Kusters, K., Buck, L., De Graaf, M., Minang, P., van Oosten, C., & Zagt, R. (2018). Participatory planning, monitoring and evaluation of multi-stakeholder platforms in integrated landscape initiatives. Environmental Management, 62, 170–181. https://doi.org/10.1007/s00267-018-1037-3
31. Lee, C. (2020). The whole-of-government approach to public sector reform. Public Administration Review, 80(1), 203–210. https://doi.org/10.1111/puar.13120
32. Lopez-Acevedo, G., Krause, P., & Mackay, K. (Eds.). (2012). Building better policies: The nuts and bolts of monitoring and evaluation systems. World Bank. https://doi.org/10.1596/978-0-8213-8186-1
33. Mabillard, V., & Zumofen, R. (2020). Transparency and accountability: The case of public procurement practices in Switzerland. Public Works Management & Policy. Advance online publication. https://doi.org/10.1177/1087724X19898699
34. Mahlala, S. (2019). Monitoring and evaluation policy as a tool for performance enhancement in the public service: The case of the Eastern Cape Department of Health [Doctoral dissertation], North-West University, Potchefstroom. Retrieved from https://repository.nwu.ac.za/handle/10394/33015
35. Mahlala, S. (2024). Linking monitoring and evaluation to performance management in the South African public service. International Journal of Research in Business and Social Science, 13(2), 136–147. https://doi.org/10.20525/ijrbs.v13i2.2045
36. Maxwell, J. A., & Mittapalli, K. (2010). Realism as a stance for mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), SAGE handbook of mixed methods in social and behavioral research (pp. 145–167). SAGE.
37. McDonough, K., & Rodriguez, L. M. (2020). Strengthening technical capacity for evaluation: Lessons from development projects. Evaluation and Program Planning, 79, 101791. https://doi.org/10.1016/j.evalprogplan.2019.101791
38. Mertens, D. M., & Wilson, A. T. (2018). Program evaluation theory and practice. Guilford Press.
39. Miggels, A., & Rulashe, T. (2022). Organisational Change as a Tool for Transforming Governance in a Local Municipality, Nelson Mandela Bay Municipality in the Eastern Cape. Journal of Public Administration, 57(3), 508-528.
40. Mofokeng, M., & Luke, R. (2014). An investigation into the effectiveness of public entities' procurement practices. Journal of Transport and Supply Chain Management, 8(1), 1-7. https://doi.org/10.4102/jtscm.v8i1.139
41. Mogalakwe, M. (2006). The use of documentary research methods in social research. African Sociological Review, 10(1), 221-230. https://doi.org/10.4314/asr.v10i1.23293
42. Mugenda, O. M., & Mugenda, A. G. (2013). Research methods: Qualitative and quantitative approaches. African Centre for Technology Studies.
43. Nalubega, T., & Uwizeyimana, D. E. (2019). Public sector monitoring and evaluation in the Fourth Industrial Revolution: Implications for Africa. Africa’s Public Service Delivery and Performance Review, 7(1), 1-12. https://doi.org/10.4102/apsdpr.v7i1.289
44. National Education, Health and Allied Workers’ Union (NEHAWU). (2019). NEHAWU statement on the PSC findings and recommendation on the Eastern Cape Department of Transport. Retrieved from: https://www.polity.org.za/article/nehawu-statement-on-the-psc-findings-and-recommendation-on-the-eastern-cape-department-of-transport-2019-11-27
45. O’Flynn, M. (2012). Theory of Change: What is it all about? ONTRAC No. 51, INTRAC. Retrieved from: https://www.intrac.org/wpcms/wp-content/uploads/2016/06/ONTRAC-51-Theory-of-Change.pdf
46. Republic of South Africa. (1996). Constitution of the Republic of South Africa, 1996. Government Printer. Retrieved from: https://www.gov.za/documents/constitution-republic-south-africa-1996
47. Republic of South Africa. (1999). Public Finance Management Act, No. 1 of 1999. Government Printer. Retrieved from: https://www.gov.za/documents/public-finance-management-act
48. Republic of South Africa. (2003). Municipal Finance Management Act, No. 56 of 2003. Government Printer. Retrieved from: https://www.gov.za/documents/local-government-municipal-finance-management-act-0
49. Republic of South Africa. (2001). Public Service Regulations, 2001. Government Notice No. R. 1 of 5 January 2001. Retrieved from: https://www.dpsa.gov.za/dpsa2g/documents/acts%26regulations/regulations1999/PSRegulations_13_07_2012.pdf
50. Republic of South Africa. Department of Planning, Monitoring and Evaluation. (2011). National Evaluation Policy Framework. Retrieved from: https://www.dpme.gov.za/keyfocusareas/evaluationsSite/Evaluations/National%20Policy%20framework%20Nov%202019.pdf
51. Republic of South Africa. Department of Planning, Monitoring and Evaluation. (2019). Medium-Term Strategic Framework 2019–2024. Retrieved from: https://www.dpme.gov.za/keyfocusareas/outcomesSite/MTSF_2019_2024/2019-2024%20MTSF%20Comprehensive%20Document.pdf
52. Republic of South Africa. National Planning Commission. (2012). National Development Plan: Vision for 2030. Government Printer. Retrieved from: https://www.gov.za/sites/default/files/gcis_document/201409/devplan2.pdf
53. Republic of South Africa. Department of Public Service and Administration. (1997). White Paper on Transforming Public Service Delivery (Batho Pele White Paper). Government Printer. Retrieved from: https://www.gov.za/documents/white-papers/transforming-public-service-delivery-white-paper-batho-pele-white-paper-01
54. Republic of South Africa. National Treasury. (2005). Treasury Regulations for departments, trading entities, constitutional institutions and public entities. Retrieved from: https://www.treasury.gov.za/legislation/pfma/act.pdf
55. Republic of South Africa. National Treasury. (2020). Framework for Strategic Plans and Annual Performance Plans. Retrieved from: https://www.treasury.gov.za/publications/guidelines/2020%20Framework%20for%20Strategic%20Plans%20and%20Annual%20Performance%20Plans.pdf
56. Rickwood, D., Paraskakis, M., Quin, D., Hobbs, N., Ryall, V., Trethowan, J., & McGorry, P. (2019). Australia’s innovation in youth mental health care: The headspace centre model. Early Intervention in Psychiatry, 13(1), 159–166. https://doi.org/10.1111/eip.12740
57. Rulashe, T., & Ijeoma, E. O. (2022). An exploration of public accountability and service delivery at the Buffalo City Metropolitan Municipality in the Eastern Cape province, South Africa. Africa’s Public Service Delivery and Performance Review, 10(1), 535. http://dx.doi.org/10.4102/apsdpr.v10i1.535
58. Simister, N. (2017). Outputs, outcomes and impact. INTRAC. Retrieved from: https://www.intrac.org/resources/outputs-outcomes-impact/
59. Taplin, D., Clark, H., Collins, E., & Colby, D. L. (2013). Theory of Change Technical Papers: A Series of Papers to Support Development of Theories of Change Based on Practice in the Field. ActKnowledge. Retrieved from: https://www.theoryofchange.org/wp-content/uploads/toco_library/pdf/ToC-Tech-Papers.pdf
60. Thornhill, C. (2012). South African public administration and management. Van Schaik.
61. Tracy, S. J. (2019). Qualitative research methods: Collecting evidence, crafting analysis, communicating impact. John Wiley & Sons.
62. United Nations Development Group (UNDG). (2011). Theory of Change UNDAF Companion Guidance. United Nations. Retrieved from: https://unsdg.un.org/sites/default/files/UNDG-ToC-UNDAF-Companion-Piece.pdf
63. Van der Knaap, P. (2004). Theory-based evaluation and learning: Possibilities and challenges. Evaluation, 10(1), 16-34. https://doi.org/10.1177/1356389004043139
64. Weiss, C. H. (1995). Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. In J. P. Connell, A. C. Kubisch, L. B. Schorr, & C. H. Weiss (Eds.), New approaches to evaluating community initiatives: Concepts, methods, and contexts (pp. 65-92). Aspen Institute.