A Failure-Order Resilience Signature for Self-Healing AI Inference Services: A Renewal-Based Framework

Abstract

AI inference services used in production environments must not only produce highly accurate predictions but also continue providing service in the presence of hardware, software, network, and orchestration failures. Traditional reliability measures mostly focus on time-based indicators but do not directly show how many failure events a self-healing service can tolerate before its first complete service interruption. In this study, the failure-order resilience signature (FORS) is defined to measure the event-based resilience of a self-healing AI inference service running on two active replicas. In the homogeneous model, replica times to failure are assumed to be exponentially distributed, recovery times are assumed to follow a general distribution, and the probability of a successful recovery cycle is expressed through the Laplace–Stieltjes transform of the recovery-time distribution. It is shown that FORS has a shifted geometric distribution under exponential recovery, and extensions are developed for heterogeneous replicas, survivor overload, time-based reliability, and recovery-time variability. Monte Carlo simulations based on one million service lifetimes per scenario validated the analytical results, with relative errors of less than approximately 0.10% in all scenarios. The mean times to first outage for systems with the same FORS value but different absolute rate scales were 50, 25, and 12.5 time units. In the survivor overload scenario, the expected failure order at the first outage decreased to 3.476, and the mean time to first outage decreased to 19.708 time units. In the production-trace application using the Alibaba Cluster Trace GPU v2025 dataset, 23,871 inference instances belonging to 156 services were examined. Among 21 service layers satisfying the exact two-replica criterion, ten completed operational races ended in a 1→2 transition, whereas six ended in a 1→0 zero-instance transition. The operational cycle-success probability was estimated at 0.625, the expected failure order at the first outage at 3.667, and the probability of not experiencing a zero-instance outage over approximately 31 days at 0.682. Because the dataset does not label the causes of instance deletion, these empirical results are based on instance lifecycle transitions rather than confirmed failure events. The proposed FORS is not a distribution-free replacement for the classical system signature, but rather a complementary, recovery-aware measure of AI service resilience that accounts for the recovery mechanism and the number of failures tolerated before the first complete interruption.

Keywords

• AI inference services
• Self-healing systems
• Failure-order resilience
• Production traces
• Replica reliability
• Renewal signature

Ethical Statement

Ethics committee approval was not required for this study because of there was no study on animals or humans.

References

1. Alibaba Group. (2025). Alibaba Cluster Trace Program: cluster-trace-gpu-v2025 [Data set]. GitHub. https://github.com/alibaba/clusterdata/tree/master/cluster-trace-gpu-v2025
2. Al-Said Ahmad, A., Al-Qora’n, L. F., & Zayed, A. (2024). Exploring the impact of chaos engineering with various user loads on cloud native applications: An exploratory empirical study. Computing, 106, 2389–2425. https://doi.org/10.1007/s00607-024-01292-z
3. Chaos Mesh. (2026). Simulate pod faults. Retrieved August 8, 2026, from https://chaos-mesh.org/docs/simulate-pod-chaos-on-kubernetes/
4. Chen, Z., Goudarzi, M., & Nadjaran Toosi, A. (2025). Resilience evaluation of Kubernetes in cloud-edge environments via failure injection [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2507.16109
5. Coolen, F. P. A., & Coolen-Maturi, T. (2013). Generalizing the signature to systems with multiple types of components. In W. Zamojski, J. Mazurkiewicz, J. Sugier, T. Walkowiak, & J. Kacprzyk (Eds.), Complex systems and dependability (pp. 115–130). Springer. https://doi.org/10.1007/978-3-642-30662-4_8
6. Cui, S., Patke, A., Nguyen, H., Ranjan, A., Chen, Z., Cao, P., Bauer, G., Bode, B., Di Martino, C., Jha, S., Narayanaswami, C., Sow, D., Kalbarczyk, Z. T., & Iyer, R. K. (2025). Story of two GPUs: Characterizing the resilience of Hopper H100 and Ampere A100 GPUs. In Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, SC 2025 (pp. 1145–1164). Association for Computing Machinery. https://doi.org/10.1145/3712285.3759821
7. Dikmen, O., & Kulaç, S. (2016). An overview of 5G technology. 4th International Symposium on Innovative Technologies in Engineering and Science (ISITES2016), Alanya/Antalya, Türkiye, 774–782.
8. Dikmen, O., & Kulaç, S. (2018). Efficiency of massive MIMO for future wireless communication systems. Journal of Engineering Research and Applied Science, 7(2), 944–952.

9. Dikmen, O. (2024). Performance analysis and simulation of IRS-aided wireless networks communication. Symmetry, 16(2), 254. https://doi.org/10.3390/sym16020254
10. Ding, W., Da, G., & Zhao, P. (2026). Generalized survival signature for repairable systems: Optimal allocation of minimal repairs. Naval Research Logistics (NRL) 73(6), 1026-1043. https://doi.org/10.1002/nav.70067
11. Eryılmaz, S. (2014). Computing reliability indices of repairable systems via signature. Journal of Computational and Applied Mathematics, 260, 229–235. https://doi.org/10.1016/j.cam.2013.09.023
12. Franko, C., & Tütüncü, G. Y. (2016). Signature-based reliability analysis of repairable weighted k-out-of-n:G systems. IEEE Transactions on Reliability, 65(2), 843–850. https://doi.org/10.1109/TR.2015.2499959
13. Gemci, Ö. B., & Dikmen, O. (2026a). MoAUXNet: An attention-enhanced mixture-of-experts approach for low-SNR modulation classification. 2026 34th Signal Processing and Communications Applications Conference (SIU). https://doi.org/10.1109/SIU71813.2026.11636714
14. Gemci, Ö. B., & Dikmen, O. (2026b). SCALNet: A lightweight attention-enhanced convolutional network for robust modulation classification using constellation diagrams. Signal, Image and Video Processing, 20, 71. https://doi.org/10.1007/s11760-026-05126-7
15. Gökdere, G., & Ng, H. K. T. (2022). Time-dependent reliability analysis for repairable consecutive-k-out-of-n:F system. Statistical Theory and Related Fields, 6(2), 139–147. https://doi.org/10.1080/24754269.2021.1971489
16. Gürcan, M., Güral, Y., & Gökdere, G. (2018). Analysis of repairable k-out-of-n system model using inter-arrival failure times. ITM Web of Conferences, 22, 01054. https://doi.org/10.1051/itmconf/20182201054
17. Kochar, S., Mukerjee, H., & Samaniego, F. J. (1999). The “signature” of a coherent system and its application to comparisons among systems. Naval Research Logistics, 46(5), 507–523.
18. Kokolis, A., Kuchnik, M., Hoffman, J., Kumar, A., Malani, P., Ma, F., DeVito, Z., Sengupta, S., Saladi, K., & Wu, C.-J. (2025). Revisiting reliability in large-scale machine learning research clusters. In 2025 IEEE International Symposium on High Performance Computer Architecture (HPCA) (pp. 1259–1274). IEEE. https://doi.org/10.1109/HPCA61900.2025.00096
19. KServe. (2026). KServe. Retrieved August 8, 2026, from https://kserve.github.io/website/
20. Kubernetes. (2026). Kubernetes self-healing. Retrieved August 8, 2026, from https://kubernetes.io/docs/concepts/architecture/self-healing/
21. Navarro, J., Samaniego, F. J., Balakrishnan, N., & Bhattacharya, D. (2008). On the application and extension of system signatures in engineering reliability. Naval Research Logistics, 55(4), 313–327.
22. Samaniego, F. J. (1985). On closure of the IFR class under formation of coherent systems. IEEE Transactions on Reliability, 34(1), 69–72.
23. Samaniego, F. J. (2007). System signatures and their applications in engineering reliability. Springer.
24. Vayghan, L. A., Saied, M. A., Toeroe, M., & Khendek, F. (2021). A Kubernetes controller for managing the availability of elastic microservice based stateful applications. Journal of Systems and Software, 175, 110924. https://doi.org/10.1016/j.jss.2021.110924
25. Yang, L., Wang, Y., Yu, Y., Weng, Q., Dong, J., Liu, K., Zhang, C., Zi, Y., Li, H., Zhang, Z., Wang, N., Dong, Y., Zheng, M., Xi, L., Lu, X., Ye, L., Yang, G., Fu, B., Lan, T., Zhang, L., Qu, L., & Wang, W. (2025). GPU-disaggregated serving for deep learning recommendation models at scale. In Proceedings of the 22nd USENIX Symposium on Networked Systems Design and Implementation (NSDI 25) (pp. 847–863).