The Evolution of Code Review: From Traditional Methods to AI-Powered Approaches

Abstract

Code review is an important process in the software development lifecycle to ensure high-quality, maintainable, and efficient code. Over time, automated code review systems have evolved from manual methods to include advanced techniques such as dynamic analysis, machine learning, deep learning, and, more recently, generative artificial intelligence. These techniques use natural language processing and predictive models to provide faster and more accurate code reviews with intelligent, context-sensitive feedback. This survey explores the evolution of automated code review, starting with traditional methods and moving toward the latest AI-powered approaches. We discuss the methods used, the challenges faced, and the benefits these systems offer. In addition, we highlight future trends and examine how AI is transforming code review processes to meet the needs of modern software development.

Keywords

• Automated code review
• Code review evaluation metrics
• Code review datasets
• Code quality
• Generative AI

References

1. Almeida, Y., Albuquerque, D., Filho, E. D., Muniz, F., de Farias Santos, K., Perkusich, M., Almeida, H., & Perkusich, A. (2024). AICodeReview: Advancing code quality with AI-enhanced reviews. SoftwareX, 26, 101677. https://doi.org/10.1016/j.softx.2024.101677
2. Azeem, M. I., Palomba, F., Shi, L., & Wang, Q. (2019). Machine learning techniques for code smell detection: A systematic literature review and meta-analysis. Information and Software Technology, 108, 115–138. https://doi.org/10.1016/j.infsof.2018.12.009
3. Bacchelli, A., & Bird, C. (2013). Expectations, outcomes, and challenges of modern code review. Proceedings of the 35th International Conference on Software Engineering (ICSE) (pp. 712–721). IEEE.
4. Ball, T., & Larus, J. R. (1994). Optimally profiling and tracing programs. ACM Transactions on Programming Languages and Systems, 16(4), 1319–1360.
5. Banerjee, S., & Lavie, A. (2005). METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization (pp. 65–72). Association for Computational Linguistics.
6. Bani-Salameh, H., Sallam, M., & Alshboul, B. (2021). A deep-learning-based bug priority detection using RNN-LSTM neural networks. e-Informatica Software Engineering Journal, 15(1), 29–45.
7. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., ... Amodei, D. (2020). Language models are few-shot learners. Proceedings of the 34th International Conference on Neural Information Processing Systems (NeurIPS) (pp. 1877–1901). Curran Associates Inc.
8. Bughin, J. (2024). The role of firm AI capabilities in generative AI-pair coding. Journal of Decision Systems, 1–22. https://doi.org/10.1080/12460125.2024.2428187

9. Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H. P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., ... Zaremba, W. (2021). Evaluating large language models trained on code. arXiv. https://arxiv.org/abs/2107.03374
10. Chouchen, M., Ouni, A., Olongo, J., & Mkaouer, M. W. (2023). Learning to predict code review completion time in modern code review. Empirical Software Engineering, 28, 82. https://doi.org/10.1007/s10664-023-10331-w
11. Cutler, A. & Cutler, D. & Stevens, J.. (2011). Random Forests. 10.1007/978-1-4419-9326-7_5.
12. Davila, N., & Nunes, I. (2021). A systematic literature review and taxonomy of modern code review. Journal of Systems and Software, 177, 110951. https://doi.org/10.1016/j.jss.2021.110951
13. Ebert, F., Castor, F., Novielli, N., & Serebrenik, A. (2019). Confusion in code reviews: Reasons, impacts, and coping strategies. Proceedings of IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER) (pp. 49–60). IEEE.
14. Fagan, M. E. (1976). Design and code inspections to reduce errors in program development. IBM Systems Journal, 15(3), 182–211.
15. Fagan, M. E. (1986). Advances in software inspections. IEEE Transactions on Software Engineering, SE-12(7), 744–751.
16. Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., & Zhou, M. (2020). CodeBERT: A pre-trained model for programming and natural languages. T. Cohn, Y. He, & Y. Liu (Eds.), Findings of the Association for Computational Linguistics: EMNLP 2020 (pp. 1536–1547). Association for Computational Linguistics.
17. Ganesan, K. A. (2018). ROUGE 2.0: Updated and improved measures for evaluation of summarization tasks. arXiv. https://arxiv.org/abs/1803.01937
18. Gerrit Code Review. (2025). Gerrit code review. https://www.gerritcodereview.com
19. GitHub. (2025). GitHub REST API documentation. https://docs.github.com/en/rest
20. Guo, D., Ren, S., Lu, S., Feng, Z., Tang, D., Liu, S., Zhou, L., Duan, N., Yin, J., Jiang, D., & Zhou, M. (2020). GraphCodeBERT: Pre-training code representations with data flow. arXiv. https://arxiv.org/abs/2009.08366
21. Hellendoorn, V. J., Tsay, J., Mukherjee, M., & Hirzel, M. (2021). Towards automating code review at scale. Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (pp. 1479–1482). ACM.
22. Hong, Y., Tantithamthavorn, C., Thongtanunam, P., & Aleti, A. (2022). CommentFinder: A simpler, faster, more accurate code review comments recommendation. Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (pp. 507–519). ACM.
23. Johnson, S. C. (1978). Lint, a C program checker. https://wolfram.schneider.org/bsd/7thEdManVol2/lint/lint.pdf
24. Lavie, A., & Denkowski, M. (2009). The METEOR metric for automatic evaluation of machine translation. Machine Translation, 23, 105–115.
25. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521, 436–444. https://doi.org/10.1038/nature14539
26. Li, Z., Lu, S., Guo, D., Duan, N., Jannu, S., Jenks, G., Majumder, D., Green, J., Svyatkovskiy, A., Fu, S., & Sundaresan, N. (2022). Automating code review activities by large-scale pre-training. arXiv. https://arxiv.org/abs/2203.09095
27. Lipton, Z. C., Elkan, C. P., & Narayanaswamy, B. (2014). Optimal thresholding of classifiers to maximize F1 score. T. Calders, F. Esposito, E. Hüllermeier, & R. Meo (Eds.), Machine Learning and Knowledge Discovery in Databases (Cilt 8725, pp. 225–239). Springer.
28. Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C. B., Drain, D., Jiang, D., Tang, D., Li, G., Zhou, L., Shou, L., Zhou, L., Tufano, M., Gong, M., Zhou, M., Duan, N., Sundaresan, N., ... Liu, S. (2021). CodeXGLUE: A machine learning benchmark dataset for code understanding and generation. arXiv. https://arxiv.org/abs/2102.04664
29. McIntosh, S., Kamei, Y., Adams, B., & Hassan, A. E. (2014). The impact of code review coverage and code review participation on software quality: A case study of the Qt, VTK, and ITK projects. Proceedings of the 11th Working Conference on Mining Software Repositories (MSR) (pp. 192–201). ACM.
30. Papineni, K., Roukos, S., Ward, T., & Zhu, W. J. (2002). BLEU: A method for automatic evaluation of machine translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL) (pp. 311–318).
31. Pornprasit, C., & Tantithamthavorn, C. K. (2024). Fine-tuning and prompt engineering for large language models-based code review automation. Information and Software Technology, 175, 107523.
32. Pornprasit, C., Tantithamthavorn, C. K., Thongtanunam, P., & Chen, C. (2023). D-ACT: Towards diff-aware code transformation for code review under a time-wise evaluation. Proceedings of the IEEE 30th International Conference on Software Analysis, Evolution and Reengineering (SANER) (pp. 296–307). IEEE.
33. Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding with unsupervised learning. OpenAI. https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf
34. Rani, P., Blasi, A., Stulova, N., Panichella, S., Gorla, A., & Nierstrasz, O. (2023). A decade of code comment quality assessment: A systematic literature review, Journal of Systems and Software, Volume 195, 2023, 111515, ISSN 0164-1212, https://doi.org/10.1016/j.jss.2022.111515.
35. Rasheed, Z., Sami, M. A., Waseem, M., Kemell, K. K., Wang, X., Nguyen, A., Systä, K., & Abrahamsson, P. (2024). AI-powered code review with LLMs: Early results. arXiv. https://arxiv.org/abs/2404.18496
36. Reiter, E. (2018). A structured review of the validity of BLEU. Computational Linguistics, 44(3), 393–401.
37. Rennie, J. D. M., Shih, L., Teevan, J., & Karger, D. R. (2003). Tackling the poor assumptions of naive Bayes text classifiers. Proceedings of the 20th International Conference on Machine Learning (ICML) (pp. 616–623).
38. Sadowski, C., Söderberg, E., Church, L., Sipko, M., & Bacchelli, A. (2018). Modern code review: A case study at Google. Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) (pp. 181–190). IEEE.
39. Shi, S. T., Li, M., Lo, D., Thung, F., & Huo, X. (2019). Automatic code review by learning the revision of source code. Proceedings of the 33rd AAAI Conference on Artificial Intelligence (pp. 4910–4917). AAAI Press.
40. Tufano, R., Dabic, O., Mastropaolo, A., Ciniselli, M., & Bavota, G. (2024). Code review automation: Strengths and weaknesses of the state of the art. IEEE Transactions on Software Engineering, 50(2), 338–353.
41. Tufano, R., Pascarella, L., Tufano, M., Poshyvanyk, D., & Bavota, G. (2021). Towards automating code review activities. Proceedings of the 43rd IEEE/ACM International Conference on Software Engineering (ICSE) (pp. 163–174). IEEE.
42. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. arXiv. https://arxiv.org/abs/1706.03762
43. Wu, Z., Zhang, H., & Liu, J. (2014). A fuzzy support vector machine algorithm for classification based on a novel PIM fuzzy clustering method. Neurocomputing, 125, 119–124.
44. Zanaty, F. E., Hirao, T., McIntosh, S., Ihara, A., & Matsumoto, K. (2018). An empirical study of design discussions in code review. Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) (pp. 1–10). ACM.
45. Zhou, X., Kim, K., Xu, B., Han, D., He, J., & Lo, D. (2023). Generation-based code review automation: How far are we? Proceedings of the IEEE/ACM 31st International Conference on Program Comprehension (ICPC) (pp. 215–226). IEEE.