Scientific article review platform using generative artificial intelligence to streamline the peer review process

Year 2025, Volume: 12 Issue: 4, 983 - 994

German Cuaya-simbro , Serguei Drago Domínguez Ruiz

https://doi.org/10.21449/ijate.1589451

Abstract

This study introduces a novel Generative Artificial Intelligence (GAI) platform designed to streamline the peer review process. By analyzing a case study of 10 scientific articles, we demonstrate that GAI effectively evaluates article quality and pinpoints specific areas requiring improvement. Our platform achieves an average similarity of 63.6% with human reviewers, enabling the automation of routine evaluation tasks while enhancing both efficiency and objectivity. By drawing on recent generative AI benchmarks across research support, educational assessments, reviewer matching, and large-scale application studies, we demonstrate a focused, practically validated solution that not only aligns with but slightly outperforms general GAI performance levels, offering a transformative approach to real-world manuscript evaluation.

Keywords

Web platform , Intelligent agent , Automatic evaluation , Improving objectivity

Project Number

19506.24-PD

Hakem değerlendirme sürecini hızlandırmak için üretken yapay zeka kullanan bilimsel makale inceleme platformu

Abstract

Bu çalışmada, hakem değerlendirme sürecini kolaylaştırmak için üretken yapay zeka (GAI) kullanan yeni bir platform tanıtılmaktadır. GAI'nin makale kalitesini etkili bir şekilde değerlendirme ve iyileştirme için kilit alanları belirleme kabiliyetini, 10 bilimsel makaleden oluşan bir vaka çalışmasını analiz ederek gösteriyoruz. Bulgularımız, GAI'nin değerlendirmeleri ile insan hakemlerin değerlendirmeleri arasında %63,6'lık bir benzerlik ortalamasıyla güçlü bir korelasyon olduğunu ortaya koyuyor ve GAI'nin ilk incelemeler için değerli bir araç olarak hizmet edebileceğini gösteriyor. Bu platform, rutin görevleri otomatikleştirerek ve objektif geri bildirim sağlayarak, geleneksel akran değerlendirmesinin zaman kısıtlamaları ve öznellik gibi zorluklarını ele almaktadır. Bu ilerleme, bilimsel araştırma değerlendirmesinin verimliliğini ve güvenilirliğini önemli ölçüde artırma potansiyeline sahiptir.

Keywords

Web platformu , Akıllı ajan , Otomatik değerlendirme , Nesnelliğin iyileştirilmesi

Project Number

19506.24-PD

References

• Aries Systems. (n.d.). Editorial Manager [Cloud-based manuscript submission and peer review system]. Aries Systems Corporation. https://www.ariessys.com/solutions/editorial-manager/
• ChatPDF GmbH. (n.d.). ChatPDF [AI-powered app]. https://www.chatpdf.com/
• Checco, A., Bracciale, L., Loreti, P., Pinfield, S., & Bianchi, G. (2021). AI-assisted peer review. Humanities and Social Sciences Communications, 8, Article 25. https://doi.org/10.1057/s41599-020-00703-8
• Cohen, J.F., & Moher, D. (2025). Generative artificial intelligence and academic writing: Friend or foe? Journal of Clinical Epidemiology, 179, Article 111646. https://doi.org/10.1016/j.jclinepi.2024.111646
• Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (Version 2). arXiv. https://doi.org/10.48550/ARXIV.1810.04805
• Fischer, I., Sweeney, S., Lucas, M., & Gupta, N. (2024). Making sense of generative AI for assessments: Contrasting student claims and assessor evaluations. The International Journal of Management Education, 22(3), Article 101081. https://doi.org/10.1016/j.ijme.2024.101081
• Google. (n.d.). Gemini [Large language model]. Google AI. https://gemini.google.com
• Gozalo-Brizuela, R., & Merchan, E.E.G. (2024). A Survey of Generative AI Applications. Journal of Computer Science, 20(8), 801-818. https://doi.org/10.3844/jcssp.2024.801.818
• Hitch, D., Richards, K., Gupta, A., & Thanekar, U. (2025). The ethical implications of using AI in qualitative research. In A. Gupta (Ed.), Artificial intelligence (AI) in social research (pp. 137-149). CABI Publishing. https://doi.org/10.1079/9781800626607.0013
• Hossain, M.M. (2024). Using ChatGPT and other forms of generative AI in systematic reviews: Challenges and opportunities. Journal of Medical Imaging and Radiation Sciences, 55(1), 11-12. https://doi.org/10.1016/j.jmir.2023.11.005
• Jan, Z., Ahamed, F., Mayer, W., Patel, N., Grossmann, G., Stumptner, M., & Kuusk, A. (2023). Artificial intelligence for industry 4.0: Systematic review of applications, challenges, and opportunities. Expert Systems with Applications, 216, Article 119456. https://doi.org/10.1016/j.eswa.2022.119456
• Joe Devanny, H.D., & Grossfeld, E. (2023). Generative AI and intelligence assessment. The RUSI Journal, 168(7), 16-25. https://doi.org/10.1080/03071847.2023.2286775
• Morande, S. (2023). Benchmarking Generative AI: A comparative evaluation and practical guidelines for responsible integration into academic research. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4571867
• Open Journal Systems. (n.d.). OJS (Version 3.1) [Open source publishing platform]. https://openjournalsystems.com/
• Saranya, A., & Subhashini, R. (2023). A systematic review of explainable artificial intelligence models and applications: Recent developments and future trends. Decision Analytics Journal, 7, Article 100230. https://doi.org/10.1016/j.dajour.2023.100230
• Sengar, S.S., Hasan, A.B., Kumar, S., & Carroll, F. (2024). Generative artificial intelligence: A systematic review and applications. Multimedia Tools and Applications, 84, 23661-23700. https://doi.org/10.1007/s11042-024-20016-1
• Sharly. (n.d.). Sharly AI [AI tool]. VOX AI Inc. https://sharly.ai/
• Silverchair Support. (n.d.). ScholarOne Manuscripts [Workflow management system]. Silverchair. https://www.silverchair.com/products/scholarone-manuscripts/