Scientific article review platform using generative artificial intelligence to streamline the peer review process

Abstract

This study introduces a novel Generative Artificial Intelligence (GAI) platform designed to streamline the peer review process. By analyzing a case study of 10 scientific articles, we demonstrate that GAI effectively evaluates article quality and pinpoints specific areas requiring improvement. Our platform achieves an average similarity of 63.6% with human reviewers, enabling the automation of routine evaluation tasks while enhancing both efficiency and objectivity. By drawing on recent generative AI benchmarks across research support, educational assessments, reviewer matching, and large-scale application studies, we demonstrate a focused, practically validated solution that not only aligns with but slightly outperforms general GAI performance levels, offering a transformative approach to real-world manuscript evaluation.

Keywords

• Web platform
• Intelligent agent
• Automatic evaluation
• Improving objectivity

Hakem değerlendirme sürecini hızlandırmak için üretken yapay zeka kullanan bilimsel makale inceleme platformu

Öz

Bu çalışmada, hakem değerlendirme sürecini kolaylaştırmak için üretken yapay zeka (GAI) kullanan yeni bir platform tanıtılmaktadır. GAI'nin makale kalitesini etkili bir şekilde değerlendirme ve iyileştirme için kilit alanları belirleme kabiliyetini, 10 bilimsel makaleden oluşan bir vaka çalışmasını analiz ederek gösteriyoruz. Bulgularımız, GAI'nin değerlendirmeleri ile insan hakemlerin değerlendirmeleri arasında %63,6'lık bir benzerlik ortalamasıyla güçlü bir korelasyon olduğunu ortaya koyuyor ve GAI'nin ilk incelemeler için değerli bir araç olarak hizmet edebileceğini gösteriyor. Bu platform, rutin görevleri otomatikleştirerek ve objektif geri bildirim sağlayarak, geleneksel akran değerlendirmesinin zaman kısıtlamaları ve öznellik gibi zorluklarını ele almaktadır. Bu ilerleme, bilimsel araştırma değerlendirmesinin verimliliğini ve güvenilirliğini önemli ölçüde artırma potansiyeline sahiptir.

Anahtar Kelimeler

• Web platformu
• Akıllı ajan
• Otomatik değerlendirme
• Nesnelliğin iyileştirilmesi

References

1. Aries Systems. (n.d.). Editorial Manager [Cloud-based manuscript submission and peer review system]. Aries Systems Corporation. https://www.ariessys.com/solutions/editorial-manager/
2. ChatPDF GmbH. (n.d.). ChatPDF [AI-powered app]. https://www.chatpdf.com/
3. Checco, A., Bracciale, L., Loreti, P., Pinfield, S., & Bianchi, G. (2021). AI-assisted peer review. Humanities and Social Sciences Communications, 8, Article 25. https://doi.org/10.1057/s41599-020-00703-8
4. Cohen, J.F., & Moher, D. (2025). Generative artificial intelligence and academic writing: Friend or foe? Journal of Clinical Epidemiology, 179, Article 111646. https://doi.org/10.1016/j.jclinepi.2024.111646
5. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (Version 2). arXiv. https://doi.org/10.48550/ARXIV.1810.04805
6. Fischer, I., Sweeney, S., Lucas, M., & Gupta, N. (2024). Making sense of generative AI for assessments: Contrasting student claims and assessor evaluations. The International Journal of Management Education, 22(3), Article 101081. https://doi.org/10.1016/j.ijme.2024.101081
7. Google. (n.d.). Gemini [Large language model]. Google AI. https://gemini.google.com
8. Gozalo-Brizuela, R., & Merchan, E.E.G. (2024). A Survey of Generative AI Applications. Journal of Computer Science, 20(8), 801-818. https://doi.org/10.3844/jcssp.2024.801.818

9. Hitch, D., Richards, K., Gupta, A., & Thanekar, U. (2025). The ethical implications of using AI in qualitative research. In A. Gupta (Ed.), Artificial intelligence (AI) in social research (pp. 137-149). CABI Publishing. https://doi.org/10.1079/9781800626607.0013
10. Hossain, M.M. (2024). Using ChatGPT and other forms of generative AI in systematic reviews: Challenges and opportunities. Journal of Medical Imaging and Radiation Sciences, 55(1), 11-12. https://doi.org/10.1016/j.jmir.2023.11.005
11. Jan, Z., Ahamed, F., Mayer, W., Patel, N., Grossmann, G., Stumptner, M., & Kuusk, A. (2023). Artificial intelligence for industry 4.0: Systematic review of applications, challenges, and opportunities. Expert Systems with Applications, 216, Article 119456. https://doi.org/10.1016/j.eswa.2022.119456
12. Joe Devanny, H.D., & Grossfeld, E. (2023). Generative AI and intelligence assessment. The RUSI Journal, 168(7), 16-25. https://doi.org/10.1080/03071847.2023.2286775
13. Morande, S. (2023). Benchmarking Generative AI: A comparative evaluation and practical guidelines for responsible integration into academic research. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4571867
14. Open Journal Systems. (n.d.). OJS (Version 3.1) [Open source publishing platform]. https://openjournalsystems.com/
15. Saranya, A., & Subhashini, R. (2023). A systematic review of explainable artificial intelligence models and applications: Recent developments and future trends. Decision Analytics Journal, 7, Article 100230. https://doi.org/10.1016/j.dajour.2023.100230
16. Sengar, S.S., Hasan, A.B., Kumar, S., & Carroll, F. (2024). Generative artificial intelligence: A systematic review and applications. Multimedia Tools and Applications, 84, 23661-23700. https://doi.org/10.1007/s11042-024-20016-1
17. Sharly. (n.d.). Sharly AI [AI tool]. VOX AI Inc. https://sharly.ai/
18. Silverchair Support. (n.d.). ScholarOne Manuscripts [Workflow management system]. Silverchair. https://www.silverchair.com/products/scholarone-manuscripts/