A Modular LLM-Driven System for Managing Academic Submissions and Editorial Workflow

Öz

In this research, an artificial intelligence based academic paper management system is developed. The major aim of the system is to assist conferences and journals in handling the entire process of academic papers from start to finish. The system offers the functionality of uploading papers, assigning users, assigning reviewers, peer review, editorial decisions, and tracking the submissions through an interactive web interface. One of the most highlighted areas in the system is the incorporation of large language models, Google Gemini, in the peer review process. The AutoReview component is used to analyze the language, structure, and originality of the submitted papers, and the Reviewer Matching component uses artificial intelligence to suggest suitable reviewers according to the semantic similarity between the manuscript information and the expertise of the reviewers. The system is developed by utilizing PHP, MySQL, and PDO for the secure back-end, and the front-end is made responsive by utilizing Bulma CSS and jQuery. The developed system is fully functional and is currently being used in a live production environment to manage real-world conference applications.

Anahtar Kelimeler

• Artificial Intelligence
• Academic Paper Management System
• Peer Review Automation
• Large Language Models
• Reviewer Matching

Akademik Makale Gönderimi ve Editörlük İş Akışı Yönetimi için Modüler LLM Tabanlı Sistem

Öz

Çalışmada, yapay zekâ tabanlı akademik bildiri ve makale yönetim sistemi geliştirilmiştir. Sistemin temel amacı, konferans ve dergilerin akademik makalelerin baştan sona tüm sürecini yönetmesine yardımcı olmaktır. Sistem, etkileşimli bir web arayüzü aracılığıyla makale yükleme, kullanıcı atama, hakem atama, akran değerlendirmesi, editoryal kararlar ve gönderimlerin takibi gibi işlevler sunmaktadır. Sistemin en öne çıkan özelliklerinden biri, akran değerlendirme sürecine büyük dil modelleri olan Google Gemini'nin dahil edilmesidir. AutoReview bileşeni, gönderilen makalelerin dilini, yapısını ve özgünlüğünü analiz etmek için kullanılırken, Reviewer Matching bileşeni yapay zekâ kullanarak makale bilgileri ile hakemlerin uzmanlıkları arasındaki anlamsal benzerliğe göre uygun hakemler önerir. Sistemin arka ucunda PHP, MySQL ve PDO kullanılarak güvenlikli olarak geliştirilmiş ve ön uç, Bulma CSS ve jQuery kullanılarak duyarlı hale getirilmiştir. Geliştirilen sistem tamamen işlevseldir ve konferans başvurularını yönetmek için canlı bir ortamda kullanılmaktadır.

Anahtar Kelimeler

• Yapay Zekâ
• Akademik Makale Yönetim Sistemi
• Hakem Değerlendirme Otomasyonu
• Büyük Dil Modelleri
• Hakem Eşleştirme

Kaynakça

1. Ahmadi A, Idahl A, Ahmadi P (2025) OpenReviewer: A fine tuned LLM for structured peer review generation. Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, United States.
2. Anjum S, Nawa NE, Karim MR (2022) Submission aware reviewer profiling for reviewer recommendation. https://arxiv.org/abs/2211.04194. Accessed 19 Dec 2025
3. Bolaños F, Salatino A, Osborne F, Motta E (2024) Artificial intelligence for literature reviews: Opportunities and challenges. https://arxiv.org/abs/2402.08565. Accessed 19 Dec 2025.
4. Checco A, Bracciale L, Loreti P, Pinfield S, Bianchi G (2021) AI-assisted peer review. Humanities and Social Sciences Communications 8(1): 1–11.
5. Committee on Publication Ethics (COPE) (2025) Ethical Guidelines for Peer Reviewers. https://publicationethics.org/guidance/guideline/ethical-guidelines-peer-reviewers. Accessed 17 Dec 2025
6. de la Torre-López J, Ramírez A, Romero J (2023) Artificial intelligence to automate the systematic review of scientific literature. Computing 105(10): 2171–2194.
7. Hsieh P, Narayan S, Teh YW (2024) Vulnerability of text matching in ML/AI conference reviewer assignment. https://arxiv.org/abs/2412.06606. Accessed 19 Dec 2025
8. Ishak WHW, Yamin FM, Mohsin MF, Mansor MF (2023) A comparative review of conference management systems. Journal of Technology and Operations Management 18(2): 87–93.

9. Kim J, Lee Y, Lee S (2025) Position: The AI conference peer review crisis demands author feedback and reviewer rewards. https://arxiv.org/abs/2505.04966. Accessed 19 Dec 2025
10. Latypova V (2023) Reviewer assignment decision support in an academic journal based on multicriteria assessment and text mining. International Conference on Information Technology and Nanotechnology (ITNT), Samara, Russia.
11. Leyton-Brown K, Nandwani Y, Zarkoob H, Cameron C, Newman N, Raghu D (2024) Matching papers and reviewers at large conferences. Artificial Intelligence 331: 104119.
12. Liang W, Izzo Z, Zhang Y, Lepp H, Cao H, Zhao X, et al. (2024) Monitoring AI-modified content at scale: A case study on the impact of ChatGPT on AI conference peer reviews. https://arxiv.org/abs/2403.07183. Accessed 19 Dec 2025
13. Okasa G, de León A, Strinzel M, Jorstad A, Milzow K, Egger M, Müller S (2025) A supervised machine learning approach for assessing grant peer review reports. Quantitative Science Studies 6: 1189–1214.
14. Schintler LA, McNeely CL, Witte J (2023) A critical examination of the ethics of AI-mediated peer review. https://arxiv.org/abs/2309.12356. Accessed 19 Dec 2025
15. Shin H, Tang J, Lee Y, Kim N, Lim H, Cho JY, et al. (2025) Mind the blind spots: A focus-level evaluation framework for LLM reviews. https://arxiv.org/abs/2502.17086. Accessed 19 Dec 2025
16. Stergiopoulos V, Vassilakopoulos M, Tousidou E, Kavvathas S, Corral A (2025) Conference management system utilizing an LLM-based recommendation system for the reviewer assignment problem. International Conference on Enterprise Information Systems (ICEIS), Porto, Portugal.
17. Sundaram G, Berleant D (2023) Automating systematic literature reviews with natural language processing and text mining: A systematic literature review. International Congress on Information and Communication Technology, Singapore.
18. Wei Q, Holt S, Yang J, Wulfmeier M, van der Schaar M (2025) The AI imperative: Scaling high quality peer review in machine learning. https://arxiv.org/abs/2506.08134. Accessed 19 Dec 2025
19. Weng Y, Zhu M, Bao G, Zhang H, Wang J, Zhang Y, Yang L (2024) CycleResearcher: Improving automated research via automated review. https://arxiv.org/abs/2411.00816. Accessed 19 Dec 2025
20. Wu W, Xi H, Zhang C (2024) Are the confidence scores of reviewers consistent with the review content? Evidence from top conference proceedings in AI. Scientometrics 129(7): 4109–4135.
21. Zhang Y, Shen Y, Kang S, Chen X, Jin B, Han J (2025) Chain-of-factors paper-reviewer matching. The ACM Web Conference, Sydney, Australia.
22. Zhuang Z, Chen J, Xu H, Jiang Y, Lin J (2025) Large language models for automated scholarly paper review: A survey. Information Fusion 331: 103332.
23. Zhyrova T, Kotenko N, Yushchenko O (2024) Architecture of the conference organization software application. Computer Science On-line Conference, Cham, Switzerland.