LLM-Supported Hybrid Reranking Architecture for Candidate-Job Matching in Recruitment Processes

Öz

Traditional applicant tracking systems miss candidates' potential due to their keyword-based structures, while pure large language model approaches carry high costs and hallucination risks. This study proposes a hybrid architecture that integrates vector-based semantic search with data validation steps and delivers the final decision through a large language model-based reranking module to overcome these issues. The proposed system processes candidate data through a multi-stage filtering funnel by separating structured and unstructured formats. Experimental analysis on a real-world dataset comprising 52 job postings and 36 candidate resumes demonstrates that this hybrid approach achieves a retrieval success rate of 91.67%, a human resources expert scoring alignment of 86.11%, and a mean absolute error of 0.200, significantly outperforming traditional applicant tracking systems and pure large language model methods in both accuracy and cost-efficiency. The findings show that the architecture achieves high retrieval success and scoring alignment, offering an explainable, scalable, and accurate decision support system for human resources processes. Ultimately, this study bridges the gap between theoretical RAG capabilities and practical human resources deployment, presenting a reliable framework for next-generation recruitment. 

Anahtar Kelimeler

• Candidate-Job Matching
• Explainable AI
• Hybrid Reranking Architecture
• Large Language Models
• Recruitment Automation
• Retrieval-Augmented Generation
• Vector Database

Etik Beyan

I declare that scientific and ethical rules were fully adhered to during the preparation of this study, that all data used were obtained with integrity, and that all sources used are fully referenced.

Kaynakça

1. [1]. Lo, FPW, Qiu, J, et al. 2025. AI Hiring with LLMs: A Context-Aware and Explainable Multi-Agent Framework for Resume Screening. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW); 4223–4232.
2. [2]. Karpukhin, V, Oguz, B, et al. 2020. Dense Passage Retrieval for Open-Domain Question Answering. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP); 6769–6781.
3. [3]. Ji, Z, Lee, N, et al. 2023. Survey of Hallucination in Natural Language Generation. ACM Computing Surveys; 55(12): Article 248, 1–38.
4. [4]. Magesh, V, Surani, F, et al. 2025. Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools. Journal of Empirical Legal Studies; 22(2): 216–242.
5. [5]. Huang, L, Yu, W, et al. 2025. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. ACM Transactions on Information Systems; 43(2): Article 42, 1–55.
6. [6]. Lewis, P, Perez, E, et al. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems; 33: 9459–9474.
7. [7]. OpenAI. 2024. New embedding models and API updates. OpenAI; https://openai.com/index/new-embedding-models-and-api-updates/ (accessed 23 December 2025).
8. [8]. Béchard, P, Marquez Ayala, O. 2024. Reducing hallucination in structured outputs via Retrieval-Augmented Generation. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 6: Industry Track); 228–238.

9. [9]. Asai, A, Wu, Z, et al. 2024. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. Proceedings of the International Conference on Learning Representations (ICLR).
10. [10]. Robertson, S, Zaragoza, H. 2009. The Probabilistic Relevance Framework: BM25 and Beyond. Foundations and Trends in Information Retrieval; 4(1–2): 1–174.
11. [11]. Sparck Jones, K. 1972. A Statistical Interpretation of Term Specificity and its Application in Retrieval. Journal of Documentation; 28(1): 11–21.
12. [12]. Gao, Y, Xiong, Y, et al. 2023. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv preprint; arXiv:2312.10997. https://arxiv.org/abs/2312.10997.
13. [13]. Clavié, B, Ciceu, A, Naylor, F, Soulié, G, Brightwell, T. 2023. Large Language Models in the Workplace: A Case Study on Prompt Engineering for Job Type Classification. Natural Language Processing and Information Systems (NLDB 2023), Lecture Notes in Computer Science; 13913: 3–17.
14. [14]. Sanh, V, Debut, L, Chaumond, J, Wolf, T. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. NeurIPS EMC² Workshop (5th Workshop on Energy Efficient Machine Learning and Cognitive Computing).
15. [15]. Omolayo, O, Babatope, O. 2025. Comparative Evaluation of Vector Embedding Frameworks for Scalable Sematic Retrieval in PDF-Based RAG Systems. International Journal of Research Publication and Reviews; 6(6): 12506–12511.
16. [16]. Nakano, R, Hilton, J, et al. 2021. WebGPT: Browser-assisted question-answering with human feedback. arXiv preprint; arXiv:2112.09332. https://arxiv.org/abs/2112.09332.
17. [17]. Brehme, L, Ströhle, T, Breu, R. 2025. Can LLMs Be Trusted for Evaluating RAG Systems? A Survey of Methods and Datasets. Proceedings of the IEEE Swiss Conference on Data Science (SDS); 16–23.
18. [18]. Johnson, J, Douze, M, Jégou, H. 2021. Billion-Scale Similarity Search with GPUs. IEEE Transactions on Big Data; 7(3): 535–547.
19. [19]. Fabris, A, Baranowska, N, et al. 2025. Fairness and Bias in Algorithmic Hiring: A Multidisciplinary Survey. ACM Transactions on Intelligent Systems and Technology; 16(1): Article 16, 1–54.
20. [20]. Bevara, RVK, Mannuru, NR, et al. 2025. Resume2Vec: Transforming Applicant Tracking Systems with Intelligent Resume Embeddings for Precise Candidate Matching. Electronics; 14(4): Article 794.
21. [21]. Liu, NF, Lin, K, et al. 2024. Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics; 12: 157–173.
22. [22]. Wu, W, Wang, H, Li, B, Huang, P, Zhao, X, Liang, L. 2025. MultiRAG: A Knowledge-Guided Framework for Mitigating Hallucination in Multi-Source Retrieval Augmented Generation. Proceedings of the IEEE 41st International Conference on Data Engineering (ICDE); 3070–3083.
23. [23]. Arora, V, Maggu, S. 2025. RAG-Enhanced LLM Job Recommendation Systems: Balancing Efficiency and Accuracy in Candidate-Job Matching. International Journal of Engineering Research & Technology; 14(10): Paper IJERTV14IS100191.
24. [24]. Barnett, S, Kurniawan, S, et al. 2024. Seven Failure Points When Engineering a Retrieval Augmented Generation System. Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering – Software Engineering for AI (CAIN); 194–199.
25. [25]. Liu, P, Shen, J, et al. 2025. Powering Job Search at Scale: LLM-Enhanced Query Understanding in Job Matching Systems. Proceedings of the 34th ACM International Conference on Information and Knowledge Management (CIKM); 4971–4975.