YOLO–SAM-Guided ROI-Based Deep Learning for Non-Invasive Neonatal Jaundice Detection

Abstract

Early and reliable detection of neonatal jaundice is of critical importance for preventing bilirubin-induced neurological damage. However, visual assessment methods widely used in clinical practice are subjective and highly dependent on the examiner and environmental conditions. In this study, three different deep learning–based approaches were systematically evaluated for the automatic detection of neonatal jaundice using the NeoJaundice dataset. In the first approach, direct Convolutional Neural Network (CNN)-based classification was performed on raw images using the ResNet18 and MobileNetV2 architectures. In the second approach, automatic pixel-level region of interest (ROI) extraction was applied prior to classification using YOLO-based coarse localization and the Segment Anything Model (SAM). In the third and most advanced approach, CNN features ex-tracted from the ROI were transferred to an Long Short-Term Memory (LSTM) network with the aim of modeling contextual dependencies between spatial representations. To the best of our knowledge, this study is the first to systematically examine the effect of YOLO–SAM–based automatic ROI extraction on classification performance using a color-sensitive and clinically labeled neonatal jaundice dataset. Experimental results show that ROI-based models improve inter-class balance and sensitivity, while the hybrid CNN–LSTM architecture achieves the highest overall accuracy (76.45%) when combined with ResNet18 features. The obtained findings demonstrate that combining precise ROI extraction with contextual feature modeling provides a more reliable and generalizable framework for camera-based neonatal jaundice detection systems, particularly in low-resource and non-invasive clinical screening scenarios.

Keywords

• Bilirubin estimation
• CNN
• CNN–LSTM
• medical image analysis
• Neonatal jaundice
• YOLO-SAM

YOLO–SAM Rehberli ROI Tabanlı Derin Öğrenme ile Non-İnvaziv Yenidoğan Sarılığı Tespiti

Abstract

Yenidoğan sarılığının erken ve güvenilir biçimde tespit edilmesi, bilirubine bağlı nörolojik hasarların önlenmesi açısından kritik öneme sahiptir. Ancak klinik uygulamalarda yaygın olarak kullanılan görsel değerlendirme yöntemleri öznel olup, uygulayıcıya ve çevresel koşullara yüksek derecede bağımlıdır. Bu çalışmada, NeoJaundice veri seti kullanılarak yenidoğan sarılığının otomatik tespiti amacıyla üç farklı derin öğrenme tabanlı yaklaşım sistematik olarak değerlendirilmiştir. İlk yaklaşımda, ham görüntüler üzerinde ResNet18 ve MobileNetV2 mimarileri kullanılarak doğrudan Evrişimsel Sinir Ağı (CNN) tabanlı sınıflandırma gerçekleştirilmiştir. İkinci yaklaşımda, sınıflandırma öncesinde YOLO tabanlı kaba lokalizasyon ve Segment Anything Model (SAM) kullanılarak piksel seviyesinde otomatik ilgi bölgesi (Region of Interest, ROI) çıkarımı uygulanmıştır. Üçüncü ve en gelişmiş yaklaşımda ise ROI üzerinden çıkarılan CNN özellikleri, uzamsal temsiller arasındaki bağlamsal bağımlılıkların modellenmesi amacıyla Uzun Kısa Süreli Bellek (Long Short-Term Memory, LSTM) ağına aktarılmıştır. Bildiğimiz kadarıyla bu çalışma, renk duyarlılığı yüksek ve klinik olarak etiketlenmiş bir yenidoğan sarılığı veri seti üzerinde YOLO–SAM tabanlı otomatik ROI çıkarımının sınıflandırma performansına etkisini sistematik biçimde inceleyen ilk çalışmadır. Deneysel sonuçlar, ROI tabanlı modellerin sınıflar arası dengeyi ve duyarlılığı iyileştirdiğini, hibrit CNN–LSTM mimarisinin ise ResNet18 özellikleri ile birlikte kullanıldığında en yüksek genel doğruluk oranına (%76,45) ulaştığını göstermektedir. Elde edilen bulgular, hassas ROI çıkarımı ile bağlamsal özellik modellemesinin birleştirilmesinin, özellikle düşük kaynaklı ve non-invaziv klinik tarama senaryolarında kamera tabanlı yenidoğan sarılığı tespit sistemleri için daha güvenilir ve genellenebilir bir çerçeve sunduğunu ortaya koymaktadır.

Keywords

• Bilirubin tahmini
• CNN
• CNN-LSTM
• tıbbi görüntü analizi
• yenidoğan sarılığı
• YOLO-SAM

References

1. [1] F. Makhloughi, "Artificial intelligence-based non-invasive bilirubin prediction for neonatal jaundice using 1D convolutional neural network," Scientific Reports, vol. 15, no. 1, p. 11571, 2025/04/04 2025, doi: 10.1038/s41598-025-96100-9.
2. [2] U. Abiha, D. S. Banerjee, and S. Mandal, "Demystifying non-invasive approaches for screening jaundice in low resource settings: a review," Frontiers in Pediatrics, vol. 11, p. 1292678, 2023.
3. [3] H. Brits et al., "The prevalence of neonatal jaundice and risk factors in healthy term neonates at National District Hospital in Bloemfontein," African Journal of Primary Health Care and Family Medicine, vol. 10, no. 1, pp. 1-6, 2018.
4. [4] V. K. Bhutani and L. H. Johnson, "Newborn jaundice and kernicterus—health and societal perspectives," The Indian Journal of Pediatrics, vol. 70, no. 5, pp. 407-416, 2003.
5. [5] F.-T.-Z. Khanam, A. Al-Naji, A. G. Perera, D. Wang, and J. Chahl, "Non-invasive and non-contact automatic jaundice detection of infants based on random forest," Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, vol. 11, no. 6, pp. 2516-2529, 2023.
6. [6] M. Eliazer, S. Amaran, K. Sreekumar, A. Vikram, G. P. Joshi, and W. Cho, "Integrating vision transformer-based deep learning model with kernel extreme learning machine for non-invasive diagnosis of neonatal jaundice using biomedical images," Scientific Reports, vol. 15, no. 1, p. 25493, 2025.
7. [7] E. Keles and U. Bagci, "The past, current, and future of neonatal intensive care units with artificial intelligence: a systematic review," NPJ Digital Medicine, vol. 6, no. 1, p. 220, 2023.
8. [8] M. Kaddes, Y. M. Ayid, A. M. Elshewey, and Y. Fouad, "Breast cancer classification based on hybrid CNN with LSTM model," Scientific Reports, vol. 15, no. 1, p. 4409, 2025.

9. [9] R. C. Bandi, K. R. Prasad, A. Kamalakumari, and A. D. Rani, "An Intuitive U-LSTM Model for Classification and Recognition for Skin Cancer Detection," International Journal of Engineering Trends and Technology, vol. 72 no. 1, pp. 20-39, 2024.
10. [10] A. Shaik, S. S. Dutta, I. M. Sawant, S. Kumar, A. Balasundaram, and K. De, "An attention based hybrid approach using CNN and BiLSTM for improved skin lesion classification," Scientific Reports, vol. 15, no. 1, p. 15680, 2025/05/05 2025, doi: 10.1038/s41598-025-00025-2.
11. [11] C. S. Odeyemi, O. M. Olaniyan, B. A. Omodunbi, I. B. Samuel, A. A. Soladoye, and D. B. Olawade, "Hybridized artificial intelligence system for reducing neonatal mortality in Nigeria," International Journal of Medical Informatics, vol. 206, p. 106162, 2026/02/01/ 2026, doi: https://doi.org/10.1016/j.ijmedinf.2025.106162.
12. [12] N. Kaur, A. Mittal, and A. Gupta, "Neonatal Jaundice Detection: A Comprehensive Survey from Manual to Computer-Aided Methods," Journal of Bionic Engineering, pp. 1-31, 2025.
13. [13] B. K. Abdulkader and M. H. Aziz, "Non-invasive Methods for Diagnosing Jaundice in Newborns: A Review," International Journal of Advanced Natural Sciences and Engineering Researches, vol. 8, pp. 204-218, 2024.
14. [14] G. Jocher, A. Chaurasia, and J. Qiu, "Ultralytics yolov8. 2023," ed, 2024.
15. [15] A. Kirillov et al., "Segment anything," in Proceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 4015-4026.
16. [16] X. Li, X. Pu, W. Ling, and X. Song, "YOLO-SAM an end-to-end framework for efficient real time object detection and segmentation," Scientific Reports, vol. 15, no. 1, p. 40854, 2025.
17. [17] J. Reddy et al., "Cotton Yield Prediction via UAV-Based Cotton Boll Image Segmentation Using YOLO Model and Segment Anything Model (SAM)," Remote Sensing, vol. 16, no. 23, p. 4346, 2024.
18. [18] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
19. [19] A. G. Howard et al., "Mobilenets: Efficient convolutional neural networks for mobile vision applications," arXiv preprint arXiv:1704.04861, 2017.
20. [20] R. Karim, M. Zaman, and W. H. Yong, "A Non-invasive methods for neonatal jaundice detection and monitoring to assess bilirubin level: A review," Annals of Emerging Technologies in Computing (AETiC), vol. 7, no. 1, pp. 15-29, 2023.
21. [21] L. A. Menon, M. A. Joseph, and D. Surya, "Non-Invasive Neonatal Jaundice Detection Using Image Processing and Machine Learning," AIJR Proceedings, vol. 7, no. 5, pp. 53-61, 2025.
22. [22] A. Y. Abdulrazzak, S. L. Mohammed, A. Al-Naji, and J. Chahl, "Real-time jaundice detection in neonates based on machine learning models," BioMedInformatics, vol. 4, no. 1, pp. 623-637, 2024.
23. [23] D. Wang et al., "A Real-world Dataset and Benchmark For Foundation Model Adaptation in Medical Image Classification," Scientific Data, vol. 10, no. 1, p. 574, 2023/09/02 2023, doi: 10.1038/s41597-023-02460-0.
24. [24] W. Dequan et al. "NeoJaundice: Neonatal Jaundice Evaluation in Demographic Images." https://doi.org/10.6084/m9.figshare.22302559 (accessed 06.06.2026.