Explainable Machine Learning for Preschool Anxiety Disorders Using Raw PAPA Interviews

Öz

Generalized Anxiety Disorder (GAD) and Separation Anxiety Disorder (SAD) are common in preschool children but are often not recognized in routine care. The Preschool Age Psychiatric Assessment (PAPA) provides detailed diagnostic information, yet its use in everyday clinical practice is limited because it is time-consuming and requires trained interviewers. In this study, we developed explainable machine learning models to classify GAD and SAD using the publicly available Preschool Anxiety Study cohort (n = 917; ages 2–5; 76 raw PAPA items). We trained Logistic Regression, Random Forest, XGBoost, and LightGBM models with SMOTE-Tomek resampling applied only to the training data and performed Optuna TPE-based hyperparameter optimization for XGBoost and LightGBM. Model calibration was evaluated using the Brier score, Expected Calibration Error, and the Hosmer–Lemeshow test, while SHAP TreeExplainer was used to interpret model decisions at both global and individual levels. LightGBM achieved the highest cross-validation AUC values (GAD: 0.9928; SAD: 0.9897), and XGBoost showed strong and well-balanced performance on the independent test set (GAD AUC = 0.9865, MCC = 0.9055; SAD AUC = 0.9948, MCC = 0.9057). LightGBM was the only well-calibrated model for GAD, whereas both gradient boosting models showed very good calibration for SAD. A reduced 10-item model provided a practical screening prototype for GAD in primary care by offering higher sensitivity with an AUC very close to the full model. However, external validation in independent samples is still needed to confirm these findings.

Anahtar Kelimeler

• Gradient Boosting
• Machine Learning
• Preschool Anxiety Disorders
• Preschool Age Psychiatric Assessment
• SHAP

Ham PAPA Görüşmeleri Kullanılarak Okul Öncesi Anksiyete Bozuklukları İçin Açıklanabilir Makine Öğrenmesi

Öz

Genelleştirilmiş Anksiyete Bozukluğu (GAD) ve Ayrılma Anksiyetesi Bozukluğu (SAD), okul öncesi dönemde sık görülen ancak çoğu zaman tanınmayan bozukluklardır. Okul Öncesi Dönem Psikiyatrik Değerlendirmesi (PAPA) ayrıntılı tanısal bilgi sağlar fakat süresinin uzunluğu ve uzman gereksinimi nedeniyle rutin klinik uygulamada sınırlı kullanılmaktadır. Bu çalışmada, herkese açık Preschool Anxiety Study kohortunu (n = 917; 2–5 yaş; 76 ham PAPA maddesi) kullanarak GAD ve SAD sınıflandırması için açıklanabilir makine öğrenmesi modelleri geliştirdik. Lojistik Regresyon, Rastgele Orman, XGBoost ve LightGBM modellerini yalnızca eğitim verisine uygulanan SMOTE-Tomek yeniden örnekleme ile eğittik ve XGBoost ile LightGBM için Optuna TPE tabanlı eniyileme yaptık. Kalibrasyonu Brier skoru, Beklenen Kalibrasyon Hatası ve Hosmer–Lemeshow testi ile değerlendirdik; SHAP TreeExplainer ile de model kararlarını küresel ve bireysel düzeyde yorumladık. LightGBM, çapraz doğrulamada en yüksek AUC değerlerine ulaştı (GAD: 0.9928; SAD: 0.9897), XGBoost ise bağımsız test setinde güçlü ayırt edicilik ve denge sağladı (GAD AUC = 0.9865, MCC = 0.9055; SAD AUC = 0.9948, MCC = 0.9057). GAD için iyi kalibre edilmiş tek model LightGBM olurken, her iki artırmalı model de SAD için çok iyi kalibrasyon gösterdi. On maddelik azaltılmış model, GAD için tam modele çok yakın AUC ile daha yüksek duyarlılık sunarak birinci basamakta uygulanabilir pratik bir tarama prototipi ortaya koydu. Yine de bulguların doğrulanması için dış geçerlilik çalışmaları gerekmektedir.

Anahtar Kelimeler

• Gradient Boosting
• Makine öğrenmesi
• Okul öncesi anksiyete bozuklukları
• Okul Öncesi Dönem Psikiyatrik Değerlendirmesi
• SHAP

Kaynakça

1. [1] Meng, J. F., & Wiznitzer, E. (2024). Factors associated with not receiving mental health services among children with a mental disorder in early childhood in the United States, 2021–2022. Preventing Chronic Disease, 21, 240126. https://doi.org/10.5888/pcd21.240126
2. [2] Leeb, R. T., Danielson, M. L., Claussen, A. H., Robinson, L. R., Lebrun-Harris, L. A., Ghandour, R., Bitsko, R. H., Katz, S. M., Kaminski, J. W., & Brown, J. (2024). Trends in mental, behavioral, and developmental disorders among children and adolescents in the US, 2016–2021. Preventing Chronic Disease, 21, 240142. https://doi.org/10.5888/pcd21.240142
3. [3] Carpenter, K. L. H., Sprechmann, P., Calderbank, R., Sapiro, G., & Egger, H. L. (2016). Quantifying risk for anxiety disorders in preschool children: A machine learning approach. PLoS ONE, 11(11), e0165524. https://doi.org/10.1371/journal.pone.0165524 [4] Franz, L., Angold, A., Copeland, W., Costello, E. J., Towe-Goodman, N., & Egger, H. (2013). Preschool anxiety disorders in pediatric primary care: Prevalence and comorbidity. Journal of the American Academy of Child & Adolescent Psychiatry, 52(12), 1294–1303. https://doi.org/10.1016/j.jaac.2013.09.008
4. [5] Hussong, J., Greiner, M., Schiedermaier, U., Mattheus, H., & Von Gontard, A. (2022). Anxiety disorders, depression and incontinence in preschool children: A population-based study. Neurourology and Urodynamics, 41(8), 1800–1808. https://doi.org/10.1002/nau.25025
5. [6] Steinsbekk, S., Ranum, B., & Wichstrøm, L. (2022). Prevalence and course of anxiety disorders and symptoms from preschool to adolescence: A 6-wave community study. Journal of Child Psychology and Psychiatry, 63(5), 527–534. https://doi.org/10.1111/jcpp.13487
6. [7] Egger, H. L., Erkanli, A., Keeler, G., Potts, E., Walter, B. K., & Angold, A. (2006). Test-retest reliability of the Preschool Age Psychiatric Assessment (PAPA). Journal of the American Academy of Child & Adolescent Psychiatry, 45(5), 538–549. https://doi.org/10.1097/01.chi.0000205705.71194.b8
7. [8] Li, Y., Song, Y., Sui, J., Greiner, R., Li, X., Greenshaw, A. J., Liu, Y. S., & Cao, B. (2024). Prospective prediction of anxiety onset in the Canadian longitudinal study on aging (CLSA): A machine learning study. Journal of Affective Disorders, 357, 148–155. https://doi.org/10.1016/j.jad.2024.04.098
8. [9] Qasrawi, R., Vicuna Polo, S. P., Abu Al-Halawa, D., Hallaq, S., & Abdeen, Z. (2022). Assessment and Prediction of Depression and Anxiety Risk Factors in Schoolchildren: Machine Learning Techniques Performance Analysis. JMIR Formative Research, 6(8), e32736. https://doi.org/10.2196/32736

9. [10] Gök, S., İş, H., & Özdaş, M. (2025). Artificial intelligence anxiety among university students and academics: The case of Batman University. Balkan Journal of Electrical and Computer Engineering, 13(4), 400–409. https://doi.org/10.17694/bajece.1649753
10. [11] Chavanne, A. V., Paillère Martinot, M. L., Penttilä, J., Grimmer, Y., Conrod, P., Stringaris, A., van Noort, B., Isensee, C., Becker, A., Banaschewski, T., Bokde, A. L. W., Desrivières, S., Flor, H., Grigis, A., Garavan, H., Gowland, P., Heinz, A., Brühl, R., Nees, F., ... Artiges, E. (2023). Anxiety onset in adolescents: A machine-learning prediction. Molecular Psychiatry, 28(2), 639–646. https://doi.org/10.1038/s41380-022-01840-z
11. [12] Jafari, S., Sharini, H., Foroughi, A., & Almasi, A. (2025). Brain mapping, biomarker identification and using machine learning method for diagnosis of anxiety during emotional face in preschool children. Brain Research Bulletin, 221, 111205. https://doi.org/10.1016/j.brainresbull.2025.111205
12. [13] Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 4765-477
13. [14] Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020). From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence, 2(1), 56–67. https://doi.org/10.1038/s42256-019-0138-9
14. [15] Ponce-Bobadilla, A. V., Schmitt, V., Maier, C. S., Mensing, S., & Stodtmann, S. (2024). Practical guide to SHAP analysis: Explaining supervised machine learning model predictions in drug development. Clinical and Translational Science, 17(11), e70056. https://doi.org/10.1111/cts.70056
15. [16] Yu, C., Kong, X., Yu, W., Ni, X., Chen, J., & Liao, X. (2025). Machine learning models for predicting the risk of depressive symptoms in Chinese college students. Frontiers in Psychiatry, 16, 1648585. https://doi.org/10.3389/fpsyt.2025.1648585
16. [17] Carpenter, K. (2016). Quantifying risk for anxiety disorders in preschool children: A machine learning approach [Data set]. Harvard Dataverse. https://doi.org/10.7910/DVN/N42LWG
17. [18] Batista, G. E. A. P. A., Prati, R. C., & Monard, M. C. (2004). A study of the behavior of several methods for balancing machine learning training data. SIGKDD Explorations Newsletter, 6(1), 20–29. https://doi.org/10.1145/1007730.1007735
18. [19] Akiba, T., Sano, S., Yanase, T., Ohta, T., & Koyama, M. (2019, July). Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (pp. 2623–2631). ACM. https://doi.org/10.1145/3292500.3330701
19. [20] Fluss, R., Faraggi, D., & Reiser, B. (2005). Estimation of the Youden index and its associated cutoff point. Biometrical Journal, 47(4), 458–472. https://doi.org/10.1002/bimj.200410135
20. [21] Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning (pp. 1321–1330). PML
21. [22] Hosmer, D. W., Lemeshow, S., & Sturdivant, R. X. (2013). Applied logistic regression (3rd ed.). Hoboken, NJ: John Wiley & Sons.” Eğer kesinlikle 2000 yılındaki 2. baskıyı kullandıysanız, Sturdivant'ı yazar listesinden çıkarmalı ve şu şekilde yazmalısınız: “Hosmer, D. W., & Lemeshow, S. (2000). Applied logistic regression (2nd ed.). New York: John Wiley & Sons.
22. [23] Carpenter, J., & Bithell, J. (2000). Bootstrap confidence intervals: When, which, what? A practical guide for medical statisticians. Statistics in Medicine, 19(9), 1141–1164. https://doi.org/10.1002/sim.564
23. [24] D. Chicco and G. Jurman, “The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation,” BMC Genomics, vol. 21, no. 1, p. 6, Jan. 2020, doi: 10.1186/s12864-019-6413-7.
24. [25] Chen, T., & Guestrin, C. (2016, August). XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785–794). ACM. https://doi.org/10.1145/2939672.2939785
25. [26] Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., & Liu, T.-Y. (2017). LightGBM: A highly efficient gradient boosting decision tree. Advances in Neural Information Processing Systems, 30. https://proceedings.neurips.cc/paper_files/paper/2017/file/6449f44a102fde848669bdd9eb6b76fa-Paper.pdf
26. [27] Lemaître, G., Nogueira, F., & Aridas, C. K. (2017). Imbalanced-learn: A Python toolbox to tackle the curse of imbalanced datasets in machine learning. Journal of Machine Learning Research, 18(17), 1–5. http://jmlr.org/papers/v18/16-365.html
27. [28] Fan, C., Zhang, Y., Ma, H., Ma, Z., Yin, X., Zhang, X., & Zhao, S. (2024). A novel imbalance fault diagnosis method based on data augmentation and hybrid deep learning models. Structural Health Monitoring, 25(1). Advance online publication. https://doi.org/10.1177/14759217241291143
28. [29] “Innab, N., Alsubai, S., Alabdulqader, E. A., Alarfaj, A. A., Umer, M., Trelova, S., & Ashraf, I. (2024). Automated approach for fetal and maternal health management using light gradient boosting model with SHAP explainable AI. Frontiers in Public Health, 12, 1462693. https://doi.org/10.3389/fpubh.2024.1462693
29. [30] Bowlby, J. (1982). Attachment and loss: Retrospect and prospect. American Journal of Orthopsychiatry, 52(4), 664–678. https://doi.org/10.1111/j.1939-0025.1982.tb01456.x