Detection of risk factors of PCOS patients with Local Interpretable Model-agnostic Explanations (LIME) Method that an explainable artificial intelligence model

Abstract

Aim: In this study, it is aimed to extract patient-based explanations of the contribution of important features in the decision-making process (estimation) of the Random forest (RF) model, which is difficult to interpret for PCOS disease risk, with Local Interpretable Model-Agnostic Explanations (LIME). Materials and Methods: In this study, the Local Interpretable Model-Agnostic Annotations (LIME) method was applied to the “Polycystic ovary syndrome” dataset to explain the Random Forest (RF) model, which is difficult to interpret for PCOS risk factors estimation. This dataset is available at https://www.kaggle.com/prasoonkottarathil/polycystic-ovary-syndrome-pcos. Results: Accuracy, sensitivity, specificity, positive predictive value, negative predictive value and balanced accuracy obtained from the Random Forest method were 86.03%, 86.32%, 85.37%, 93.18%, 72.92% and 85.84% respectively. According to the obtained results, the observations whose results were obtained, the values of Follicle (No) L. and Follicle (No) R. in different value ranges were positively correlated with the absence of PCOS. For the observations whose absence of PCOS results were obtained, the variables RBS(mg/dl), bmi_y, fsh_lh, TSH (mIU/L), Endometrium (mm) also played a role in obtaining the results. In addition, for the observations whose results were obtained, the values of Follicle No L and Follicle No R in different value ranges were also found to be positively correlated with PCOS. In addition, beta-HCG(mIU/mL), PRG(ng/mL), RBS(mg/dl), bmi_y, Endometrium (mm), fsh_lh variables also played a role in obtaining the results for PCOS. Conclusion: When the observations obtained from the results are examined, it can be said that the Follicle (No) L. and Follicle (No) R. variables are the most effective variables on the presence or absence of PCOS. For different value ranges of these two variables, the result of PCOS or not varies. Based on this, it can be said that different values of Follicle (No) L. and Follicle (No) R. variables for PCOS status may be effective in determining the disease.

Keywords

• PCOS
• random forest
• Explainable Artificial Intelligence
• Local Interpretable Model-agnostic Explanations (LIME).

References

1. Yu K-H, Beam AL, Kohane IS: Artificial intelligence in healthcare. Nature biomedical engineering 2:719-731, 2018.
2. Sokol K, Santos-rodriguez R, Hepburn A, et al: Surrogate Prediction Explanations Beyond LIME. no HCML, 2019.
3. Kononenko I: Machine learning for medical diagnosis: history, state of the art and perspective. Artificial Intelligence in medicine 23:89-109, 2001.
4. Deo RC: Machine learning in medicine. Circulation 132:1920-1930, 2015.
5. He D, Mathews SC, Kalloo AN, et al: Mining high-dimensional administrative claims data to predict early hospital readmissions. Journal of the American Medical Informatics Association 21:272-279, 2014.
6. Pederson JL, Majumdar SR, Forhan M, et al: Current depressive symptoms but not history of depression predict hospital readmission or death after discharge from medical wards: a multisite prospective cohort study. General hospital psychiatry 39:80-85, 2016.
7. Futoma J, Morris J, Lucas J: A comparison of models for predicting early hospital readmissions. Journal of biomedical informatics 56:229-238, 2015.
8. Katuwal GJ, Chen R: Machine learning model interpretability for precision medicine. arXiv preprint arXiv:161009045, 2016.

9. Bastani O, Kim C, Bastani H: Interpreting blackbox models via model extraction. arXiv preprint arXiv:170508504, 2017.
10. Stiglic G, Kocbek P, Fijacko N, et al: Interpretability of machine learning‐based prediction models in healthcare. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 10:e1379, 2020.
11. Escalante HJ, Escalera S, Guyon I, et al: Explainable and interpretable models in computer vision and machine learning, Springer, 2018.
12. Garreau D, Luxburg U: Explaining the explainer: A first theoretical analysis of LIME, Proceedings, International Conference on Artificial Intelligence and Statistics, 2020 (available from PMLR)
13. Hu L, Chen J, Nair VN, et al: Locally interpretable models and effects based on supervised partitioning (LIME-SUP). arXiv preprint arXiv:180600663, 2018.
14. Mehrotra P, Chatterjee J, Chakraborty C, et al: Automated screening of Polycystic Ovary Syndrome using machine learning techniques, Proceedings, 2011 Annual IEEE India Conference, 2011 (available from IEEE)
15. Denny A, Raj A, Ashok A, et al: i-HOPE: Detection And Prediction System For Polycystic Ovary Syndrome (PCOS) Using Machine Learning Techniques, Proceedings, TENCON 2019-2019 IEEE Region 10 Conference (TENCON), 2019 (available from IEEE).
16. Meena K, Manimekalai M, Rethinavalli S: Correlation of Artificial Neural Network Classification and NFRS Attribute Filtering Algorithm for PCOS Data. Int J Res Eng Technol 4:519-524, 2015.
17. Vikas B, Anuhya B, Chilla M, et al: A Critical Study of Polycystic Ovarian Syndrome (PCOS) Classification Techniques. International Journal of Computational Engineering & Management 21:1-7, 2018.
18. Kahsar-Miller MD, Nixon C, Boots LR, et al: Prevalence of polycystic ovary syndrome (PCOS) in first-degree relatives of patients with PCOS. Fertility and sterility 75:53-58, 2001.
19. Breiman L: Bagging predictors. Machine learning 24:123-140, 1996.
20. Breiman L: Random forests. Machine learning 45:5-32, 2001.
21. Ho TK: The random subspace method for constructing decision forests. IEEE transactions on pattern analysis and machine intelligence 20:832-844, 1998.
22. Izquierdo-Verdiguier E, Zurita-Milla R: An evaluation of Guided Regularized Random Forest for classification and regression tasks in remote sensing. International Journal of Applied Earth Observation and Geoinformation 88:102051, 2020.
23. Prasad AM, Iverson LR, Liaw A: Newer classification and regression tree techniques: bagging and random forests for ecological prediction. Ecosystems 9:181-199, 2006.
24. Panov P, Džeroski S: Combining bagging and random subspaces to create better ensembles, Proceedings, International Symposium on Intelligent Data Analysis, 2007 (available from Springer).
25. Shi S, Zhang X, Fan W: A modified perturbed sampling method for local interpretable model-agnostic explanation. arXiv preprint arXiv:200207434, 2020.
26. Ribeiro MT, Singh S, Guestrin C: " Why should i trust you?" Explaining the predictions of any classifier, Proceedings, Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 2016 (available from)
27. Kumarakulasinghe NB, Blomberg T, Liu J, et al: Evaluating local interpretable model-agnostic explanations on clinical machine learning classification models, Proceedings, 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS), 2020 (available from IEEE).
28. Zafar MR, Khan NM: DLIME: a deterministic local interpretable model-agnostic explanations approach for computer-aided diagnosis systems. arXiv preprint arXiv:190610263, 2019.
29. McLuskie I, Newth A: New diagnosis of polycystic ovary syndrome. BMJ: British Medical Journal 356, 2017.
30. Khan MJ, Ullah A, Basit S: Genetic basis of polycystic ovary syndrome (PCOS): current perspectives. The application of clinical genetics 12:249, 2019.
31. Arrieta AB, Díaz-Rodríguez N, Del Ser J, et al: Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion 58:82-115, 2020.
32. Gunning D, Stefik M, Choi J, et al: XAI—Explainable artificial intelligence. Science Robotics 4, 2019.