A Model for Smart Detection: Modified Explainable Machine Learning for Interpreting Detection Results
Article Sidebar
Main Article Content
Abstract
Medical imaging analysis using artificial intelligence has become a powerful system for assisting the doctor in diagnosing some diseases. Most of CAD performed excellent performance with average accuracy of more than 80%. Regardless of the excellent performance of artificial intelligence in the CAD, implementation of artificial intelligence in medical cases is still causing controversy. It happened due to the black-box principle of AI. Actually, both machine learning and deep learning worked in the black-box direction in which it was difficult to recognize how the model performed and how it analyzed the data. Hence, it became controversial since there remained some big questions about “how can the doctor trust the AI result?” Regarding this problem, a continued solution was needed. In this study we proposed a modified SHAP for explaining the artificial intelligence result. The modification itself is conducted by performing correlation in the perturbation process of SHAP. Our proposed solution was performed into two different datasets to evaluate the significance and the reliability of the proposed solution. According to both the visual analysis and statistical test, we conclude that the proposed solution gave a more rational explanation compared to the original SHAP.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
Calisto FM, Nunes N, Nascimento JC. Modeling adoption of intelligent agents in medical imaging. Int J Hum Comput Stud [Internet].
;168:102922. Available from: https://www.sciencedirect.com/science/article/pii/S1071581922001422
Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med [Internet]. 2019;25(1):44–56. Available from: https://doi.org/10.1038/s41591-018-0300-7
Sundaram L, Gao H, Padigepati SR, McRae JF, Li Y, Kosmicki JA, et al. Predicting the clinical impact of human mutation with deep neural networks. Nat Genet [Internet]. 2018;50(8):1161–70. Available from: https://doi.org/10.1038/s41588-018-0167-z
Bickmore TW, Trinh H, Olafsson S, O’Leary TK, Asadi R, Rickles NM, et al. Patient and consumer safety risks when using conversational assistants for medical information: an observational study of Siri, Alexa, and Google Assistant. J Med Internet Res. 2018 Sep;20(9):e11510.
Madani A, Ong JR, Tibrewal A, Mofrad MRK. Deep echocardiography: dataefficient supervised and semi-supervised deep learning towards automated diagnosis of cardiac disease. npj Digit Med [Internet]. 2018;1(1):59. Available from: https://doi.org/10.1038/s41746-
-0065-x
Nugroho HA, Frannita EL, Hutami AHT, Chondah L, Nugroho A, Fauzi RN, et al. Deep learning for analyzing thyroid nodule malignancy based on the composition characteristic of ultrasonography images. In: 2020 International Conference on Advanced Computer Science and Information Systems (ICACSIS). 2020. p. 77–82.
Prananda AR, Nugroho HA, Ardiyanto I. Enumeration of Plasmodium parasites on thin blood smear digital microscopic images. In: 2019 5th International Conference on Science in Information Technology (ICSITech). 2019. p. 223–8.
Nugroho HA, Frannita EL, Hutami AHT. Thyroid nodules stratification based on orientation characteristics using machine learning approach. In: 2020 3rd International Conference on Computer and Informatics Engineering (IC2IE). 2020. p. 52–7.
Coudray N, Ocampo PS, Sakellaropoulos T, Narula N, Snuderl M, Fenyö D, et al. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat Med [Internet]. 2018;24(10):1559–67. Available from:
https://doi.org/10.1038/s41591-018-0177-5
Ahmad MA, Eckert C, McKelvey G, Zolfagar K, Zahid A, Teredesai A. Death versus data science: predicting end of life. Proc 30th Innov Appl Artif Intell Conf IAAI 2018. 2018;7719–26.
Currie G, Hawk KE, Rohren E, Vial A, Klein R. Machine learning and deep learning in medical imaging: intelligent imaging. J Med Imaging Radiat Sci [Internet]. 2019;50(4):477–87. Available from:https://www.sciencedirect.com/science/article/pii/S1939865419305041
Gleichgerrcht E, Munsell BC, Alhusaini S, Alvim MKM, Bargalló N, Bender B, et al. Artificial intelligence for classification of temporal lobe epilepsy with ROIlevel MRI data: a worldwide ENIGMAEpilepsy study. NeuroImage Clin [Internet]. 2021;31:102765. Available from:
https://www.sciencedirect.com/science/article/pii/S2213158221002096
Mikołajczyk A, Grochowski M. Data augmentation for improving deep learning in image classification problem. In: 2018 International Interdisciplinary PhD Workshop (IIPhDW). 2018. p. 117–22.
Peng WV, Mayorga RV, Hussein EMA. An automated confirmatory system for analysis of mammograms. Comput Methods Programs Biomed [Internet]. 2016;125:134–44. Available from: https://www.sciencedirect.com/science/article/pii/S0169260715002461
Koh J, Yoon Y, Kim S, Han K, Kim E-K. Deep learning for the detection of breast cancers on chest computed tomography. Clin Breast Cancer [Internet]. 2021; Available from: https://www.sciencedirect.com/science/article/pii/S1526820921001154
Nurfauzi R, Nugroho HA, Ardiyanto I, Frannita EL. Autocorrection of lung boundary on 3D CT lung cancer images. J King Saud Univ - Comput Inf Sci [Internet]. 2021;33(5):518–27. Available from: https://www.sciencedirect.com/science/article/pii/S1319157818308632
Haenssle HA, Fink C, Schneiderbauer R, Toberer F, Buhl T, Blum A, et al. Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann Oncol [Internet]. 2018;29(8):1836–42. Available from: https://www.sciencedirect.com/science/article/pii/S0923753419341055
Frannita EL, Nugroho HA, Nugroho A, Zulfanahri, Ardiyanto I. Thyroid nodule classification based on characteristic of margin using geometric and statistical features. In: 2018 2nd International Conference on Biomedical Engineering (IBIOMED). 2018. p. 54–9.
Nugroho HA, Frannita EL. Thyroid cancer classification using transfer learning. In: 2021 International Conference on Computer Science and Engineering (IC2SE). 2021. p. 1–5.
Ouahabi A, Taleb-Ahmed A. Deep learning for real-time semantic segmentation: application in ultrasound imaging. Pattern Recognit Lett [Internet]. 2021;144:27–34. Available from: https://www.sciencedirect.com/science/article/pii/S0167865521000234
Ma J, Wu F, Jiang T, Zhao Q, Kong D. Ultrasound image-based thyroid nodule automatic segmentation using convolutional neural networks. Int J Comput Assist Radiol Surg. 2017 Jul 1;12:1–16.
Zhang S, Du H, Jin Z, Zhu Y, Zhang Y, Xie F, et al. A novel interpretable computer-aided diagnosis system of thyroid nodules on ultrasound based on clinical experience. IEEE Access. 2020;8:53223–31.
Nugroho HA, Marsiano AFD, Xaphakdy K, Sihakhom P, Frannita EL, Nurfauzi R, et al. Multithresholding approach for segmenting Plasmodium parasites. In: 2019 11th International Conference on Information Technology and Electrical Engineering (ICITEE). 2019. p. 1–5.
Yu S, Xiao D, Frost S, Kanagasingam Y. Robust optic disc and cup segmentation with deep learning for glaucoma detection. Comput Med Imaging Graph [Internet]. 2019;74:61–71. Available from: https://www.sciencedirect.com/science/article/pii/S0895611118305573
Deperlioglu O, Kose U, Gupta D, Khanna A, Giampaolo F, Fortino G. Explainable framework for glaucoma diagnosis by image processing and convolutional neural network synergy: Analysis with doctor evaluation. Futur Gener Comput Syst [Internet]. 2022;129:152–69. Available from: https://www.sciencedirect.com/science/article/pii/S0167739X21004556
Vinícius dos Santos Ferreira M, Oseas de Carvalho Filho A, Dalília de Sousa A, Corrêa Silva A, Gattass M. Convolutional neural network and texture descriptorbased automatic detection and diagnosis of glaucoma. Expert Syst Appl [Internet]. 2018;110:250–63. Available from: https://www.sciencedirect.com/science/article/pii/S0957417418303567
Mugasa H, Dua S, Koh JEW, Hagiwara Y, Lih OS, Madla C, et al. An adaptive feature extraction model for classification of thyroid lesions in ultrasound images. Pattern Recognit Lett [Internet]. 2020;131:463–73. Available from: https://www.sciencedirect.com/science/
article/pii/S0167865520300490
Chen J, You H, Li K. A review of thyroid gland segmentation and thyroid nodule segmentation methods for medical ultrasound images. Comput Methods Programs Biomed [Internet]. 2020;185:105329. Available from: https://www.sciencedirect.com/science/ article/pii/S0169260719308454
Yang W, Dong Y, Du Q, Qiang Y, Wu K, Zhao J, et al. Integrate domain knowledge in training multi-task cascade deep learning model for benign–malignant thyroid nodule classification on ultrasound images. Eng Appl Artif Intell [Internet]. 2021;98:104064. Available from: http://www.sciencedirect.com/science/article/pii/S0952197620303304
Tulsani A, Kumar P, Pathan S. Automated segmentation of optic disc and optic cup for glaucoma assessment using improved UNET++ architecture. Biocybern Biomed Eng [Internet]. 2021;41(2):819–32. Available from: https://www.sciencedirect.com/science/
article/pii/S0208521621000656
Khudhair Abbas S. Thyroid tissue segmentation and classification in ultrasound image of artificial intelligence. Mater Today Proc [Internet]. 2021; Available from: https://www.sciencedirect.com/science/article/pii/S221478532103279X
Nugroho HA, Zulfanahri, Frannita E, Ardiyanto I, Choridah L. Computer aided diagnosis for thyroid cancer system based on internal and external characteristics. J King Saud Univ Comput Inf Sci. 2019 Jan 23;
Kesarkar XA, Kulhalli KV. Thyroid nodule detection using artificial neural network. In: 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS). 2021. p. 11–5.
Bayrak EA, Kırcı P, Ensari T. Comparison of machine learning methods for breast cancer diagnosis. In: 2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT). 2019. p. 1–3.
Bayrak EA, Kirci P. Fractal analysis of thyroid ultrasound image data evaluation. In: 2020 IEEE 2nd International Conference on System Analysis & Intelligent Computing (SAIC). 2020. p. 1–4.
Ribeiro MT, Singh S, Guestrin C. “Why should I trust you?”: Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining [Internet]. New York (NY): Association for Computing Machinery; 2016. p. 1135–44.
Seethi VDR, LaCasse Z, Chivte P, Bland J, Kadkol SS, Gaillard ER, et al. An explainable AI approach for diagnosis of COVID-19 using MALDI-ToF mass spectrometry. Expert Syst Appl [Internet]. 2024;236:121226. Available from:https://www.sciencedirect.com/science/
article/pii/S0957417423017281
Deepanshi, Budhiraja I, Garg D, Kumar N. Choquet integral based deep learning model for COVID-19 diagnosis using explainable AI for NGIoT models. Comput Commun [Internet]. 2023;212:227–38. Available from: https://www.sciencedirect.com/science/ article/pii/S0140366423003481
Caroprese L, Vocaturo E, Zumpano E. Argumentation approaches for explainable AI in medical informatics. Intell Syst Appl [Internet]. 2022;16:200109. Available from: https://www.sciencedirect.com/science/article/pii/S2667305322000473
Di Martino F, Delmastro F, Dolciotti C. Explainable AI for malnutrition risk prediction from m-health and clinical data. Smart Health [Internet]. 2023;30:100429. Available from: https://www.sciencedirect.com/science/article/pii/S2352648323000570
Niranjan K, Shankar Kumar S, Vedanth S, Chitrakala DS. An explainable AI driven decision support system for COVID-19 diagnosis using fused classification and segmentation. Procedia Comput Sci [Internet]. 2023;218:1915–25. Available from: https://www.sciencedirect.com/science/article/pii/S1877050923001680
Cao Y, Kunaprayoon D, Ren L. Interpretable AI-assisted clinical decision making for dose prescription in radiosurgery of brain metastases. Radiother Oncol [Internet]. 2023;187:109842. Available from: https://www.sciencedirect.com/science/ article/pii/S0167814023897365
Kruk M. SHAP-NET, a network based on Shapley values as a new tool to improve the explainability of the XGBoost-SHAP model for the problem of water quality. Environ Model Softw [Internet]. 2025;188:106403. Available from: https://www.sciencedirect.com/science/
article/pii/S1364815225000878
Zhou H, Ma X, Guan H, Yang J, Wei B, Zhang Y, et al. Prediction of maize crude fat content based on improved conditional mutual information maximization and SHAP analysis. Food Chem [Internet]. 2025;493:146054. Available from: https://www.sciencedirect.com/science/article/pii/S0308814625033059
Lee MJ, Choi SY. The impact of financial statement indicators on bank credit ratings: Insights from machine learning and SHAP techniques. Financ Res Lett [Internet]. 2025;85:107758. Available from: https://www.sciencedirect.com/science/ article/pii/S1544612325010165
Nourani V, Dehghan M, Baghanam AH, Kantoush SA. Dual purpose of Shapley additive explanation (SHAP) in model explanation and feature selection for artificial intelligence-based digital twin of wastewater treatment plant. J Water Process Eng [Internet]. 2025;75:107947. Available from: https://www.sciencedirect.com/science/article/pii/S2214714425010190
Qurat Ul Ain S, Islam Rather KU. Integrated statistical modeling and machine learning techniques with SHAP for epidemiological data analysis. Ann Epidemiol [Internet]. 2025;108:85–91. Available from: https://www.sciencedirect.com/science/ article/pii/S1047279725001334
Zhao J, Wang F, Wen J, Dai H, Wu L, Guo Y, et al. Enhanced magnetite separation prediction by PLIMS multifactor coupling based on machine learning and SHAP interpretability analysis. Powder Technol [Internet]. 2026;467:121546. Available from: https://www.sciencedirect.com/science/article/pii/S0032591025009416
UCI. Breast Cancer Wisconsin (Diagnostic) Data Set [Internet]. Diagnostic Wisconsin Breast Cancer Database. 1995. Available from: https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+(Diagnostic)
Lundberg SM, Lee SI. A unified approach to interpreting model predictions. Adv Neural Inf Process Syst. 2017;2017-Decem(Section 2):4766–75.
Molnar C. A guide for making black box models explainable [Internet]. Interpretable Machine Learning. 2021 [cited 2021 Mar 13]. Available from: https://christophm.github.io/interpretableml-book/
Lundberg S. Math behind LinearExplainer with correlation feature perturbation [Internet]. SHAP Documentation. 2018 [cited 2021 Jul 15]. Available from: https://shap.readthedocs.io/en/latest/example_notebooks/tabular_examples/linear_-models/Math%20behind%20LinearExplainer%20with%20correlation%20feature-%20perturbation.html
