How to explain a machine learning model: HbA1c classification example

Deniz Topcu

Article Details

How to explain a machine learning model: HbA1c classification example

Authors:

Deniz Topcu

Publisher:

MediHealth Academy Yayıncılık

Main Topics:

Sağlık Kurumları Yönetimi

Journal:

Journal of Medicine and Palliative Care Vol.4 Issue 2, pp.117-125

Document Type:

Article

Publication Date:

3/2023

Document Language:

English

Document ID (Org No.):

2001286136

pISSN:

2717-7505

Keywords:

Abstract:

 Aim: Machine learning tools have various applications in healthcare. However, the implementation of developed models is still limited because of various challenges. One of the most important problems is the lack of explainability of machine learning models. Explainability refers to the capacity to reveal the reasoning and logic behind the decisions made by AI systems, making it straightforward for human users to understand the process and how the system arrived at a specific outcome. The study aimed to compare the performance of different model-agnostic explanation methods using two different ML models created for HbA1c classification. Material and Method: The H2O AutoML engine was used for the development of two ML models Gradient boosting machine GBM and default random forests DRF using 3,036 records from NHANES open data set. Both global and local model-agnostic explanation methods, including performance metrics, feature important analysis and Partial dependence, Breakdown and Shapley additive explanation plots were utilized for the developed models. Results: While both GBM and DRF models have similar performance metrics, such as mean per class error and area under the receiver operating characteristic curve, they had slightly different variable importance. Local explainability methods also showed different contributions to the features. Conclusion: This study evaluated the significance of explainable machine learning techniques for comprehending complicated models and their role in incorporating AI in healthcare. The results indicate that although there are limitations to current explainability methods, particularly for clinical use, both global and local explanation models offer a glimpse into evaluating the model and can be used to enhance or compare models.

 Aim: Machine learning tools have various applications in healthcare. However, the implementation of developed models is still limited because of various challenges. One of the most important problems is the lack of explainability of machine learning models. Explainability refers to the capacity to reveal the reasoning and logic behind the decisions made by AI systems, making it straightforward for human users to understand the process and how the system arrived at a specific outcome. The study aimed to compare the performance of different model-agnostic explanation methods using two different ML models created for HbA1c classification. Material and Method: The H2O AutoML engine was used for the development of two ML models Gradient boosting machine GBM and default random forests DRF using 3,036 records from NHANES open data set. Both global and local model-agnostic explanation methods, including performance metrics, feature important analysis and Partial dependence, Breakdown and Shapley additive explanation plots were utilized for the developed models. Results: While both GBM and DRF models have similar performance metrics, such as mean per class error and area under the receiver operating characteristic curve, they had slightly different variable importance. Local explainability methods also showed different contributions to the features. Conclusion: This study evaluated the significance of explainable machine learning techniques for comprehending complicated models and their role in incorporating AI in healthcare. The results indicate that although there are limitations to current explainability methods, particularly for clinical use, both global and local explanation models offer a glimpse into evaluating the model and can be used to enhance or compare models.

 Aim: Machine learning tools have various applications in healthcare. However, the implementation of developed models is still limited because of various challenges. One of the most important problems is the lack of explainability of machine learning models. Explainability refers to the capacity to reveal the reasoning and logic behind the decisions made by AI systems, making it straightforward for human users to understand the process and how the system arrived at a specific outcome. The study aimed to compare the performance of different model-agnostic explanation methods using two different ML models created for HbA1c classification. Material and Method: The H2O AutoML engine was used for the development of two ML models Gradient boosting machine GBM and default random forests DRF using 3,036 records from NHANES open data set. Both global and local model-agnostic explanation methods, including performance metrics, feature important analysis and Partial dependence, Breakdown and Shapley additive explanation plots were utilized for the developed models. Results: While both GBM and DRF models have similar performance metrics, such as mean per class error and area under the receiver operating characteristic curve, they had slightly different variable importance. Local explainability methods also showed different contributions to the features. Conclusion: This study evaluated the significance of explainable machine learning techniques for comprehending complicated models and their role in incorporating AI in healthcare. The results indicate that although there are limitations to current explainability methods, particularly for clinical use, both global and local explanation models offer a glimpse into evaluating the model and can be used to enhance or compare models.

 GirişAmaç Makine öğrenimi araçlarının sağlık alanında birçok uygulamaya sahiptir. Ancak, geliştirilen modellerin uygulanması hala çeşitli zorluklar nedeniyle sınırlıdır. Bu konuda en önemli sorunlardan biri makine öğrenimi modellerinin açıklanabilirliğinin eksikliğidir. Açıklanabilirlik, yapay zeka sistemlerinin karar verme sürecinin nedenlerini ve mantığını ortaya koyma kapasitesini ifade eder ve, kullanıcılar için sürecin nasıl anlaşılacağını ve sistemin nasıl belirli bir sonuca ulaştığını açık hale getirir. Çalışma, HbA1c sınıflandırması için iki farklı ML modeli kullanarak farklı model-agnostik açıklama yöntemlerinin performansını karşılaştırmayı amaçlamaktadır. Yöntemler HbA1c sınıflandırması için iki ML modeli Gradient boosting machine GBM ve default random forests DRF H2O AutoML motoru kullanılarak 3,036 kayıt içeren NHANES açık veri kümesi kullanılarak geliştirilmiştir. Geliştirilen modeller için, performans metrikleri, özellik parametre analizi ve kısmi bağımlılık, kesit ayrıştırma ve Shapley açıklama grafikleri gibi global ve yerel model-agnostik açıklama yöntemleri kullanılmıştır. Sonuçlar GBM ve DRF modelleri benzer performans metriklerine sahip olmasına rağmen, parametre öneminde hafif farklılıklar vardı. Yerel açıklanabilirlik yöntemleri de özelliklere farklı katkılar gösterdi. Sonuç Bu çalışmada, sağlık alanında yapay zekâ entegrasyonu ve modellerin anlaşılmasında açıklanabilir makine öğrenimi tekniklerinin önemini değerlendirilmiştir. Sonuçlar, mevcut açıklanabilirlik yöntemlerinin sınırlılığına rağmen hem global hem de yerel açıklama modellerinin makine öğrenmesi modellerini değerlendirmek için bir fikir verdiğini ve modeli geliştirmek veya karşılaştırmak için kullanabileceğini göstermektedir.

Article Details

How to explain a machine learning model: HbA1c classification example

Authors:

Deniz Topcu

Publisher:

MediHealth Academy Yayıncılık

Main Topics:

Sağlık Kurumları Yönetimi

Journal:

Journal of Medicine and Palliative Care Vol.4 Issue 2, pp.117-125

Document Type:

Article

Publication Date:

3/2023

Document Language:

English

Document ID (Org No.):

2001286136

pISSN:

2717-7505

Machine Learning

Explainable artificial intelligence

Glycated Hemoglobin

Makine öğrenmesi

açıklanabilir yapay zekâ

glikolize hemoglobin

Issues

Article Details

How to explain a machine learning model: HbA1c classification example

Authors: Deniz Topcu

Publisher: MediHealth Academy Yayıncılık

Main Topics: Sağlık Kurumları Yönetimi

Journal: Journal of Medicine and Palliative Care Vol.4 Issue 2, pp.117-125

Document Type: Article

Publication Date: 3/2023

Document Language: English

Document ID (Org No.): 2001286136

pISSN: 2717-7505

Machine Learning

Explainable artificial intelligence

Glycated Hemoglobin

Makine öğrenmesi

açıklanabilir yapay zekâ

glikolize hemoglobin

Issues

Authors:

Deniz Topcu

Publisher:

MediHealth Academy Yayıncılık

Main Topics:

Sağlık Kurumları Yönetimi

Journal:

Journal of Medicine and Palliative Care Vol.4 Issue 2, pp.117-125

Document Type:

Article

Publication Date:

3/2023

Document Language:

English

Document ID (Org No.):

2001286136

pISSN:

2717-7505